Paper: Adversarial Data Robustness via Implicit Neural Representation (ICLR-2024)

Reviewer comment (Writing):
Since data-level robustness is a new area, the meaning about robustness designed with spatial coordinates is not clear.

Author response:
Robustness is not directly related to spatial coordinates. The spatial coordinates are used as the input of the INR to output the corresponding RGB values. Our method allows for the incorporation of the classifier's relevant information into the parameters of the INR to force the INR to reach a robust state.

Reviewer comment (Theory):
The explanation about why the robust data can only be created via adversarial training in Section 3.1 is not convincing enough.

Author response:
Thank you for your suggestion. We think adversarial training could be an effective method for robust data generation. So, we change this sentence into:

"Considering the effectiveness of adversarial training in countering adversarial attacks, it could also be a valuable approach for generating robust data."

Reviewer comment (Writing):
Since data-level robustness is a new area, the meaning about robustness designed with spatial coordinates is not clear.

Author response:
Robustness is not directly related to spatial coordinates. The spatial coordinates are used as the input of the INR to output the corresponding RGB values. Our method allows for the incorporation of the classifier's relevant information into the parameters of the INR to force the INR to reach a robust state.

Reviewer comment (Theory):
The explanation about why the robust data can only be created via adversarial training in Section 3.1 is not convincing enough.

Author response:
Thank you for your suggestion. We think adversarial training could be an effective method for robust data generation. So, we change this sentence into:

"Considering the effectiveness of adversarial training in countering adversarial attacks, it could also be a valuable approach for generating robust data."

Reviewer comment (Reproducibility):
**(Method)** The paper introduces the concept of storing data as INRs for adversarial robustness, but it lacks clarity on how these representations can be derived for testing data. Given that testing data is typically only accessible during the testing phase, the paper should address the challenge of generating specific INRs for each testing image. For now, I would treat INR as a mere image generator.

Author response:
As we said in our submission, even for a new image, we only need to represent it to its corresponding INR format by incorporating the classifier property. Thus, in our experiments, for each classifier, we directly transfer their corresponding testing data into the INR format by incorporating the classifier property. Then, we can achieve the envisioned robustness.

Reviewer comment (Experiments):
**(Experiment)** The experimental design in the paper appears to lack fairness, which fails to prove the efficacy of the proposed method. Also, it is unclear if the proposed framework could be used to defend against unseen attacks (e.g., [1]).

Author response:
We propose to incorporate the classifier's related information into the data format. Therefore, we should train INR for each image individually. We spend time on INR creation, but not on training models adversarially. Therefore, it is hard to compare the efficacy with other methods simply.

Reviewer comment (Theory):
Please provide a more in-depth justification for the choice of INR as a data representation method for adversarial robustness.

Author response:
We just utilize the distinguished overfitting property of INR [1] to incorporate the classifier property into the data format. Then, we can achieve higher robustness.  We can also consider image pixels as learnable parameters and optimize these learnable parameters adversarially. However, such a straightforward solution significantly undermines image quality since the pixel values are directly manipulated. 

[1] Strümpler, Yannick, et al. "Implicit neural representations for image compression." ECCV. 2022.

Reviewer comment (Experiments):
In the proposed setup, how can users derive INR representations for testing data, considering that in adversarial training, testing data is typically not accessible during training?

Author response:
Even for a totally new image, we only need to overfit it to its corresponding INR format by considering the classifier property. Thus, in our experiments, for each classifier, we directly transfer their corresponding testing data into the INR format by incorporating the classifier property. Then, we can achieve the envisioned robustness.

Reviewer comment (Experiments):
Are there any potential biases or confounding factors in the experimental setup that need to be addressed?

Author response:
We have emphasized in the article that due to the distinct nature of the envisioned scenario from traditional adversarial defense, our experimental setup differs accordingly. We obtain robust data representations for each image through defense-in-creation, which can withstand attacks during both creation and transmission. Similar to other INR methods, we do not specifically require a training dataset, which sets it apart from the traditional scenario.

Reviewer comment (Experiments):
Can the proposed framework be used to defend unseen attacks?

Author response:
Yes. In our experiment setting, we use 10-step PGD for INR training, while we test FGSM, 100-step PGD, and CW attacks for evaluation. We present the results of the experiments in Table 2.

Reviewer comment (Evaluation):
There are two extra computations introduced by the proposed method: first is the optimization process of $f\_{\theta}$, which is required to be optimized for each input image (i.e., cannot be amortized); second is the defense-in-creation process of Eq (4), which requires adversarial training by perturbing model parameters $\theta$. The authors should report the empirical cost (e.g., computational time) for these two operations.

Author response:
The time for the first process is 7.12 seconds. The time for the second process is 175.50 seconds when no optimization for computational costs is considered. Besides, our setup does not need any time for training an adversarially robust downstream network model like previous methods. Since conventional adversarial
training methods like PGD-AT take 3-30 times longer than standard training [1], the time consumption of our method is also acceptable.

[1] Bai, Tao, et al. "Recent Advances in Adversarial Training for Adversarial Robustness." IJCAI 2021

Reviewer comment (Evaluation):
The considered attacking methods such as PGD/DPGD, CW and FGSM are not strong. The authors should evaluate their methods under strong attacks like AutoAttack[3] and compare with the state-of-the-art models listed on RobustBench[4].

Author response:
The AutoAttack has not been specifically studied for INR. We conducted experiments by adapting AutoAttack for images during INR creation, and the results show that our defense-in-creation can reach high accuracy. For attack during transmission, due to the data being represented as INR, it is not feasible to directly apply AutoAttack to the INR parameters.

Reviewer comment (Evaluation):
From my experience, the proposed method (with 92.51% accuracy against PGD) probably has gradient obfuscation [1,2]. The authors should evaluate their method under adaptive attacks that involving the defense mechanism (i.e., INR mechanism), as well as strong off-the-shelf attacks such as AutoAttack.

Author response:
As outlined in our paper, our method focuses on enhancing data-level adversarial robustness, and its motivation differs from traditional adversarial training approaches. In our envisioned scenario, the creators of INR, during the defense-in-creation process, are already aware of how the data will be used in a downstream network (such as certain fixed large-scale model architectures). In this case, their objective is to perform adversarial training on each INR, taking into account the knowledge of the downstream task network. This approach enables us to achieve an accuracy of 92.51%. During this process, gradient information is not obfuscated.

Regarding adaptive attack during creation, since our defense-in-creation can adaptively eliminate adversarial perturbations present in the original image, we only consider adaptive attack during transmission. We employ PGD to apply perturbations to the reconstructed image and then fine-tune the resulting image with adversarial perturbations to obtain the final INR sample. Under this attack scenario, the accuracy on the CIFAR-10 dataset decreases to 84.47\% for 200-iteration fine-tuning and 2.00\% for 500-iteration fine-tuning.

Reviewer comment (Novelty):
The paper starts its discussion by arguing that adversarial training is hard to deploy. The paper reads: "_Despite its effectiveness, adversarial training requires that users possess a detailed understanding of training settings._" In my view, this argument can even be used for using vanilla neural networks. One can make the same argument that even for using/training regular neural networks users need a detailed understanding of model architecture, optimization process, etc. There is no major differences that separates adversarial training from vanilla training, and I feel that motivating the core problem around such arguments is weak.

Author response:
We want to emphasize that adversarial training must consider potential attacks during model training or finetune the specific model to achieve adversarial robustness. However, when large-scale foundational model has become more popular today. Retraining specific models may become more difficult or even impossible [1]. In this situation, those model creators may define a specific data format suitable for their large-scale foundational models. Then, rather than directly retraining the model for adversarial robustness, our data robustness provides a more versatile solution. Besides, our experiments demonstrates that this unique robust data is able to achieve high robustness for specific models.

[1] Hoffmann, Jordan, et al. "An empirical analysis of compute-optimal large language model training." Advances in Neural Information Processing Systems 35 (2022): 30016-30030.

Reviewer comment (Experiments):
More importantly, the threat model is not intuitive. In particular, the paper assumes that one uses the INRs to encode the data, then send them to a model, which again decodes the INR to query a classifier. Why this process is efficient? Why the user doesn't send their data directly? What is special about this threat model? I believe that the proposed threat model is not making intuitive sense, and it requires a better design. For instance, using NeRF to encode a scene and then transmitting it would have made a much better threat model as NeRFs are encoding multiple scenes in one representation. However, using the current threat model for 2D images seems less intuitive and might not make any practical sense.

Author response:
This process may undermine the efficiency at the data side. However, it eliminates the need for time-consuming retraining or fine-tuning of the specific model. By incorporating the classifiers during the creation of INR, our method has achieved higher robustness for specific models in the current setting. 

If we directly send images, such incorporation can significantly undermine the image quality as shown in the "Pixel manipulation" experiment. 

**Response:** INR can encode the data into network parameters. By representing the data into INR, we are able to easily incorporate information of specific classifier into INR. Then, such unique data representation is able to achieve high robustness for specific models. 

Since we utilize INR for data representation, we need to consider perturbations to the INR parameters in the threat model.

**Response:** NeRF is designed to encode a single, specific scene, utilizing multiple images from that scene for training. It is not capable of encoding multiple scenes within a single model. NeRF is also inapproproate for the tasks in our scenario requiring the representation of individual images. Besides, the use of INR for 2D image storage and transmission has been extensively studied. For example, [1] proposed that INR can achieve super-resolution image storage. [2] [3] considered using INR for image compression and transmission. [4] considered representing images as INR for neural network training. [5] implemented INR representation on the ImageNet dataset. Therefore, adopting INR as a cornerstone is practical and feasible.

[1] Nguyen, Quan H., and William J. Beksi. "Single image super-resolution via a dual interactive implicit neural network." Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. 2023.

[2] Strümpler, Yannick, et al. "Implicit neural representations for image compression." European Conference on Computer Vision. 2022.

[3] Dupont, Emilien, et al. "COIN: COmpression with Implicit Neural representations." Neural Compression: From Information Theory to Applications--Workshop@ ICLR 2021. 2021.

[4] Rusu, Andrei A., et al. "Hindering Adversarial Attacks with Implicit Neural Representations." International Conference on Machine Learning. PMLR, 2022.

[5] Singh, Rajhans, Ankita Shukla, and Pavan Turaga. "Polynomial Implicit Neural Representations For Large Diverse Datasets." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023.

Reviewer comment (Experiments):
The empirical evaluations are also limited. The paper only uses small scale datasets such as CIFAR and SVHN. It would be crucial to see how the proposed attacks and defense work in more large scale datasets such as ImageNet. Given that there are many pre-trained models available for the ImageNet dataset online, I believe that such evaluations would be feasible.

Author response:
We further conduct experiment on ImageNet. We used pre-trained weights from the publicly available torchvision.models package, specifically the 'ResNet50_Weights.IMAGENET1K_V2'. The result on ImageNet is shown as below. Our method keeps robust performance. 

|                     | DPGD    | PGD for INR | CW for INR |
|---------------------|---------|-------------|------------|
|   Normal training   | 12.25\% | 12.88\%     | 1.88\%     |
| Defense-in-creation | 98.00\% | 99.87\%     | 83.75\%    |

Reviewer comment (Writing):
Please clarify what do you mean by arguments like "_our setup is rooted in the reality that many model users often fail to understand the settings associated with adversarial training._"

Author response:
In our paper, our goal is to transfer data into robust form for publicly available models (they could be large-scale pretrained models in the future). Since it is difficult to retrain all these models adversarially, these models may lack the ability to defend against adversarial attacks. Therefore, traditional data form can be easily perturbated to fool these models. Consequently, we encode the data into INR parameters incorporating specific classifier information for robust data representation. We will change this sentence to:

"*Our setup is rooted in the reality that many publicly available models lack the ability to defend against adversarial attacks, and training them adversarially is difficult.*"

Reviewer comment (Theory):
Given that $\ell_p$ norms are used to enforce visual image similarity in the image domain, its use for other domains such as INR weights seems less intuitive. What are other alternatives for the similarity in the weight space that could have been used? In other words, are there any other alternatives to the $\ell_p$ norm used in Eq. (3)?

Author response:
Using $\ell\_p$ norm in parameter space between networks is a common setting [1]. In our experiments, to maintain the efficiency and simplicity of the whole framework, we directly use the $\ell\_p$ norm, which has already shown promising performance. We agree that there is no consensus on the definition of network similarity [2]. In the future, we will explore other methods to measure weights similarity for better performance.

[1] Bansal, Arpit, et al. "Certified neural network watermarks with randomized smoothing." International Conference on Machine Learning. PMLR, 2022.

[2] Kornblith, Simon, et al. "Similarity of neural network representations revisited." International conference on machine learning. PMLR, 2019.

Reviewer comment (Theory):
It would be nice to add a few more sentences to the last paragraph of Section 3.2 explaining why projecting the gradient of the image space would help with having a better image quality. In other words, what motivated this step?

Author response:
In PGD attacks against images, the perturbation is projected to a specified pixel range in each iteration. However, attacks on INR do not directly manipulate the image. If only the fluctuation range of INR parameters is considered, the image decoded from the perturbed INR could be severely damaged. Therefore, we consider adding constraints directly on the image level for the gradients. The detailed analysis is as follows. The loss function is defined as $ J(\theta) = \mathcal{L}\_{CE}(g\_{\phi}(\hat{I}), y) $. Here, $ \hat{I} = \Psi(f\_\theta) $ represents the image reconstructed by INR $ f\_\theta $. The original input's INR weight is $ \theta\_{org} $, and the image it reconstructs is $ \hat{I}\_{org} $. In the $ t $-th iteration, the image reconstructed by the INR with weights $ \theta\_{t-1} $ before the attack is $ \hat{I}\_{t-1} $, and the loss is $ J(\theta\_{t-1}) $. The gradient of the loss at $ \theta\_{t-1} $ is calculated as $ \nabla\_{\theta\_{t-1}} = \frac{\partial J}{\partial \theta\_{t-1}} $. Updating against the gradient direction, we get $ \theta\_{t} = \theta\_{t-1} + \alpha \frac{\partial J}{\partial \theta\_{t-1}} $. At this point, the new loss $ J' $ can be estimated as:


$$
J(\theta\_{t}) \approx J(\theta\_{t-1}) + (\theta\_{t}-\theta\_{t-1})\frac{\partial J}{\partial \theta\_{t-1}}
$$
$$
 = J(\theta\_{t-1}) + \alpha (\frac{\partial J}{\partial \theta\_{t-1}})^2.
$$
Since the second term in the equation above is always positive, it allows for updates in a direction that increases the loss gradually. Now, we consider the impact of weight updates on the reconstructed image. The reconstructed image can be approximated as
$$
\hat{I}\_{t} \approx \hat{I}\_{t-1} + (\theta\_{t}-\theta\_{t-1})\frac{\partial \hat{I}}{\partial \theta\_{t-1}}
$$
$$
=\hat{I}\_{t-1} + \alpha \frac{\partial J}{\partial \theta\_{t-1}}\frac{\partial \hat{I}}{\partial \theta\_{t-1}}
$$
$$
=\hat{I}\_{t-1} + \alpha \frac{\partial J}{\partial \hat{I}} \frac{\partial \hat{I}}{\partial \theta\_{t-1}} \frac{\partial \hat{I}}{\partial \theta\_{t-1}}
$$
$$
= \hat{I}\_{t-1} + \alpha (\frac{\partial \hat{I}}{\partial \theta\_{t-1}})^2 \frac{\partial J}{\partial \hat{I}}.
$$
To prevent significant differences between the reconstructed image and the original image, we project $\hat{I}{t}$ onto the $\ell\_p$ ball around the original reconstructed image $\hat{I}{org}$. Therefore, the following constraint is added:
$$
||\hat{I}\_{t}-\hat{I}\_{\mathrm{org}}||\_p \leq \zeta.
$$
That is
$$
||\hat{I}\_{t-1} + \alpha (\frac{\partial \hat{I}}{\partial \theta\_{t-1}})^2 \frac{\partial J}{\partial \hat{I}}-\hat{I}\_{\mathrm{org}}||\_p \leq \zeta.
$$
Since $\alpha (\frac{\partial \hat{I}}{\partial \theta\_{t-1}})^2$ is always positive, for ease of computation, we simplify it as $\alpha (\frac{\partial \hat{I}}{\partial \theta\_{t-1}})^2=1$, which leads to line 7 in Algorithm 2.

We will add an explanation to the last paragraph of Section 3.2 in the revised paper.

Reviewer comment (Evaluation):
What is the training time difference between finding an INR using Eq. (1) versus the defense in creation in Eq. (4)?

Author response:
With our experimental setting, the training time using Eq. (1) is 7.12 seconds, while the training time using Eq. (4) is 175.50 seconds. Since conventional adversarial training methods like PGD-AT take 3-30 times longer than standard training [1], the time consumption of our method is also acceptable.

[1] Bai, Tao, et al. "Recent Advances in Adversarial Training for Adversarial Robustness." IJCAI 2021

Reviewer comment (Experiments):
Did the paper also test the transferability of the defense in creation? In other words, what happens if we find the robust INRs for a classifier $g\_{\boldsymbol{\phi}}^{(1)}$ while trying to defend another model $g\_{\boldsymbol{\phi}}^{(2)}$ during inference?

Author response:
Our method is model-specific, which means that we aim at producing robust data with high robustness for specific models. Thus, the transferability is beyond our setup. For example, we train our robust INRs for a classifier using PreActResNet-18 architecture, and test on WideResNet34-10 architecture. The accuracy under CW attack is 22.67\%, higher than normal training INR (12.80\%), but much lower than accuracy using the same PreActResNet-18 architecture. However, the users can train with different models simutenously when generating robust INRs. Then, the robust data can have robust performance on those specific models incorporated during INR generation.

Reviewer comment (Experiments):
Run experiments on large scale datasets such as ImageNet-1k or ImageNet-100.

Author response:
We have addressed this in the above experiments.

Reviewer comment (Experiments):
How do you specify the upper-bound for adversarial attack against INR weights? In other words, what makes a good $\delta$ for Eq. (3)? Because using $\ell_p$ norm in the INR weight space is not intuitive.

Author response:
As said in our paper, we evaluate the attack using two criteria: one is invisibility of perturbations in the reconstructed image, and the other is the attacking efficacy of the perturbations. We believe that a good $\delta$ should result in an INR-reconstructed image that is as consistent as possible with the original image while being able to fool the downstream applications. In our experiments, we tested different values of $\delta$, and the results are shown in Figure 4 (right). The horizontal axis represents the PSNR between the reconstructed image and the original image, while the vertical axis shows the accuracy after the attack. A good $\delta$ should strike a balance between invisibility and attacking efficacy.

Reviewer comment (Evaluation):
In my view, using attack success rate (ASR) would be a better measure when trying to evaluate attacks. Using accuracy as the current version makes it difficult to interpret the results.

Author response:
Thank you for your suggestion. We will use attack success rate (ASR) for evaluating attacks. We will modify the first two rows in Table 1 as

| Method | FGSM on images | Creation after FGSM | PGD on images | Creation after PGD | CW on images | Creation after CW |
|--------|----------------|---------------------|---------------|--------------------|--------------|-------------------|
| ASR    | 60.45\%        | 59.26\%             | 100.00\%      | 99.83\%            | 99.93\%      | 99.84\%           |

We will also make revisions to Figure 4 in our paper.

Reviewer comment (Evaluation):
How is it possible that the natural accuracies in Table 2 are 100%?

Author response:
In our setup, by incorporating the specifical classifier during the creation of INR, we create this envisioned data format that is able to achieve envisioned high data robustness on specific models Thus, they can reach $100\%$. You can see more details from the demo in our supplementary. Since our setting differs from the traditional approach, to avoid ambiguity, we will changed the "Natural" to "Accuracies without attacks."

Reviewer comment (Presentation):
Use a larger font size for the tables.

Author response:
Thanks for your suggestion. We will use a larger font size in our revised paper.

Reviewer comment (Theory):
The proposed threat model makes no practical sense. The paper assumes that INRs instead of the query images are sent over the communication channels.

Author response:
This process may undermine the efficiency at the data side. However, it eliminates the need for time-consuming retraining or fine-tuning of the specific model. By incorporating the classifiers during the creation of INR, our method has achieved higher robustness for specific models in the current setting. 

If we directly send images, such incorporation can significantly undermine the image quality as shown in the "Pixel manipulation" experiment. 

**Response:** INR can encode the data into network parameters. By representing the data into INR, we are able to easily incorporate information of specific classifier into INR. Then, such unique data representation is able to achieve high robustness for specific models. 

Since we utilize INR for data representation, we need to consider perturbations to the INR parameters in the threat model.

**Response:** NeRF is designed to encode a single, specific scene, utilizing multiple images from that scene for training. It is not capable of encoding multiple scenes within a single model. NeRF is also inapproproate for the tasks in our scenario requiring the representation of individual images. Besides, the use of INR for 2D image storage and transmission has been extensively studied. For example, [1] proposed that INR can achieve super-resolution image storage. [2] [3] considered using INR for image compression and transmission. [4] considered representing images as INR for neural network training. [5] implemented INR representation on the ImageNet dataset. Therefore, adopting INR as a cornerstone is practical and feasible.

[1] Nguyen, Quan H., and William J. Beksi. "Single image super-resolution via a dual interactive implicit neural network." Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. 2023.

[2] Strümpler, Yannick, et al. "Implicit neural representations for image compression." European Conference on Computer Vision. 2022.

[3] Dupont, Emilien, et al. "COIN: COmpression with Implicit Neural representations." Neural Compression: From Information Theory to Applications--Workshop@ ICLR 2021. 2021.

[4] Rusu, Andrei A., et al. "Hindering Adversarial Attacks with Implicit Neural Representations." International Conference on Machine Learning. PMLR, 2022.

[5] Singh, Rajhans, Ankita Shukla, and Pavan Turaga. "Polynomial Implicit Neural Representations For Large Diverse Datasets." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023.