Paper: Towards robust unlearnable examples via deep hiding (ICLR-2024)

Reviewer comment (Novelty):
The method consists of three modules: The Deep Hiding Scheme, which introduces the concept of Image Hiding, an existing work; the Semantic Image Generation Module, which utilizes the existing ControlNet; and the Latent Feature Concentration Module, which is very similar to the idea of EntF mentioned in the related work. In summary, the ideas presented in this paper are intriguing, but the innovation is insufficient.

Author response:
**Ans:** Thank you for the comments. We would like to clarify that our Latent Feature Concentration module (LFC) differs from EntF in various aspects. Firstly, EntF aims to generate entangled features associated with perturbed data, causing confusion in models; while our module encourages the concentration of the perturbation itself, creating stronger shortcuts. Secondly, EntF focuses on robustness against adversarial training, while we use LFC to achieve more general robustness. Besides, we would like to emphasize our novelty in semantic-based unlearnable examples. The proposed modules (i.e., Semantic Image Generation (SIG) and LFC) all combine to increase the inter-class distance and reduce the intra-class variance, enhancing the general robustness of unlearnable examples, shown in Table 4.




**Table 4: Ablation studies on CIFAR10 for designed Latent Feature Concentration module (LFC), and Semantic Images Generation module (SIG), including Text Prompts Clustering (TPC) and Stable Diffusion model and ControlNet (SD+C).**


| Setting | LFC | SIG: TPC | SIG: SD+C | Vanilla | Cutout | Cutmix | Mixup | MeanF | MedianF | BDR   | Gray  | GaussN | GaussF | JPEG10 | JPEG50 | AT    | Mean  |
|:-------:|:---:|:--------:|:---------:|:-------:|:------:|:------:|:-----:|:-----:|:-------:|:-----:|:-----:|:------:|:------:|:------:|:------:|:-----:|:-----:|
| Sample-wise|   ×  |       ×   |        ×   | 94.08   | 94.52  | 94.07  | 88.23 | 65.42 | 87.04   | 89.46 | 91.45 | 88.36  | 94.07  | 83.29  | 89.97  | 86.26 | 88.17 |
|    Sample-wise     | ×    | ✓        | ✓         | 14.77   | 22.20  | 13.06  | 23.44 | 30.18 | 51.43   | 35.66 | 17.31 | 37.50  | 15.80  | 81.97  | 81.48  | 81.24 | 38.93 |
|  Sample-wise       | ✓   |    ×       | ✓         | 10.00   | 16.53  | 20.81  | 17.14 | 18.51 | 21.73   | 24.98 | 13.85 | 22.07  | 10.59  | 80.09  | 82.90  | 46.54 | 29.67 |
|    Sample-wise     | ✓   |    ×       |    ×        | 94.09   | 94.34  | 93.84  | 94.00 | 64.55 | 85.94   | 88.86 | 91.00 | 88.70  | 94.20  | 82.84  | 90.31  | 85.31 | 88.38 |
|     Sample-wise    | ✓   | ✓        | ✓         | 15.36   | 10.79  | 10.00  | 14.72 | 17.68 | 17.00   | 21.12 | 17.61 | 22.78  | 11.16  | 80.41  | 81.03  | 38.31 | **27.54** |
| Class-wise |  ×    |     ×      |    ×        | 12.16   | 11.02  | 11.43  | 10.81 | 10.14 | 13.28   | 10.10 | 12.12 | 10.00  | 10.00  | 78.21  | 15.32  | 12.87 | 16.73 |
|    Class-wise     |  ×    | ✓        | ✓         | 10.00   | 10.00  | 10.06  | 10.82 | 13.26 | 25.76   | 15.83 | 14.20 | 10.03  | 10.00  | 76.26  | 20.28  | 14.78 | 18.56 |
|    Class-wise     | ✓   |      ×     | ✓         | 10.00   | 10.00  | 10.00  | 12.32 | 9.72  | 10.00   | 10.00 | 10.59 | 10.00  | 10.00  | 71.44  | 30.27  | 10.00 | 16.49 |
|    Class-wise     | ✓   |     ×      |    ×        | 10.00   | 10.00  | 11.41  | 10.00 | 10.01 | 11.71   | 10.00 | 10.01 | 10.00  | 10.00  | 58.38  | 10.05  | 10.00 | **13.97** |
|    Class-wise     | ✓   | ✓        | ✓         | 10.00   | 10.00  | 11.25  | 10.02 | 10.59 | 10.04   | 13.53 | 10.00 | 10.00  | 10.00  | 72.97  | 23.62  | 10.00 | 16.31 |

Reviewer comment (Novelty):
Previous research has shown that some Unlearnable Examples possess an inherent resistance to data augmentation, which contradicts the paper's claim that prior methods have overlooked these issues.

Author response:
**Ans:** We agree that previous works have shown resistance to data augmentations to some extent. However, the recent work ISS[G] found that these methods are extremely vulnerable to simple data processing like Grayscaling and JPEG compression with low quality factors. We follow the protocol in ISS and gain insights from the results in Table 1:
- The existing methods are not generally robust against different types of data augmentations and processing. For example, though EM and REM are resistant to popular data augmentations like cutout and mixup, they are very vulnerable to grayscaling and JPEG compression.
- The existing methods are not resistant to the countermeasures with higher-level severity. We find most of the existing methods will be mitigated once the quality factor of JPEG decreases to 10.
As a result, we argue that the resistance of UEs against countermeasures is not generally solved. We aim to improve general robustness with the deep hiding technique in this paper.

```
[G] Zhuoran Liu, Zhengyu Zhao, and Martha A. Larson. "Image Shortcut Squeezing: Countering Perturbative Availability Poisons with Compression." In International Conference on Machine Learning, 2023.
```

Reviewer comment (Experiments):
In the experimental settings of the paper, the perturbation for adversarial training is set to $8/255$, and the perturbation for protection noise also appears to be set to $8/255$ according to the experimental table. Under this setting, which is consistent with EntF, the paper's experiments lack a comparison with this method.

Author response:
**Ans:** Thank you for suggesting a comparison with EntF. EntF mainly focuses on robustness against adversarial training, and its performance on other countermeasures is relatively poor. Hence, we didn't include its comparison in the initial paper. In the modified version, we have included the results of EntF on CIFAR10, CIFAR100 in Table 1. Besides, we study its transferability across architectures and its effectiveness in clean-poison mixed settings. The results show that EntF remains robust against adversarial training, demonstrating competitive performance among the surrogate-dependent UEs including EM, REM, TAP. However, its general robustness is not as good as other methods.

Reviewer comment (Evaluation):
The paper doesn't explain why it's also effective under adversarial training.

Author response:
**Ans:** Our proposed method hides semantic-based perturbation, instead of gradient-based perturbation. We hypothesize they fall in different latent spaces, so our perturbation cannot be erased by adversarial training, similar to OPS [H]. We added the explanation in Section 4.3.

```
[H] Wu S, Chen S, Xie C, et al. One-pixel shortcut: on the learning preference of deep neural networks. ICLR, 2023.
```

Reviewer comment (Novelty):
Although the introduction of Image hiding in the paper is interesting, the three important parts of the article are existing work and lack innovation. In addition, there is a lack of comparison of EntF methods, a lack of inquiry about defense against training, and a lack of inquiry about the effect of the generated hiding image on protection.

Author response:
**Ans:** Thank you for your summary. We respond to the first 3 points in the previous questions. For the inquiry about the effect of generated hiding images, we have conducted more comprehensive ablation studies to show the effect of each component. We show the complete result in the modified Table 4 and Table D in the supplementary. The results show that each component can contribute to the unlearnability. We hope the response can address your concerns well.

Reviewer comment (Experiments):
An ablation experiment on the semantic image generation module is missing to demonstrate its advantages compared with randomly selected hidden images.

Author response:
**Q2:** An ablation experiment on the semantic image generation module is missing to demonstrate its advantages compared with randomly selected hidden images.

**Ans:** Thank you for the comments. We conduct a comprehensive ablation study on the effectiveness of the semantic image generation module. We show the results in Table 4 and we have added it into the paper. In the class-wise setting, we find that the improvement of the generation module is marginal; however, in the sample-wise setting, the semantic generation module (SIG) can degrade the mean accuracy from 88% to 38%. It is further reduced to 27.5% with latent feature concentration (LFC). When we disentangle the text prompt clustering (TPC) and Stable Diffusion+ControlNet generation (SD+C), we find that SD+C contributes the most and TPC contributes around a 2% reduction. The ablation study shows that each component in our proposed method plays an important role in the generally robust unlearnable examples.

**Table 4: Ablation studies on CIFAR10 for designed Latent Feature Concentration module (LFC), and Semantic Images Generation module (SIG), including Text Prompts Clustering (TPC) and Stable Diffusion model and ControlNet (SD+C).**

| Setting | LFC | SIG: TPC | SIG: SD+C | Vanilla | Cutout | Cutmix | Mixup | MeanF | MedianF | BDR   | Gray  | GaussN | GaussF | JPEG10 | JPEG50 | AT    | Mean  |
|:-------:|:---:|:--------:|:---------:|:-------:|:------:|:------:|:-----:|:-----:|:-------:|:-----:|:-----:|:------:|:------:|:------:|:------:|:-----:|:-----:|
| Sample-wise|   ×  |       ×   |        ×   | 94.08   | 94.52  | 94.07  | 88.23 | 65.42 | 87.04   | 89.46 | 91.45 | 88.36  | 94.07  | 83.29  | 89.97  | 86.26 | 88.17 |
|    Sample-wise     | ×    | ✓        | ✓         | 14.77   | 22.20  | 13.06  | 23.44 | 30.18 | 51.43   | 35.66 | 17.31 | 37.50  | 15.80  | 81.97  | 81.48  | 81.24 | 38.93 |
|  Sample-wise       | ✓   |    ×       | ✓         | 10.00   | 16.53  | 20.81  | 17.14 | 18.51 | 21.73   | 24.98 | 13.85 | 22.07  | 10.59  | 80.09  | 82.90  | 46.54 | 29.67 |
|    Sample-wise     | ✓   |    ×       |    ×   | 94.09   | 94.34  | 93.84  | 94.00 | 64.55 | 85.94   | 88.86 | 91.00 | 88.70  | 94.20  | 82.84  | 90.31  | 85.31 | 88.38 |
|     Sample-wise    | ✓   | ✓  | ✓  | 15.36   | 10.79  | 10.00  | 14.72 | 17.68 | 17.00   | 21.12 | 17.61 | 22.78  | 11.16  | 80.41  | 81.03  | 38.31 | **27.54** |
| Class-wise |  ×    |     ×      |    ×        | 12.16   | 11.02  | 11.43  | 10.81 | 10.14 | 13.28   | 10.10 | 12.12 | 10.00  | 10.00  | 78.21  | 15.32  | 12.87 | 16.73 |
|    Class-wise     |  ×    | ✓   | ✓   | 10.00   | 10.00  | 10.06  | 10.82 | 13.26 | 25.76   | 15.83 | 14.20 | 10.03  | 10.00  | 76.26  | 20.28  | 14.78 | 18.56 |
|    Class-wise     | ✓   |  ×     | ✓   | 10.00   | 10.00  | 10.00  | 12.32 | 9.72  | 10.00   | 10.00 | 10.59 | 10.00  | 10.00  | 71.44  | 30.27  | 10.00 | 16.49 |
|    Class-wise     | ✓   |  ×  |    ×  | 10.00   | 10.00  | 11.41  | 10.00 | 10.01 | 11.71   | 10.00 | 10.01 | 10.00  | 10.00  | 58.38  | 10.05  | 10.00 | **13.97** |
|    Class-wise     | ✓   | ✓ | ✓  | 10.00   | 10.00  | 11.25  | 10.02 | 10.59 | 10.04   | 13.53 | 10.00 | 10.00  | 10.00  | 72.97  | 23.62  | 10.00 | 16.31 |

Reviewer comment (Theory):
There is a backward revealing process for the INN-based hiding model used in the paper, but what is the significance of the existence of this process?

Author response:
**Q3:** There is a backward revealing process for the INN-based hiding model used in the paper, but what is the significance of the existence of this process?

**Ans:** The backward revealing process is necessary to ensure that the semantic image is successfully hidden into the clean images.
In the case of without revealing process, the network tends to embed nothing for satisfying the invisibility.
In practice, during inference when generating UE, we do not require the revealing process but only utilize the hiding process.

Thank you for your prompt response and insightful speculation.

To demonstrate the impact of solely training with perturbation loss (i.e., **$\mathcal{L}_{hide}$**), we conduct relevant experiments. **1.** The training plot (Figure A in the supp) shows that the perturbation loss remains at 0, which indicates there is no optimization in hiding semantic images. **2.** The perturbation maps generated by the hiding model trained on only **$\mathcal{L}_{hide}$** ( Figure B in the supp) show minimal information with no semantic patterns (i.e., black images). **3.** Our evaluation of unlearnable examples generated with only **$\mathcal{L}_{hide}$** reveals that the test accuracy is close to that of clean images (Table. K). This indicates that minimal information is hidden in clean images, leading to ineffective unlearnability. 

**Table. K: Evaluation of the different hiding models trained by solely controlling the hiding loss (**$\mathcal{L}_{\text{hide}}$**) and using our designed loss ($\mathcal{L}_{\text{total}}$). The test accuracy (\%) is evaluated on CIFAR-10 in the class-wise setting.**
| Method | Vanilla | Cutout | Cutmix | Mixup | MeanF | MedianF | BDR | Gray | GaussN | GaussF | JPEG10 | JPEG50 | AT | Mean |
| :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
| Clean | 94.59 | 95.00 | 94.77 | 94.96 | 49.70 | 86.64 | 89.07 | 92.80 | 88.71 | 94.54 | 85.22 | 90.89 | 84.19 | 87.78 |
| only $\mathcal{L}_{\text {hide }}$ | 94.71 | 95.05 | 94.68 | 95.4 | 38.88 | 87.39 | 89.12 | 93.1 | 88.55 | 87.39 | 85.04 | 91.04 | 88.59 | 86.84 |
| ours | 10.00 | 10.00 | 11.25 | 10.02 | 10.59 | 10.04 | 13.53 | 10.00 | 10.00 | 10.00 | 72.97 | 23.62 | 10.00 | **16.31** |

This phenomenon is because only controlling the perturbation loss (i.e., $\mathcal{L}_{hide}$) alone will lead to unstructured minimal perturbation. When solely optimizing the perturbation loss, the network tends to discard the information of the semantic image to meet the requirement of minimizing the difference between the clean image and generated UE rather than hide it. Thus, the revealing loss is important to enforce the network to hide the information of semantic image into the clean image by ensuring the revealing of the hidden information from the UE. 

Based on the experimental results and our analysis, we argue that the revealing process with reveal loss is necessary to hide semantic information to create UEs. We hope it can address your concerns.

Reviewer comment (Reproducibility):
The paper proposed a concentration loss (Eq. 7), and how are sample i and sample j selected in the specific implementation?

Author response:
**Q4:** The paper proposed a concentration loss (Eq. 7), and how are sample i and sample j selected in the specific implementation?

**Ans:** Sample ij refers to two samples within the same minibatch that have the same label. This is because our LFC only operates on pairs of samples with matching labels.

Reviewer comment (Experiments):
Additional Requirements of generating a large dataset of semantic images using paired text prompts and canny edge maps.

Author response:
**Ans:** Thank you for your invaluable comments on the ‘semantic image generation’ module. As we mention in Section 3.2.3, we use the paired text prompt and canny edge maps to control the generated semantic content more precisely, which leads to better robustness of the UEs.
We conduct ablation studies using random natural images, and next-class images, without using paired text prompts and canny edge maps.
Table D demonstrates that without the ControlNet based generation, the performance is still competitive in the class-wise setting while dropping dramatically in the sample-wise setting.
The competitive results in the class-wise setting prove the effectiveness of unlearnability brought from hiding semantic images, highlighting robust generalizability across various countermeasures.
However, simply using next-class images or random natural images in the sample-wise setting fails to achieve unlearnability due to without controlling the similarity of hidden semantic images within intra-class data, which validates the effectiveness of generating a large dataset of semantic images using paired text prompts and canny edge maps.

**Table D: Additional experimental results on CIFAR10 by using different hidden semantic images, including images from the next class, random natural images (CIFAR100), and our semantic image generation module.**

| Setting         | Hidden Semantic Images       | Vanilla | Cutout | Cutmix | Mixup | MeanF | MedianF | BDR   | Gray  | GaussN | GaussF | JPEG10 | JPEG50 | AT    | Mean  |
|:---------------:|:---------------------:|:-------:|:------:|:------:|:-----:|:-----:|:-------:|:-----:|:-----:|:------:|:------:|:------:|:------:|:-----:|:-----:|
| Sample-wise | Next Class            | 80.19   | 75.79  | 74.79  | 71.26 | 60.57 | 82.25   | 72.89 | 67.46 | 72.38  | 78.66  | 82.01  | 87.68  | 82.53 | 76.04 |
|      Sample-wise           | Random Natural Images | 94.09   | 94.34  | 93.84  | 94.17 | 64.55 | 85.94   | 88.86 | 91.81 | 88.70  | 94.20  | 82.84  | 90.31  | 85.31 | 88.38 |
|      Sample-wise           | Ours                  | 15.36   | 10.79  | 10.00  | 14.72 | 17.68 | 17.00   | 21.12 | 17.61 | 22.78  | 11.16  | 80.41  | 81.03  | 38.31 | **27.54** |
|                 |                       |         |        |        |       |       |         |       |       |        |        |        |        |       |       |
| Class-wise  | Next Class            | 10.00   | 10.00  | 9.96   | 10.18 | 10.47 | 10.18   | 10.00 | 10.00 | 10.00  | 10.00  | 62.91  | 17.26  | 10.01 | 14.69 |
|         Class-wise        | Random Natural Images | 10.00   | 10.00  | 11.41  | 10.00 | 10.01 | 11.71   | 10.00 | 10.01 | 10.00  | 10.00  | 58.38  | 10.05  | 10.00 | **13.97** |
|         Class-wise        | Ours                  | 10.00   | 10.00  | 11.25  | 10.02 | 10.59 | 10.04   | 13.53 | 10.00 | 10.00  | 10.00  | 72.97  | 23.62  | 10.00 | 16.31 |

Reviewer comment (Evaluation):
The sample-wise setting may leak information about hidden images. If some hackers know an image is protected in this way, they may find a countermeasure for the unlearnable examples based on the proposed hidden semantic generations.

Author response:
**Ans:** Thank you for your insights. We would like to clarify that we reduce the exposure risk of the sample-wise setting from 2 aspects:

1. We hide different semantic images into the images in each class. Though the semantic images share similar semantics, they are totally different in textures and colors.
2. Our deep hiding model will generate content-dependent perturbations. When we hide the semantic image into the original image, the perturbation will embed into the original content adaptively, leading to more pixel-wise changes.

In data protection and adversarial learning research, we will always face challenges from hackers who wish to mitigate the effects of the perturbations. We are consistently improving the robustness of the defensive perturbations, so the hackers will need more effort to remove them. It is the initiative of our work as well, and we believe the research community is working towards more generally robust unlearnable examples endlessly.

Reviewer comment (Theory):
From the above, the connection between the hidden image and the unlearnable example is unclear. The hidden semantic image may not directly contribute to unlearnability.

Author response:
**Ans:** We agree that it is difficult to give a certain answer about the source of unlearnability, since the image generation and hiding network are still black boxes. We want to reclaim our hypothesis in Section 3.3 that deep hiding semantic images can create shortcuts, leading to unlearnability. Furthermore, based on the ablation results in Table D(**Q1**) and Table 4(**Q3**), we show the hidden semantic image contributes to the unlearnability in the class-wise setting; In the sample-wise setting, the unlearnability mainly comes from the controlled semantic image generation. These results highlight the effectiveness of the hidden semantic image.

Reviewer comment (Experiments):
The robustness and effectiveness may come from the pre-trained ResNet-18, CLIP, Stable Diffusion, or ControlNet. Therefore, the effectiveness and robustness may not come from the architecture and the idea; instead, these may only come from the extra information from the four pre-trained models.

Author response:
**Ans:** Similar to the previous responses, we would like to clarify that the effectiveness of the unlearnable examples comes from the semantic hiding images instead of the pre-trained models, based on the analysis and experimental results.

Reviewer comment (Experiments):
Although an LFC (a pretrained ResNet18) is used to regularize intra-class variance, and the CLIP, Stable Diffusion, and ControlNet are used to maximize inter-class separation, the author should consider the alignment between these pretrained models.

Author response:
**Ans:** We've conducted an ablation study by removing the LFC (Table 4(Q3)) and found that there is a slight decrease on the experimental results.
The reason is that these components serve as auxiliary elements to control the semantic image. Therefore, alignment is not an urgent requirement and does not significantly affect the results.

Reviewer comment (Evaluation):
Although Grad-CAM is used to visualize the attention of DNNs, the intra-class and inter-class relationship with the classification and semantic generation should be more fully exploited. Evaluating the inter-class and intra-class statistics directly could also substantiate the claims around controlled semantics, see questions.

Author response:
**Ans:** We investigate the intra-/inter-class distance in latent features using a trained (for unlearnable examples) or pre-trained (for clean images) ResNet18, shown in Table J. We observe that our unlearnable examples, in both class-wise and sample-wise settings, exhibit significantly reduced intra-class distances, as evidenced by a higher cosine similarity approaching 1.0000. Furthermore, by compacting the semantics within each class, we also achieve an increased inter-class distance. These outcomes suggest that our method successfully generates unlearnable examples characterized by minimal intra-class distances and maximized inter-class distances, thereby enhancing unlearnability.

The calculation detail of intra-/inter-class distance is shown below.
We consider the output of the last CNN layer as the latent feature for each sample and average these to determine the mean latent feature per class. For intra-class distance, we compute and report the mean cosine similarity between each sample's latent feature and its class mean. For inter-class distance, we calculate the mean cosine similarity between each class's mean latent feature and the overall dataset mean.

**Table J: Evaluation of Inter-Class and Intra-Class Distances.**

| Data         | Intra-class | Inter-class |
|:------------:|:-----------:|:-----------:|
| Clean images | 0.8457      | 0.8742      |
| Ours(S)      | 0.9702      | 0.9030      |
| Ours(C)      | 0.9882      | 0.8815      |

Reviewer comment (Experiments):
Using clean images from different classes as hidden semantic images, or using random natural images as hidden semantic images, rather than generated ones. As you noted, this removes the control over the consistency of semantics within a class. The drop in unlearnability can show the importance of controlled generation.

Author response:
**Ans:** Thank you so much for the detailed suggestions on ablation studies. The ablations significantly improve our papers. We have conducted the following ablations and show the results in the respective tables:
- Ablation on each component in Semantic Generation Module. The results are shown in Table D and Table 4.
- Ablation on LFC. The results are shown in the modified Table 4.

We hope ablation studies can address your concerns and provide better insights into the components.

Reviewer comment (Experiments):
Removing the Latent Feature Concentration (LFC) module. This would demonstrate the impact of the proposed module in regularizing intra-class perturbations.

Author response:
**Ans:** Thank you so much for the detailed suggestions on ablation studies. The ablations significantly improve our papers. We have conducted the following ablations and show the results in the respective tables:
- Ablation on each component in Semantic Generation Module. The results are shown in Table D and Table 4.
- Ablation on LFC. The results are shown in the modified Table 4.

We hope ablation studies can address your concerns and provide better insights into the components.

Reviewer comment (Experiments):
Removing the CLIP-based clustering of text prompts. Using random prompts for generation removes controlled inter-class differences.

Author response:
**Ans:** Thank you so much for the detailed suggestions on ablation studies. The ablations significantly improve our papers. We have conducted the following ablations and show the results in the respective tables:
- Ablation on each component in Semantic Generation Module. The results are shown in Table D and Table 4.
- Ablation on LFC. The results are shown in the modified Table 4.

We hope ablation studies can address your concerns and provide better insights into the components.

Reviewer comment (Evaluation):
Evaluating inter-class and intra-class separation quantitatively using metrics like mean intra-class distance and mean inter-class distance. This can formally validate the claims.

Author response:
**Ans:** Thank you so much for the detailed suggestions on ablation studies. The ablations significantly improve our papers. We have conducted the following ablations and show the results in the respective tables:
- Ablation on each component in Semantic Generation Module. The results are shown in Table D and Table 4.
- Ablation on LFC. The results are shown in the modified Table 4.

We hope ablation studies can address your concerns and provide better insights into the components.

Reviewer comment (Presentation):
The paper can bring the hiding process into the main text in comparison to adding it to the supplementary file.

Author response:
**Q1:** The paper can bring the hiding process into the main text in comparison to adding it to the supplementary file.


**Ans:**  Thank you for your suggestion. We've added the specific hiding process to the main text, making it easier for readers to understand our method.

Reviewer comment (Experiments):
The experimental section of the paper is weak. The ablation studies with several parameters used in the proposed algorithm are missing. For example: ablation concerning the role of individual loss items, concerning values of $w_i$.

Author response:
**Q2:** The experimental section of the paper is weak. The ablation studies with several parameters used in the proposed algorithm are missing. For example: ablation concerning the role of individual loss items, concerning values of wi.


**Ans:** Thank you for your comments on the experimental sections. We have added several ablation results based on your suggestions:
- Ablation on weights of individual loss wi: we conduct a grid search on the 3 loss weights separately. We search from 0.01 to 10 for w1 and w2, and search from 1e-5 to 1e-2 for w3, since loss associated with w3 is generally 1000 times larger. The results in Table A&B&C show that when we set weights to 1, 1, 0.0001, the mean accuracy against countermeasures archives the best performance. Hence, we adopt these hyperparameters in all our experiments.
- Ablation on generation module: we further conduct ablation study on the impact of the semantic image generation module (Table 4 and Table D). We show that each module plays significant role to further improve the robustness of the deep hiding unlearnable examples.


We hope these amendments address your concerns and further strengthen our paper.


**Table A: The experimental results of different settings on parameters of $\omega_1$.**
| Setting     | $\omega_1$ | Vanilla | Cutout | Cutmix | Mixup | MeanF | MedianF | BDR   | Gray  | GaussN | GaussF | JPEG10 | JPEG50 | AT    | Mean      |
|:-----------:|:----------:|:-------:|:------:|:------:|:-----:|:-----:|:-------:|:-----:|:-----:|:------:|:------:|:------:|:------:|:-----:|:---------:|
| Sample-wise | 10         | 16.00   | 16.35  | 13.69  | 17.28 | 30.86 | 64.48   | 23.40 | 18.34 | 24.11  | 13.70  | 82.03  | 70.60  | 48.92 | 33.83     |
|  Sample-wise  (Ours)         | 1    | 15.36   | 10.79  | 10.00  | 14.72 | 17.68 | 17.00   | 21.12 | 17.61 | 22.78  | 11.16  | 80.41  | 81.03  | 38.31 | **27.54** |
|      Sample-wise       | $10^{-1}$  | 13.54   | 17.32  | 14.11  | 17.24 | 29.90 | 37.61   | 28.54 | 19.15 | 25.25  | 16.23  | 81.48  | 63.96  | 58.30 | 32.51     |
|    Sample-wise         | $10^{-2}$  | 11.31   | 12.48  | 10.01  | 11.12 | 17.11 | 23.48   | 17.46 | 13.89 | 22.41  | 10.05  | 80.04  | 84.38  | 45.20 | 27.61     |
| Class-wise  | 10         | 11.60   | 10.10  | 12.52  | 10.83 | 11.92 | 21.24   | 17.48 | 11.45 | 12.42  | 10.00  | 76.42  | 20.26  | 17.56 | 18.75     |
|    Class-wise  (Ours)     | 1    | 10.00   | 10.00  | 11.25  | 10.02 | 10.59 | 10.04   | 13.53 | 10.00 | 10.00  | 10.00  | 72.97  | 23.62  | 10.00 | **16.31** |
|      Class-wise     | $10^{-1}$  | 10.00   | 10.00  | 9.99   | 10.00 | 10.11 | 10.09   | 12.44 | 10.00 | 10.00  | 10.00  | 72.36  | 28.89  | 10.05 | 16.46     |
|      Class-wise      | $10^{-2}$  | 10.00   | 10.00  | 10.02  | 10.00 | 10.09 | 10.00   | 16.34 | 10.00 | 10.00  | 10.00  | 72.45  | 58.50  | 10.00 | 19.03     |

**Table B: The experimental results of different settings on parameters of $\omega_2$.**
| Setting | $\omega_2$ | Vanilla | Cutout | Cutmix | Mixup | MeanF | MedianF | BDR   | Gray  | GaussN | GaussF | JPEG10 | JPEG50 | AT    | Mean      |
|:-------:|:----------:|:-------:|:------:|:------:|:-----:|:-----:|:-------:|:-----:|:-----:|:------:|:------:|:------:|:------:|:-----:|:---------:|
| Sample-wise  | 10         | 15.12   | 10.35  | 10.34  | 13.58 | 17.94 | 23.41   | 20.41 | 16.00 | 24.53  | 10.01  | 81.89  | 80.81  | 39.88 | 28.02     |
|    Sample-wise (Ours)     | 1    | 15.36   | 10.79  | 10.00  | 14.72 | 17.68 | 17.00   | 21.12 | 17.61 | 22.78  | 11.16  | 80.41  | 81.03  | 38.31 | **27.54** |
|    Sample-wise      | $10^{-1}$  | 16.99   | 16.81  | 11.01  | 12.22 | 25.72 | 35.44   | 26.33 | 20.33 | 27.34  | 16.66  | 82.40  | 81.74  | 69.43 | 34.03     |
|     Sample-wise     | $10^{-2}$  | 10.57   | 15.16  | 21.60  | 14.12 | 19.67 | 35.44   | 22.87 | 18.17 | 29.63  | 10.82  | 82.99  | 87.50  | 76.03 | 34.20     |
| Class-wise   | 10         | 10.00   | 10.00  | 11.22  | 11.94 | 10.09 | 10.10   | 13.27 | 10.00 | 10.00  | 10.08  | 77.23  | 22.81  | 10.12 | 16.68     |
|  Class-wise  (Ours) | 1    | 10.00   | 10.00  | 11.25  | 10.02 | 10.59 | 10.04   | 13.53 | 10.00 | 10.00  | 10.00  | 72.97  | 23.62  | 10.00 | **16.31** |
|   Class-wise      | $10^{-1}$  | 10.04   | 10.00  | 11.64  | 13.81 | 15.49 | 10.43   | 17.09 | 10.16 | 10.94  | 10.05  | 77.79  | 25.29  | 18.35 | 19.25     |
|     Class-wise    | $10^{-2}$  | 10.00   | 10.00  | 10.03  | 10.28 | 10.41 | 10.36   | 15.29 | 10.87 | 10.00  | 10.00  | 79.46  | 29.51  | 10.00 | 17.40     |

Reviewer comment (Experiments):
The authors have mentioned that $100$ semantic images are generated for image hiding. Are these all $100$ images used and how?

Author response:
**Q3:** The authors have mentioned that 100 semantic images are generated for image hiding. Are these all 100 images used and how?

**Ans:** We generate 100 images for each class, resulting in a total of 1k, 10k, and 10k semantic images for CIFAR10, CIFAR100, and ImageNet-100, respectively. Thus, these semantic images have the same class number as the target dataset.

During the hiding process, we differ in the image selection for two types of hiding settings:

- In the class-wise setting, we randomly select only one semantic image for all the clean images in the same class. In other words, each image in a class shares the same selected semantic images, while images in different classes are assigned with different semantic images.

- In the sample-wise setting, for each class, we generate 100 semantic images with the same text prompt and canny edge map, to balance the diversity and generation efficiency of the semantic images. For each image in the same class, we randomly pick one semantic image from the 100-image pool and hide it into the clean image.

Reviewer comment (Experiments):
Why the existing algorithms are re-implemented? Are they implemented and fine-tuned using the parameters provided in the papers? Can't we use the protocols of existing work for direct comparisons?

Author response:
**Q4:** Why the existing algorithms are re-implemented? Are they implemented and fine-tuned using the parameters provided in the papers? can't we use the protocols of existing work for direct comparisons?


**Ans:** Thanks for your question regarding the re-implementation of existing algorithms.
We would like to clarify that we follow the same protocols of existing works based on their papers and the public code repositories. Besides, we implement and fine-tune the parameters provided in the papers. The ‘re-implementation’ indicates that we integrate all the existing methods into a unified code framework.

The primary reason for ‘re-implementing’ these algorithms was to ensure consistency and fairness in our comparisons. To evaluate the general robustness of the UEs, our study involved 13 different countermeasures. Since the existing papers did not provide all the necessary results for these specific conditions, we opted to implement the existing methods in a unified framework.

Reviewer comment (Experiments):
Any reason why *JPEG10* is yielding significantly higher robustness across each unlearnable example? Any explainable reason for this and have the authors tried with a lower **JPEG** value?

Author response:
**Q5:** Any reason why JPEG10 is yielding significantly higher robustness across each unlearnable example? Any explainable reason for this and have the authors tried with a lower JEPG value?

**Ans:** We show examples of images compressed by JPEG(quality factor=10) in the supplementary. The examples show that the compression level of JPEG10 can result in significant distortion to the images, leading to a decrease in the performance of UE.
In response to your suggestion, we experimented with lower JPEG values (Table I). We found that lower JPEG values result in even lower test accuracy, which confirms that stronger compression not only damages the unlearnable example perturbations but also distorts the original image features significantly.

**Table I: Test accuracy (\%) of model train on unlearnable examples from CIFAR10 against JPEG compression.**
| JPEG Quality Factor |2|     4     | 6     | 8     | 10    | 50    |
|:-------------------:|:-----:|:-----:|:-----:|:-----:|:-----:|:-----:|
| Ours(S)         | 68.01 | 74.35 | 77.39 | 78.56 | 80.41 | 81.03 |
| Ours(C)      | 64.53 | 70.18 | 72.56 | 72.72 | 72.97 | 23.62 |

Reviewer comment (Experiments):
Have the authors studied the transferability of the proposed examples in the presence of defenses?

Author response:
**Q6:** Have the authors studied the transferability of the proposed examples in the presence of defenses?

**Ans:** To ensure a thorough validation, we tested our UEs on CIFAR10 across multiple architectures, which provided us with a broad view of their transferability.
The detailed results have also been included in the supplementary file for reference. This additional data supports our claim that the proposed deep hiding UEs maintain their efficacy against different countermeasures across architectures.

**Table E: Test accuracy (\%) of model train on unlearnable examples from CIFAR-10 with five architectures, including ResNet-18 (R18), ResNet-50 (R50), VGG-19 (V19), and DenseNet-121 (D121), and Vision Transformer (ViT), against data augmentations, data preprocessing, and adversarial training.**

| Settings    | Model | Vanilla | Cutout | Cutmix | Mixup | MeanF | MedianF | BDR   | Gray  | GaussN | GaussF | JPEG10 | JPEG50 | AT    | Mean  |
|:-----------:|:-----:|:-------:|:------:|:------:|:-----:|:-----:|:-------:|:-----:|:-----:|:------:|:------:|:------:|:------:|:-----:|:-----:|
| Ours(S) | R18   | 15.36   | 10.79  | 10.00  | 14.72 | 17.68 | 17.00   | 21.12 | 17.61 | 22.78  | 11.16  | 80.41  | 81.03  | 38.31 | 27.54 |
|    Ours(S)         | R50   | 13.32   | 12.42  | 12.99  | 11.60 | 18.70 | 22.70   | 23.77 | 12.62 | 17.24  | 16.23  | 80.70  | 78.30  | 37.11 | 27.52 |
|     Ours(S)        | V19   | 10.45   | 17.25  | 14.37  | 17.87 | 23.62 | 32.27   | 22.77 | 17.72 | 22.28  | 14.89  | 80.61  | 80.64  | 52.55 | 31.33 |
|      Ours(S)       | D121  | 18.88   | 21.16  | 12.52  | 18.81 | 53.41 | 53.10   | 28.10 | 18.06 | 12.54  | 18.22  | 77.93  | 76.46  | 70.58 | 36.91 |
|      Ours(S)       | ViT   | 15.80   | 20.64  | 21.93  | 10.67 | 54.05 | 54.57   | 25.40 | 15.20 | 22.45  | 21.38  | 65.38  | 64.91  | 49.50 | 33.99 |
| Ours(C)  | R18   | 10.00   | 10.00  | 11.25  | 10.02 | 10.59 | 10.04   | 13.53 | 10.00 | 10.00  | 10.00  | 72.97  | 23.62  | 10.00 | 16.31 |
|     Ours(C)        | R50   | 10.00   | 10.00  | 11.02  | 10.04 | 10.06 | 10.45   | 17.31 | 10.00 | 10.00  | 10.00  | 74.43  | 24.15  | 11.09 | 16.81 |
|   Ours(C)            | V19   | 10.58   | 10.17  | 10.00  | 17.46 | 10.78 | 12.86   | 15.79 | 10.30 | 10.03  | 10.03  | 71.77  | 24.84  | 13.44 | 17.54 |
|      Ours(C)         | D121  | 10.00   | 10.00  | 10.60  | 10.23 | 10.49 | 10.01   | 11.63 | 10.00 | 10.00  | 10.53  | 72.85  | 22.06  | 10.00 | 16.03 |
|    Ours(C)           | ViT   | 10.00   | 10.01  | 10.02  | 10.67 | 11.23 | 22.34   | 12.35 | 10.12 | 10.00  | 10.00  | 61.92  | 30.69  | 18.79 | 17.55 |

Reviewer comment (Evaluation):
It is surprising to see that even at 80\%, the test accuracy is significantly high (84.40\%), which suddenly drops to 10.00\% in presence of 100\% examples. Am I correct? If yes, what is the reason?

Author response:
**Q7:** It is surprising to see that even at 80%, the test accuracy is significantly high (84.40%), which suddenly drops to 10.00% in presence of 100% examples. Am I correct? If yes, what is the reason?

**Ans:** You are right. We hypothesize that this is because 20% of clean data provides useful features to guide the classifier, thereby affecting the shortcut learning towards hidden semantic images. This phenomenon aligns with the results of previously published papers on unlearnable examples [A][B][C]. In existing methods, a significant drop in performance has been observed for a subset of data subjected to perturbations.
```
[A] Hanxun Huang, Xingjun Ma, Sarah Monazam Erfani, James Bailey, and Yisen Wang. Unlearnable examples: Making personal data unexploitable. ICLR, 2021.
[B] Shaopeng Fu, Fengxiang He, Yang Liu, Li Shen, and Dacheng Tao. Robust unlearnable examples: Protecting data privacy against adversarial learning. ICLR, 2022.
[C] Sadasivan, Vinu Sankar, Mahdi Soltanolkotabi, and Soheil Feizi. Cuda: Convolution-based unlearnable datasets. CVPR, 2023.
```
**Table G: Test accuracy (\%) of CIFAR-10 on the models trained by
the clean data mixed with different percentages of unlearnable examples.**
| Percentage  | 20%    | 40%    | 60%    | 80%    | 85%    | 90%    | 92%    | 94%    | 96%    | 98%    | 100%   |
|:-----------:|:-----:|:-----:|:-----:|:-----:|:-----:|:-----:|:-----:|:-----:|:-----:|:-----:|:-----:|
| Ours(S) | 93.73 | 92.41 | 90.08 | 84.40 | 84.07 | 81.37 | 79.92 | 77.30 | 71.93 | 59.34 | 10.00 |
| Ours(C)  | 93.53 | 92.67 | 89.99 | 84.47 | 84.20 | 81.41 | 78.42 | 75.71 | 66.90 | 53.00 | 15.36 |

Reviewer comment (Experiments):
What about transferability in terms of limited samples used to learn unlearnable examples?

Author response:
**Q8:** What about transferability in terms of limited samples used to learn unlearnable examples?

**Ans:** We conducted the transferability study across architectures with limited unlearnable examples, and we show the results in Table F in the supplementary. The test accuracy decreases in a similar trend when we increase the percentage of the unlearnable examples.

**Table F: Test accuracy (\%) of CIFAR-10 on the different models trained by
the clean data mixed with different percentages of unlearnable examples.**

| Setting                      | Model | 20\%  | 40\%  | 60\%  | 80\%  |
|:----------------------------:|:-----:|:-----:|:-----:|:-----:|:-----:|
| Ours(S) | R18   | 93.73 | 92.41 | 90.08 | 84.40 |
|            Ours(S)                   | R50   | 94.16 | 92.82 | 90.51 | 85.66 |
|          Ours(S)                     | V19   | 92.11 | 91.14 | 88.77 | 83.48 |
|           Ours(S)                    | D121  | 89.02 | 87.77 | 84.94 | 81.39 |
|            Ours(S)                   | ViT   | 75.95 | 74.77 | 74.09 | 70.50 |
| Ours(C)  | R18   | 93.53 | 92.67 | 89.99 | 84.47 |
|             Ours(C)                  | R50   | 94.04 | 92.31 | 90.58 | 85.15 |
|              Ours(C)                 | V19   | 92.13 | 90.23 | 87.99 | 80.60 |
|               Ours(C)                | D121  | 88.28 | 86.56 | 83.57 | 77.79 |
|                Ours(C)               | ViT   | 75.44 | 74.58 | 69.34 | 64.80 |

Reviewer comment (Novelty):
The paper also needs to discuss existing contemporary image-hiding works effective in generating adversarial examples and how they are different from this work. Why they can not be used as compared to the proposed DH?

Author response:
**Q9:** The paper also needs to discuss existing contemporary image-hiding works effective in generating adversarial examples and how they are different from this work. Why they can not be used as compared to the proposed DH?
```
1. Din SU, Akhtar N, Younis S, Shafait F, Mansoor A, Shafique M. Steganographic universal adversarial perturbations. Pattern Recognition Letters. 2020 Jul 1;135:146-52.
2. A. Agarwal, N. Ratha, M. Vatsa and R. Singh, "Crafting Adversarial Perturbations via Transformed Image Component Swapping," in IEEE Transactions on Image Processing, vol. 31, pp. 7338-7349, 2022, doi: 10.1109/TIP.2022.3204206.
```
**Ans:** Thank you for suggesting a comparison with contemporary image-hiding techniques in the context of adversarial examples. Our work indeed shares the image-hiding framework with the studies you mentioned, but our goals and methodologies differ substantially.

The specific aim of our work is to disrupt the learning process of a model. We seek to ensure high training accuracy on the unlearnable examples we generate, while deliberately reducing accuracy on a clean testing set. The adversarial examples crafted in the studies you listed are intended to lead a fully trained model to make incorrect predictions, a different goal than ours.

Additionally, we have expanded our research to different image-hiding networks, notably the ISGAN [D]. Our experiments assess the effectiveness of ISGAN-hidden unlearnable examples. The findings reveal that while unlearnability can be achieved with other deep hiding models like ISGAN, the performance is not as optimal as with our applied Invertible Neural Network (INN). INN demonstrates superior performance in deep hiding [E][F], which is why we chose it as our baseline model to validate our concepts.

We believe these distinctions, along with our comprehensive evaluations, underscore the unique contribution of our work in the field of image hiding and data privacy.

```
[D] Zhang, R., Dong, S., & Liu, J. (2019). Invisible steganography via generative adversarial networks. Multimedia tools and applications, 78, 8559-8575.
[E] Xu Y, Mou C, Hu Y, et al. Robust invertible image steganography. CVPR, 2022: 7875-7884.
[F] Mou C, Xu Y, Song J, et al. Large-capacity and flexible video steganography via invertible neural network. CVPR, 2023: 22606-22615.
```

**Table H: Test accuracy (\%) of model train on unlearnable examples generated by using another deep hiding model (ISGAN).**
| ISGAN      | Vanilla | Cutout | Cutmix | Mixup | MeanF | MedianF | BDR   | Gray  | GaussN | GaussF | JPEG10 | JPEG50 | AT    | Mean  |
|------------|---------|--------|--------|-------|-------|---------|-------|-------|--------|--------|--------|--------|-------|-------|
| Sample-wise| 54.05   | 53.74  | 55.97  | 72.61 | 48.83 | 85.11   | 86.77 | 53.04 | 88.45  | 54.25  | 84.35  | 90.55  | 87.22 | 70.38 |
| Class-wise| 23.99   | 24.07  | 28.42  | 42.69 | 37.56 | 79.3    | 83.64 | 30.06 | 87.92  | 25.24  | 84.46  | 90.28  | 87.46 | 55.78 |