SlowGuess/NeurIPS_2023_OCR huggingface dataset:
We can also observe that DDNM shows better clean accuracy than ZIP, but it cannot effectively defend against backdoor attacks like Blended. This is because DDNM restores the attack patterns in its diffusion process, which decreases the defense performance.
H.2 Qualitative Results Comparison with Image Restoration Methods
In this subsection, we conduct a qualitative analysis of the purification effect between our proposed method and DDNM.
(a) Attack Image
(b) Restored by DDNM
(c) Purified by ZIP
(d) Attack Image
(g) Attack Image
Figure 12: Comparison of DDNM and ZIP on defending Blended attack.
(e) Restored by DDNM
(h) Restored by DDNM
(f) Purified by ZIP
(i) Purified by ZIP
(d) Attack Image
(g) Attack Image
Figure 13: Comparison of DDNM and ZIP on defending BadNet attack.
Due to our reliance on a pre-trained diffusion model to implement zero-shot purification, the effectiveness of generating purified images may be weakened when our model is applied to highly specific images that fall outside the distribution of pre-processed data. To mitigate this issue, we suggest two possible solutions in future work: 1) replacing the current pre-trained diffusion model with a more suitable pre-trained model for such specific images, and 2) collecting a subset of highly specific images to perform fine-tuning on the pre-trained model.