# Backdoor Attacks on Deep Neural Networks via Transfer Learning from Natural Images
## 4 Discussion  
Our results show that backdoors set up in models pretrained on the ImageNet dataset with a very small trigger were transferable to models fine-tuned using clean separate datasets via transfer learning (Table 1). Previous studies [14, 15, 20] have also indicated the transferability of backdoors; however, they were limited to transferability between similar prediction tasks. Our results show that backdoors are transferable via transfer learning in more realistic situations in which pretrained models (in this case, ImageNet models for natural image classification) are utilized to obtain DNN models for different prediction tasks (e.g., pneumonia classification from chest X-ray images and emergency response monitoring from aerial images). Given that transfer learning using models pretrained on the ImageNet dataset is widely used as a common technique in computer vision, the transferability of backdoor attacks may be considered to pose a relatively serious security threat. Adversaries can set up backdoors in many DNN models used for wide-ranging applications by leading users to download and use backdoored ImageNet models (e.g., by providing source code or sample data). Backdoor transferability can cause DNN models to misclassify samples, leading to various potentially serious failure modes for different applications; for example, an automated vehicle could be involved in an otherwise-avoidable traffic accident [9, 13, 15]. We found that the effectiveness of backdoors in the DNN models fine-tuned from backdoored models was high, despite being limited in previous models and methods [14, 15, 20]. Specifically, given that a number of layers need to be frozen during the transfer learning process for a high ASR [20], the results of the present study are remarkable in that a high (>90%) ASR was observed (Table 1), although all trainable layer parameters were fine-tuned. This may have occurred because the DNN models (e.g., InceptionV3 and ResNet-50) used in this study were too large (overparameterized) for the prediction tasks. Studies have shown that backdoors can be set up in redundant parts of DNN models [14, 33]; thus, they can be configured more easily in overparameterized DNN models. In addition, the weight parameters of fine-tuned DNN models are known to be similar to those of the original pretrained DNN models owing to their overparameterization, despite the fine-tuning process [34]. Therefore, backdoors remain effective in large DNN models after transfer learning. However, small (i.e., not overparameterized) DNN models can be predicted to exhibit lower transferability of backdoors. In fact, the ASR values were low (equivalent to the ASR baseline) for relatively small DNN models (VGG-16, VGG-19, and MobileNet; Table 1). Hence, simpler DNN models can be used as a straightforward defense against backdoor attacks via transfer learning. However, this approach may be unrealistic, given that DNN models with high prediction accuracy are required for real-world applications. Figure 1: Anomaly index for the models fine-tuned from the backdoored and clean ImageNet models for pneumonia classification from chest X-ray images (ChestX), emergency response monitoring from aerial images (Emergency), face recognition (Face), and age classification from face images (Age): InceptionV3 (**a**) and DenseNet121 (**b**) architectures. The well-known neural cleanse [17] backdoor detection method failed to detect the backdoors in the models fine-tuned from the backdoored ImageNet models (Figure 1). This may have occurred because the trigger size was significantly smaller than that in previous studies; specifically, the size in this study was \(<\)0.05% of the entire image, whereas in a previous study [17] it was \(\sim\)1% of the entire image. Moreover, the detection failure could have been caused by the DNN models being too large for the prediction tasks (i.e., overparameterization). Backdoors are difficult to locate because they are set up in relatively few neurons in complex DNN models.