# A Study of Adversarial Attacks and Detection on Deep Learning-Based Plant Disease Identification
## 3 Experiments and Results
### Results of Adversarial Sample Detection  
The testsets generated for adversarial attacks with \(\epsilon=1/255\) in Section 3.3 are used as the datasets for evaluating the performance of adversarial sample detection methods. Detection features are extracted from the fine-tuned DNN models of Section 3.2. The KD and BU features are estimated from the second-last layer and the SoftMax layer of the network, respectively. The LID features are calculated from the outputs of all layers of the network with a min-batch size of 100. \(DeepF\) and \(DiscF\) features are obtained from the second-last layer of the network. According to [46; 48], logistic regression classifier is used for KD + BU and LID features. RBF-SVM classifier is used for \(DeepF\) and \(DiscF\) features based on the SafetyNet architecture [47]. We apply a 5-fold cross-validation to evaluate the performance of different adversarial sample detection methods. Tables 3 and 4 show the results of adversarial sample detection for fine-tuned 2-class and multi-class DNN models, respectively. The results show that KD + BU features achieve significantly better performance than other features with a nearly perfect detection rate. This demonstrates that adversarial attacks on DL-based plant disease identification models can be effectively defended by using adversarial sample detection with an appropriate choice of features. Surprisingly, LID features are the worst performed ones which were shown superior performance over KD + BU features on three benchmark image datasets: MNIST, CIFAR-10, and SVHN in [48]. This implies that the intrinsic characteristics of leaf images used for plant disease identification are different from those of benchmark images. \begin{table}
\begin{tabular}{l l l l l l} \hline \hline
**DNN Model** & **Features** & **FGSM** & **BIM** & **PGD** & **CW** \\ \hline \multirow{4}{*}{VGG-16} & KD + BU & **0.998** & **1** & **1** & **0.998** \\  & LID & 0.738 & 0.666 & 0.633 & 0.637 \\  & DeepF & 0.882 & 0.899 & 0.902 & 0.732 \\  & DiscF & 0.918 & 0.921 & 0.92 & 0.737 \\ \hline \multirow{4}{*}{ResNet-101} & KD + BU & **1** & **1** & **1** & **1** \\  & LID & 0.736 & 0.698 & 0.607 & 0.807 \\  & DeepF & 0.94 & 0.948 & 0.957 & 0.945 \\  & DiscF & 0.946 & 0.968 & 0.956 & 0.91 \\ \hline \multirow{4}{*}{Inception-V3} & KD + BU & **1** & **1** & **0.998** & **0.998** \\  & LID & 0.754 & 0.74 & 0.705 & 0.587 \\  & DeepF & 0.942 & 0.907 & 0.912 & 0.839 \\  & DiscF & 0.954 & 0.905 & 0.885 & 0.838 \\ \hline \multirow{4}{*}{DenseNet-121} & KD + BU & **1** & **1** & **1** \\  & LID & 0.738 & 0.606 & 0.571 & 0.666 \\ \cline{1-1}  & DeepF & 0.962 & 0.942 & 0.948 & 0.94 \\ \cline{1-1}  & DiscF & 0.967 & 0.984 & 0.967 & 0.916 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Adversarial sample detection results for 2-class DNN models using different features. Finally, we investigate the transferability of detection models built with KD + BU features. The detection model for a DNN model is trained with a testset generated in Section 3.3 that consists of normal samples and adversarial samples generated by one of the four attacks. The model is then applied for detecting adversarial samples generated by other three attacks. Tables 5 and 6 show the detection performance in terms of accuracy for 2-class and multi-class DNN models, respectively. It can be seen that detection models for 2-class and multi-class DNN models have comparable transferability. Detection models trained with adversarial samples generated by CW attack have the best transferability which can perfectly detect adversarial samples generated by other three attacks except one case (Inception V3, 2-class).