# ShuffleDetect: Detecting Adversarial Images against Convolutional Neural Networks
## 9 Intrinsic Performance of ShuffleDetect\({}^{\mathbf{C},R_{th},t}\)
\begin{table}
\begin{tabular}{c c c c c c c c c c c c c} \multicolumn{11}{c}{**Untargeted Attacks**} \\ \hline \hline \multicolumn{11}{c}{\(R_{th}\)} & \multicolumn{1}{c}{\(\mathbf{C_{1}}\)} & \(\mathbf{C_{2}}\) & \(\mathbf{C_{3}}\) & \(\mathbf{C_{4}}\) & \(\mathbf{C_{5}}\) & \(\mathbf{C_{6}}\) & \(\mathbf{C_{7}}\) & \(\mathbf{C_{8}}\) & \(\mathbf{C_{9}}\) & \(\mathbf{C_{10}}\) & **Avg** \\ \hline \multirow{10}{*}{0.51} & DR & 100 & 100 & 100 & 100 & 100 & 100 & 100 & 100 & 100 & 100 \\ \cline{2-11}  & TP & 68.3 & 63.5 & 74.0 & 70.0 & 65.0 & 72.8 & 82.3 & 68.0 & 92.0 & 88.0 & 74.4 \\ \cline{2-11}  & FP & 26.5 & 26.5 & 22.0 & 25.3 & 25.0 & 21.8 & 20.3 & 19.8 & 36.8 & 33.8 & 25.8 \\ \cline{2-11}  & FN & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\ \cline{2-11}  & Precision & 0.71 & 0.68 & 0.76 & 0.73 & 0.71 & 0.76 & 0.80 & 0.77 & 0.71 & 0.72 & 0.73 \\ \cline{2-11}  & Recall & 1 & 1 & 1 & 1 & 1 & 1 & 1 & 1 & 1 & 1 & 1 \\ \cline{2-11}  & F1 & 0.82 & 0.81 & 0.86 & 0.84 & 0.82 & 0.86 & 0.89 & 0.87 & 0.83 & 0.83 & 0.84 \\ \cline{2-11}  & F1\({}_{Normal}\) & 0.78 & 0.76 & 0.85 & 0.83 & 0.80 & 0.85 & 0.88 & 0.86 & 0.83 & 0.83 & 0.82 \\ \hline \multirow{10}{*}{0.91} & DR & 100 & 100 & 100 & 98.73 & 98.6 & 98.55 & 99.03 & 99.7 & 100 & 98.83 & 99.94 \\ \cline{2-11}  & TP & 68.25 & 63.5 & 74 & 69.25 & 64.25 & 71.75 & 81.5 & 67.75 & 92 & 87 & 73.93 \\ \cline{1-1} \cline{2-11}  & FP & 18.5 & 10 & 10.25 & 16.25 & 10.5 & 11 & 9.5 & 7.5 & 22.25 & 14.5 & 13.03 \\ \cline{1-1} \cline{2-11}  & FN & 0 & 0 & 0 & 0.75 & 0.75 & 1 & 0.75 & 0.25 & 0 & 1 & 0.45 \\ \cline{1-1} \cline{2-11}  & Precision & 0.77 & 0.84 & 0.87 & 0.8 & 0.84 & 0.86 & 0.89 & 0.89 & 0.79 & 0.85 & 0.84 \\ \cline{1-1} \cline{2-11}  & Recall & 1 & 1 & 1 & 0.99 & 0.98 & 0.99 & 0.99 & 0.99 & 1 & 0.99 & 0.99 \\ \cline{1-1} \cline{2-11}  & F1 & 0.87 & 0.91 & 0.93 & 0.88 & 0.91 & 0.92 & 0.93 & 0.94 & 0.88 & 0.91 & 0.90 \\ \cline{1-1} \cline{2-11}  & F1\({}_{Normal}\) & 0.83 & 0.88 & 0.92 & 0.86 & 0.88 & 0.91 & 0.92 & 0.93 & 0.88 & 0.9 & 0.89 \\ \hline \hline \end{tabular}
\end{table} 
Table 13: Average for all indicators (worst case for F1) per CNN over all 4 targeted attacks. \begin{table}
\begin{tabular}{c c c c c c c c c c c c} \multicolumn{11}{c}{**All Attacks**} \\ \hline \hline \multicolumn{11}{c}{\(R_{th}\)} & \multicolumn{1}{c}{\(\mathbf{C_{1}}\)} & \(\mathbf{C_{2}}\) & \(\mathbf{C_{3}}\) & \(\mathbf{C_{4}}\) & \(\mathbf{C_{5}}\) & \(\mathbf{C_{6}}\) & \(\mathbf{C_{7}}\) & \(\mathbf{C_{8}}\) & \(\mathbf{C_{9}}\) & \(\mathbf{C_{10}}\) & **Avg** \\ \hline \multirow{10}{*}{0.51} & DR & 88.73 & 89.86 & 94.68 & 97.11 & 94.86 & 93.86 & 95.55 & 97.07 & 93.50 & 93.21 & 93.84 \\ \cline{2-11}  & TP & 70.54 & 68.61 & 79.14 & 79.86 & 74.36 & 79.02 & 83.70 & 78.50 & 87.43 & 84.71 & 78.59 \\ \cline{1-1} \cline{2-11}  & FP & 30.75 & 30.11 & 24.93 & 28.77 & 27.29 & 23.09 & 21.27 & 22.02 & 35.59 & 32.73 & 27.65 \\ \cline{1-1} \cline{2-11}  & FN & 10.50 & 9.21 & 5.00 & 2.71 & 4.71 & 5.93 & 4.07 & 2.71 & 6.14 & 6.36 & 5.74 \\ \cline{1-1} \cline{2-11}  & Precision & 0.69 & 0.68 & 0.75 & 0.73 & 0.72 & 0.77 & 0.79 & 0.77 & 0.71 & 0.72 & 0.73 \\ \cline{1-1} \cline{2-11}  & Recall & 0.88 & 0.90 & 0.94 & 0.97 & 0.95 & 0.94 & 0.95 & 0.97 & 0.93 & 0.93 & 0.94 \\ \cline{1-1} \cline{2-11}  & F1 & 0.76 & 0.76 & 0.83 & 0.83 & 0.81 & 0.84 & 0.86 & 0.86 & 0.80 & 0.80 & 0.82 \\ \cline{1-1} \cline{2-11}  & F1\({}_{Normal}\) & 0.71 & 0.69 & 0.81 & 0.82 & 0.79 & 0.81 & 0.85 & 0.84 & 0.78 & 0.77 & 0.78 \\ \hline \hline \end{tabular}
\end{table} 
Table 15: Average for all indicators (worst case for F1) per CNN over all attacks.