## 4 Experiments
### Signature Verification Results  
Table 3 shows the signature verification results of the proposed method and a comparison with the results of a previous work [3]. As mentioned above, AE (one-class model) showed a better result for distinguishing skilled forgery than MLP (two-class model), but not for others. In distinguishing forgery, CNN_D-AE that was trained with the forged signatures of the subject showed the best accuracy (3.2%). Although this method could not be practically applied, this result showed that a specially trained CNN could learn characteristics for that purpose, as we expected. The CNN_A-AE that was trained with all original and all forged signatures showed the second-best accuracy (4.3%). However, it did not perform \begin{table}
\begin{tabular}{c c c} \hline \hline
**CNN Name** & **Class** & **Data** \\ \hline CNN\_A & two-class & all original, all forged \\ CNN\_B & N-class & all original \\ CNN\_C & two-class & subject’s original, others’ original \\ CNN\_D & two-class & subject’s original, forged for the subject \\ \hline \hline \end{tabular}
\end{table} 
Table 1: CNN feature extractors’ data set. \begin{table}
\begin{tabular}{c c c} \hline \hline
**Verification Test** & **Class 0 (True)** & **Class 1 (False)** \\ \hline Subject/Others & 10 (original) & 20 \(\times\) 19 others (original) \\ Time difference & 20 (time-deferred) & 20 \(\times\) 9 others (original) \\ Skilled forgery & 10 (original) & 20 (forged for the subject) \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Signature verification test data for each subject. \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Models** & **Subject/Others** & **Time Difference** & **Skilled Forgery** \\ \hline MLP [3] & 3.8\% & 7.7\% & 18.1\% \\ AE & 3.5\% & 9.1\% & 13.7\% \\ CNN\_A-AE & 18.1\% & 23.7\% & 4.3\% \\ CNN\_B-AE & 9.1\% & 11.3\% & 23.3\% \\ CNN\_C-AE & 4.1\% & 4.5\% & 4.6\% \\ CNN\_D-AE & 3.5\% & 4.8\% & 3.2\% \\
**CNN\_AC-AE (The Proposed Scheme)** & **2.7\%** & **4.5\%** & **4.4\%** \\ \hline \hline \end{tabular}
\end{table} 
Table 3: CNN-AE classification comparison with EER. well in the subject/others and time difference tests. CNN_C-AE showed better performances in the subject/others (4.1%) and time difference (4.5%) tests, but it did not perform well in the skilled forgery test compared to CNN_A-AE. Therefore, we tried a combination of CNN_A and CNN_C (called CNN_AC-AE) by concatenating _S-vector_ from both networks and doubling the size of the AE input/output. The performance of the CNN_AC-AE was better than that of CNN_C-AE in the skilled forgery and subject/others tests, and was the same in the time difference test. It showed 1.1% (subject/others), 3.2% (time difference), and 13.7% (skilled forgery) better results than those of the previous work [3] and showed 1.8%, 4.6%, and 9.3%, respectively, better performances than those of the test using AE only. Figure 8: ROC curve of: (**a**) subject/others; (**b**) time difference; and (**c**) skilled forgery. Figure 8 shows the experiment results as receiver operation characteristic (ROC) curves. An ROC curve refers to a graphical plot that illustrates the correlation between sensitivity and specificity of a classifier system. The ROC curve is created by plotting the true positive rate (TPR) against the false positive rate (FPR). Therefore, the closer a curve is to 1, the more accurate is the corresponding classifier. Figure 8a shows ROC curves for subject/others signature authentication. The CNN_A-AE showed the worst accuracy and the CNN_AC-AE showed the best performance. Because the CNN_A-AE was trained with the forged signatures, which are similar to those of the subject, it has a higher probability of misclassifying subject signatures as forged signatures. Figure 8b shows the ROC curves for the time difference test. The worst result was obtained by the CNN_A-AE and the best one was obtained by the CNN_AC-AE. The CNN_A-AE has higher probability of misclassifying subject signatures changed over time as forged signatures for the same reason mentioned above.