## 4 Performance Evaluation
### Experimental Results
(\%) & 72.07/77.78 & 77.78/33.89 & 63.89/89.06 & 66.67/11.11/69.42 & 68.69 \(\pm\) 64.43/65.28 \(\pm\) 76.22 \(\pm\) 72.22 & 69.74 \(\pm\) 76.91 \(\pm\) 76.91 \(\pm\) 76.91 \(\pm\) 76.95 & 66.72 \(\pm\) 76.55 \(\pm\) 76.91 \(\pm\) 76.91 \(\pm\) 76.95 \\ \cline{2-11}  & & Acc. (\%) & 72.07/77.78 & 80.56/11.14 & 80.56/66.67 & 64.64/64.64 & 55.56/55.56 & 70.56 \(\pm\) 64.01 \(\pm\) 75.40 \(\pm\) 64.11 \(\pm\) 7.54 \\ \cline{2-11}  & & Acc. (\%) & 66.57/77.88 & 80.71/61.11 & 80.54/62.50 & 69.47/68.84 & 55.42/74.29 & 70.42 \(\pm\) 76.34/64.09 \(\pm\) 73.93 \(\pm\) 76.66 \(\pm\) 9.10 \\ \cline{2-11}  & & Acc. (\%) & 0.35/05.56 & 0.436/0.222 & 61.02/44.07 & 0.390/0.465 & 10.12/0.118 & 0.417 \(\pm\) 0.193/0.350 \(\pm\) 0.158 & 0.38 \(\pm\) 0.180 \\ \cline{2-11}  & & Acc. (\%) & 69.44/77.78 & 83.33/55.56 & 77.78/66.47 & 83.72/72.22 & 83.83/77.78 & 70.56 \(\pm\) 9.00 \(\pm\) 83.17 & 70.28 \(\pm\) 8.70 \\ \cline{2-11}  & & Acc. (\%) & 69.44/77.77 & 83.33/55.50 & 77.71/62.50 & 63.68/72.71 & 83.30/77.71 & 70.44 \(\pm\) 9.00/0.81 \(\pm\) 8.94 & 69.72 \(\pm\) 9.02 \(\pm\) 9.02 \\ \cline{2-11}  & & Acc. (\%) & 0.39/0.599 & 0.684/11.14 & 0.59/0.447 & 0.282/0.447 & 0.167/0.599 & 0.418 \(\pm\) 0.166/0.425 \(\pm\) 0.143 & 0.412 \(\pm\) 0.175 \\ \hline \multirow{1}{*}{ \begin{tabular}{c} RNN-SH \\ (tanh) \\ \end{tabular}  } & Acc. (\%) & 61.11/5.782 & 83.33/11.117 & 27.11/51.67 & 66.67/86.87 & 66.89/83.81 & 68.18 \(\pm\) 50.52/83.11 \(\pm\) 9.69 & 62.50 \(\pm\) 11.06 \\ \cline{2-11}  & & Acc. (\%) & 61.18/92.23 & 83.13/10.00 & 70.79/60.56 & 52.63/83.83 & 66.68 \(\pm\) 50.36 \(\pm\) 50.25 \(\pm\) 11.18 & 59.21 \(\pm\) 1.29 \\ \cline{2-11}  & & Acc. (\%) & 0.48/10.169 & 0.48/0.266 & 0.49/0.226 & 0.40/0.353 & 0.428 \(\pm\) 0.43 \(\pm\) 0.123/0.163 & 0.218 \(\pm\) 0.225 \\ \cline{2-11} \cline{2-11}  & & Acc. (\%) & 72.22/0.694 & 72.22/83.83 & 63.89/66.67 & 61.11/55.56 & 68.89 \(\pm\) 53.69/62.78 \(\pm\) 51.5 & 65.83 \(\pm\) 69.09 \\ \cline{2-11}  & & Acc. (\%) & 72.07/69.23 & 72.07/69.18 & 42.17/53.56 & 64.46/66.46 & 54.18/44.62 & 68.71 \(\pm\) 79.99 \(\pm\) 8.79 \(\pm\) 8.78 & 63.15 \(\pm\) 8.82 \\ \clAs a result, from Table 1, there was no significant difference in the performance between RNNs. Statistically significant difference was not found between the top three models (GRU: two layers and 256 units, LSTM: two layers, 64 units, and RNN-SH (tanh): one layer and 256 units) by the Friedman test. Here, Figure 8 shows the average ROC curve with AUC values of RNN-SH and GRU, and Figure 9 shows a graph of the relationship between the number of units and the parameters of the PD detection model. From Figure 8, it is clear that the AUC value of GRU is higher than the value of RNN-SH (tanh); however, from Figure 9, the parameters of RNN-SH (tanh) with one layer and 256 units are less than a third of the parameters of GRU with two layers and 256 units, although the difference in the performance is small. In addition, the parameter amount of LSTM with two layers and 64 units is not much different from that of RNN-SH with one layer and 256 units. However, in the case of the LSTM, though the total average is high, there is a difference between the validation average and the test average, and it can be confirmed that overfitting has occurred. On the other hand, for the RNN-SH, the validation average and test average are about the same value. Figure 10 shows the examples of confusion matrix in the first trial of LSTM and RNN-SH (tanh). In Figure 10, LSTM has a better performance for the validation set than RNN-SH, but not so much for the test set. Figure 7: The graphs of the average value of each model in Table 1. The left is the validation average, the middle is the test average, and the right is the total average. Here, the standard deviation is omitted to make the graphs easier to see. Figure 8: The average ROC curve with AUC values of GRU and RNN-SH. (**a**) is the ROC curve of the GRU, and (**b**) is that of the RNN-SH (tanh). The average here is an average of the validation set and the test set.