# The Need for Speed: A Fast Guessing Entropy Calculation for Deep Learning-Based SCA
## 5 Experimental Results
### Random Hyperparameter Search with Different Validation Metrics
As expected, the empirical GE and GEEA methods provide the largest overheads, although GEEA is faster than empirical GE. The mutual information method provides the second-best results, which is related to the more straightforward calculation than guessing entropy. Table 4 provides the % that each metric can select a generalizing model with early stopping (model that reaches GE = 1 in the attack phase, which is indicated by line 13 in Algorithm 1) from the random search. Together with GEEA, the fast GE is a highly efficient metric (top two performance in all considered scenarios). Most importantly, we verified that FGE is always superior to the situation where no early stopping is used (200 epochs in the table) and with negligible overhead. For the case of the identity leakage models, FGE shows the best results. Figure 2 shows the results for the ASCADf dataset. When side-channel traces are labeled according to the Hamming weight leakage model, the correct key is recovered with 514 traces for the GEEA metric and 534 traces (the second best) with FGE early stopping metric. In the case of the identity leakage model, the best results are achieved for the FGE metric, where 101 attack traces are needed to achieve a GE equal to 1, which is aligned with state-of-the-art results [8; 9; 21]. The good performing results from the mutual information metric and the GE obtained with 200 epochs indicate the effectiveness of early stopping metrics in preventing the best model from overfitting. Again, we confirm that FGE is highly competitive in both leakage models and requires 10\(\times\) fewer validation traces. \begin{table}
\begin{tabular}{c c c c c c} \hline  & \multicolumn{2}{c}{**ASCADf**} & \multicolumn{2}{c}{**ASCADr**} & \multicolumn{2}{c}{**CHES CTF**} \\  & **HW** & **Identity** & **HW** & **Identity** & **HW** \\ \hline Fast GE & 56.52\% & 43.46\% & 49.63\% & 34.13\% & 21.85\% \\ Emp. GE & 59.66\% & 43.16\% & 50.00\% & 33.23\% & 20.79\% \\ GEEA & 59.66\% & 43.46\% & 54.34\% & 29.30\% & 21.18\% \\ MI & 50.74\% & 37.38\% & 43.84\% & 33.53\% & 19.78\% \\ \hline
200 epochs & 49.75\% & 40.42\% & 45.83\% & 32.62\% & 15.06\% \\ \hline \end{tabular}
\end{table} 
Table 4: Percentage of times a generalizing DNN was selected from each metric and from the training with all 200 epochs. \begin{table}
\begin{tabular}{c c c c c c} \hline  & \multicolumn{2}{c}{**ASCADf**} & \multicolumn{2}{c}{**ASCADr**} & \multicolumn{2}{c}{**CHES CTF**} \\  & **HW** & **Identity** & **HW** & **Identity** & **HW** \\ \hline Fast GE & 2.66\% & 3.35\% & 1.19\% & 1.49\% & 2.74\% \\ Emp. GE & 20.70\% & 28.19\% & 18.59\% & 24.21\% & 20.61\% \\ GEEA & 11.48\% & 23.95\% & 9.31\% & 20.17\% & 13.56\% \\ MI & 9.28\% & 7.46\% & 7.81\% & 6.30\% & 11.28\% \\ \hline \end{tabular}
\end{table} 
Table 3: Average time overhead of different early stopping methods. For the ASCADr dataset, the results for FGE are also very promising, as shown in Figure 3. For the Hamming weight leakage model, FGE provides the best results, followed by the mutual information metric. In the case of the identity leakage model, the best result is obtained with all 200 epochs, showing that this number of epochs is appropriate for this best model found through a random search. The best results are obtained with the FGE metric when early stopping is considered. Figure 4 provides results for the CHES CTF dataset. The FGE metric provides the second-best results after GEEA. The results for the CHES CTF dataset are only shown for the Hamming weight leakage model, as this dataset provides bad results with the identity leakage model, as discussed in [9]. Furthermore, the performance of the best models selected from empirical GE as an early stopping metric provided less efficient results. As already mentioned in [15], empirical GE requires a very large validation set, and a more stable GE estimation can be obtained with the selection of larger validation sets.