# The Need for Speed: A Fast Guessing Entropy Calculation for Deep Learning-Based SCA
## 4 Fast GE for Early Stopping  
Running a hyperparameter search without pre-selecting efficient ranges for each hyperparameter may fail to find powerful attack models. A solution could be searching for small models with restricted search ranges, as proposed in [8; 21] or by setting the objective of the search as being a small model, as proposed by [PERSON] et al. [9]. Small models are usually self-regularized, but they still suffer from a limited fitting capacity, which is particularly problematic for noisy and protected targets. An alternative is to allow larger models and add regularization to prevent overfitting [29]. Although regularization improves model generalization, regularized models with increased size require more training epochs, reducing the efficiency in a hyperparameter search process. As the number of training epochs is a critical hyperparameter to be determined, early stopping may become a standard approach. To allow efficient early stopping with GE, we propose a fast GE calculation to reduce the empirical GE overheads. When used as an early stopping metric, FGE provides very small overheads to the training process, usually between 1.5% and 3.3%, while, e.g., empirical GE shows overheads between 18.59% and 28.19%, as reported in Section 5. Our idea consists in reducing the number of validation traces when computing GE for each processed epoch, which has multiple benefits in DL-based SCA. The pseudo-code showing how FGE is obtained is provided in Algorithm 1. As the algorithm shows, the main application of FGE is for the hyperparameter search process. ```
1:Set \(\Theta\) as the set of models
2:for new search attempt \(S\)do
3: Generate new hyperparameter set \(\mathcal{H}\)
4: Initialize model parameters \(\theta\)
5: Select a small validation set \(X_{val-fast}\)
6:for Epoch \(E\) in Epochs do
7: Train neural network model \(\mathcal{F}_{\mathcal{H}}(\theta,X_{prof},Y_{prof})\)
8: Compute FGE: \(GE_{fast}[E]=GE(\theta,X_{val-fast},Y_{val-fast})\)
9: Save model parameters \(\theta[E]\) at epoch \(E\)
10:endfor
11: Retrieve model parameters from best epoch: \(\Theta[S]=\theta[argmin(GE_{fast})]\)
12: Select a full validation set \(X_{val}\)
13: Compute \(GE(\theta_{best},X_{val},Y_{val})\) and the corresponding number of validation traces to reach GE = 1, \(N_{GE1}[S]\). 14:endfor
15: Return best model: \(\theta_{best\_model}=\Theta[argmin(N_{GE1})]\) (model that requires a minimum number of validation traces to reach GE = 1). ```  
**Algorithm 1** Hyperparameter search with early stopping and fast guessing entropy. If the model converges, the attack is successful, and GE for a small number of validation traces can also indicate the best epoch to stop training efficiently. Using large validation sets for the metric calculation may obscure the real performance of the model: a model that overfits may also slowly decrease guessing entropy to 1 after processing enough validation traces. In contrast, FGE is more sensitive to the model's performance change, thanks to its low usage of the validation traces. This situation is illustrated in Figure 1. As we can see in Figure 1, with \(Q=3000\) validation traces for empirical GE, the best epoch will be returned at the moment when GE is equal to 1. If the next epochs indicate a model that requires even fewer traces to succeed (which means better generalization), empirical GE will not capture that. On the other hand, using fewer traces allows us to obtain this convergence and recover the key with fewer attack traces, as shown in Figure 1. Of course, the question here is: why would this be a problem if reaching a GE of 1 allows an adversary to recover the key? We can observe two main problems in this scenario. First, empirical GE with more traces provides more overhead and limits the number of hyperparameter search attempts, preventing us from finding a model that eventually breaks the target (which is the example of the model found in Figure 1).