# Biomedical Image Classification via Dynamically Early Stopped Artificial Neural Network
## 4 Numerical Experiments
### Biomedical Image Classification
#### 4.2.1 Hyperparameters Setting  
Because the performance of a stochastic gradient method is strictly related to the configuration of its hyperparameters, this section aims to fix the best hyperparameters setting for the algorithms employed to train the ResNet18. This preliminary study is carried out on the MedMNIST2D OrganSMNIST dataset and the best found hyperparameters configurations will be used for all the other experiments. First of all, we compare the performance of the ES method for different values of the patience \(p\). In particular, given \(\eta_{0}=10^{-3}\) and \(n_{0}=128\), we report in Table 2 the values of the accuracy reached by ES with \(p\) equal to 5, 20 and 30. In the same table, the results corresponding to the Original optimizer are also reported (for the same setting of \(\eta_{0}\) and \(n_{0}\)). Figure 2: Some examples from MedMNIST2D OrganSMNIST (**left half**) and OCTMNIST (**right half**). The value of \(p\), which ensures the best accuracy on the test set, is 20. In the following, \(p\) is always set to this value for the ES method. Setting \(\eta_{0}\) and \(n_{0}\) for the Original and the ES Methods  
In order to properly tune the LR and the MBS for both the Original and the ES schemes, we performed several experiments with different settings, illustrated in Table 3.  
From the results of Table 3, the best hyperparameters setting for both the Original and the ES approaches is \(\eta_{0}=10^{-2}\) and \(n_{0}=64\). We remark that to find this setting was very demanding in terms of computational costs. Robustness of the LRBS Method against Hyperparameters  
The proposed method aims to get rid of the dependence on its intrinsic hyperparameters while maintaining a high performance. In this section, we investigate the response of the LRBS method to the variation in the values of the hyperparameters used in Algorithm 1. In particular, we consider different values for: \(c_{1},c_{2},dim\) and \(\gamma\). It is worth highlighting that we do not need to also properly tune the values for the patience, the LR and the MBS as performed for the Original and the ES schemes. Indeed, the LRBS algorithm automatically adjusts the values of these hyperparameters along the epochs. For this reason, we just consider \(p=20\), \(\eta_{0}=10^{-2}\) and \(n_{0}=64\): we select a quite large value for both the patience \begin{table}
\begin{tabular}{l c c c} \hline \hline
**Configuration** & **Test Accuracy (\%)** & \(\sigma\) **(\%)** & **\# Epochs** \\ \hline Original & 88.14 & 0.20 & 50 \\ \hline ES (\(p=5\)) & 88.22 & 0.03 & 3.4 \\ \hline ES (\(p=20\)) & 88.51 & 0.53 & 21 \\ \hline ES (\(p=30\)) & 87.84 & 0.51 & 27.6 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Results on MedMNIST2D OrganSMNIST dataset in a maximum of 50 epochs. \(\sigma\) is the standard deviation and each result is the mean of five trials. \begin{table}
\begin{tabular}{l l c c c} \hline \hline
**Hyperparameters** & **Configuration** & **Test Accuracy (\%)** & \(\sigma\) **(\%)** & **\# Epochs** \\ \hline \(\eta_{0}=10^{-2}\), \(n_{0}=dim=64\) & Original & 92.80 & 0.22 & 50 \\  & ES & 92.68 & 0.36 & 23.2 \\ \hline \(\eta_{0}=10^{-2}\), \(n_{0}=dim=128\) & Original & 92.49 & 0.31 & 50 \\  & ES & 92.58 & 0.33 & 16.2 \\ \hline \(\eta_{0}=10^{-2}\), \(n_{0}=dim=256\) & Original & 91.54 & 0.53 & 50 \\  & ES & 91.51 & 0.42 & 13.6 \\ \hline \(\eta_{0}=10^{-3}\), \(n_{0}=dim=64\) & Original & 91.05 & 0.82 & 50 \\  & ES & 91.08 & 0.72 & 19.8 \\ \hline \(\eta_{0}=10^{-3}\), \(n_{0}=dim=128\) & Original & 88.61 & 0.60 & 50 \\  & ES & 88.62 & 0.59 & 18.8 \\ \hline \(\eta_{0}=10^{-3}\), \(n_{0}=dim=256\) & Original & 86.89 & 0.41 & 50 \\  & ES & 86.86 & 0.37 & 24.6 \\ \hline \(\eta_{0}=10^{-4}\), \(n_{0}=dim=64\) & Original & 86.06 & 0.48 & 50 \\  & ES & 86.19 & 0.50 & 17.8 \\ \hline \(\eta_{0}=10^{-4}\), \(n_{0}=dim=128\) & Original & 84.84 & 0.89 & 50 \\  & ES & 85.48 & 0.28 & 38.4 \\ \hline \(\eta_{0}=10^{-4}\), \(n_{0}=dim=256\) & Original & 83.16 & 0.52 & 50 \\  & ES & 84.55 & 0.67 & 38 \\ \hline \hline \end{tabular}
\end{table}