## 3 Experimental Results
### Experimental Results
#### 3.3.3 Experiments with the Used Local Search Optimizer  
An important issue of the proposed method is the selection of the local search optimizer, that will be applied periodically to chromosomes selected randomly. In the current work, a modified version of the BFGS method [63] was chosen, since it can satisfactorily handle the constraints placed on the network parameters for the optimization. However, an additional experiment was executed using different local optimization techniques. The results for the classification datasets are presented in Table 10, and the results for the regression datasets are shown in Table 11. The following notation is used in the experimental tables:  
1. The column ADAM denotes the incorporation of the ADAM local optimization method [18] in the current technique. 2. The column LBFGS denotes the incorporation of the Limited Memory BFGS (L-BFGS) method [98] as the local search procedure. 3. The column BFGS represents the incorporation of the BFGS method, modified by [PERSON] [63], as the local search procedure. \begin{table}
\begin{tabular}{c c c c} \hline \hline
**DATASET** & **ADAM** & **LBFGS** & **BFGS** \\ \hline APPENDICITIS & **14.60\%** & 14.60\% & 14.70\% \\ \hline AUSTRALIAN & **14.78\%** & 15.29\% & 14.80\% \\ \hline BALANCE & 16.89\% & 11.47\% & **8.66\%** \\ \hline CIRCULAR & 10.79\% & 8.15\% & **5.32\%** \\ \hline CLEVELAND & 49.10\% & 48.45\% & **47.93\%** \\ \hline DERMATOLOGY & 22.32\% & 23.03\% & **20.89\%** \\ \hline ECOLI & **48.15\%** & 49.48\% & 48.21\% \\ \hline FERT & **18.10\%** & 20.10\% & 20.50\% \\ \hline HABERMAN & **26.27\%** & 26.63\% & 26.70\% \\ \hline HAYES-ROTH & 36.23\% & 36.31\% & **31.77\%** \\ \hline HEART & 15.71\% & 15.00\% & **14.74\%** \\ \hline HEARTATTACK & 21.33\% & 20.07\% & 20.43\% \\ \hline HOUSEVOTES & 3.39\% & 3.39\% & **3.26\%** \\ \hline IONOSPHERE & **10.88\%** & 11.29\% & 11.92\% \\ \hline LIVERDISORDER & 31.80\% & 33.18\% & **31.77\%** \\ \hline MAMMOGRAPHIC & 16.71\% & 16.30\% & **15.81\%** \\ \hline PARKINSONS & **12.10\%** & 12.32\% & 12.53\% \\ \hline PIMA & 25.49\% & 24.97\% & **24.00\%** \\ \hline POPFAILURES & **6.20\%** & 6.31\% & 6.44\% \\ \hline REGIONS2 & 24.20\% & 24.06\% & **23.18\%** \\ \hline SAHEART & 28.87\% & 28.96\% & **28.09\%** \\ \hline SEGMENT & 50.16\% & 46.63\% & **43.12\%** \\ \hline SPIRAL & 46.33\% & 45.24\% & 43.99\% \\ \hline STUDENT & 4.05\% & **3.98\%** & 4.55\% \\ \hline TRANSFUSION & 24.74\% & 24.56\% & **23.43\%** \\ \hline WDBC & 6.30\% & 5.66\% & **4.41\%** \\ \hline \hline \end{tabular}
\end{table} 
Table 10: Experimental results with different local optimization techniques in the proposed method with application on the classification datasets. The bold notation marks the method with the lowest test error. The BFGS method achieved lower values for the test error in the majority of cases, and this is also evident in Figure 13, where a box plot is depicted for the classification datasets. \begin{table}
\begin{tabular}{c c c c} \hline
**DATASET** & **ADAM** & **LBFGS** & **BFGS** \\ \hline ABALONE & 4.73 & 4.49 & **4.33** \\ \hline AIRFOIL & 0.003 & 0.003 & **0.002** \\ \hline BASEBALL & 53.98 & 51.88 & **48.42** \\ \hline BK & **0.05** & 0.08 & 0.07 \\ \hline BL & 0.013 & 0.007 & **0.002** \\ \hline CONCRETE & 0.006 & 0.006 & **0.005** \\ \hline DEE & 0.24 & **0.23** & 0.23 \\ \hline HO & 0.012 & 0.012 & **0.01** \\ \hline HOUSING & 20.20 & 20.49 & **16.01** \\ \hline FY & **0.042** & 0.042 & 0.042 \\ \hline LASER & 0.012 & **0.005** & 0.005 \\ \hline LW & **0.011** & 0.011 & 0.012 \\ \hline MORTGAGE & 0.068 & 0.084 & **0.026** \\ \hline MUNDIAL & 0.033 & **0.029** & 0.034 \\ \hline PL & 0.031 & 0.023 & **0.022** \\ \hline QUAKE & **0.036** & 0.037 & 0.04 \\ \hline REALSTATE & 74.66 & 74.49 & **70.99** \\ \hline SN & 0.025 & 0.024 & **0.023** \\ \hline TREASURY & 0.114 & 0.084 & **0.066** \\ \hline TZ & **0.03** & 0.08 & 0.03 \\ \hline VE & **0.024** & 0.132 & 0.025 \\ \hline
**AVERAGE** & **7.35** & **7.25** & **6.69** \\ \hline \end{tabular}
\end{table} 
Table 10: _Cont._