# Deep Learning-Based Step Size Determination for Hill Climbing Metaheuristics
## 4 Results  
### ML-Based Step Count  
The original idea of the ML-based hybrid method was to use the BF method to set the \(\epsilon\) value for the first \(k\) steps and switch to the ML-based \(\epsilon\) estimation for further steps. In practice, however, this simple solution did not always work. For an understanding of this, it is worth noting that the step size usually decreases continually during the execution of a HC algorithm. At the beginning of the search, the exploration phase has the advantage of high \(\epsilon\) values, but later, it is better to decrease this value for the exploitation phase. This phenomenon is visible in the training database as follows: smaller fitness and \(\epsilon\) inputs usually lead to small \(\epsilon\) predictions (meaning both the absolute values and the changes in the fitness and \(\epsilon\)). In rare cases, this causes an erroneous loop as follows: (a) as it is just a prediction, the ML model sometimes underestimates the step size; (b) small \(\epsilon\) prediction leads to a small step size and a resultant small fitness change; and (c) based on these small values, the ML model tends to predict more smaller step sizes and so on. In practice, this manifests as the algorithm starting to take smaller and smaller steps at some undefined point, slowing down the convergence.  
An algorithm representing alternating behavior has been developed to avoid this loop. This variant starts with the BF method for the first \(k\) iterations and switches to the alternating mode. In this mode, it performs \(l\) steps using the ML predictions, and after these, one step is carried out using the BF method. If the ML prediction cannot give a better position, it immediately tries one Brute Force step. If this also fails, the algorithm is stuck in a local minima. If not, the search will continue with ML-based predictions.  
To find the optimal number of ML steps, an experiment was carried out using different values, containing small \(l\) values (\(1\ldots 10\)) and large ones (\(20\), \(50\), \(100\), \(200\), \(500\), \(1000\), and \(5000\)). Based on 20,000 Hybrid ML-HC executions, the results show (Table 2) that the optimal number is \(5\). However, the achieved best fitness values are very similar, and there are huge differences in the number of iterations, which will be discussed later.  
The HC runs for this and subsequent tests were completely independent of the database used to teach the neural network.  
\begin{table}
\begin{tabular}{l c c c} \hline \hline \(I\) **Value** & **Fitness Average** & **Fitness Median** & **Fitness std.dev.** \\ \hline
1 & \(5.91\times 10^{-15}\) & \(5.90\times 10^{-15}\) & \(5.78\times 10^{-16}\) \\
2 & \(5.77\times 10^{-15}\) & \(5.76\times 10^{-15}\) & \(6.63\times 10^{-16}\) \\
3 & \(5.86\times 10^{-15}\) & \(5.87\times 10^{-15}\) & \(6.20\times 10^{-16}\) \\
4 & \(5.92\times 10^{-15}\) & \(6.00\times 10^{-15}\) & \(6.46\times 10^{-16}\) \\
5 & \(5.76\times 10^{-15}\) & \(5.69\times 10^{-15}\) & \(6.10\times 10^{-16}\) \\
6 & \(5.78\times 10^{-15}\) & \(5.78\times 10^{-15}\) & \(6.51\times 10^{-16}\) \\
7 & \(5.89\times 10^{-15}\) & \(5.87\times 10^{-15}\) & \(6.49\times 10^{-16}\) \\
8 & \(5.87\times 10^{-15}\) & \(5.87\times 10^{-15}\) & \(6.49\times 10^{-16}\) \\ \hline \hline \end{tabular}
\end{table}
Table 2: Best fitness values for different numbers of ML steps (\(l\) value).