# Deep Learning-Based Step Size Determination for Hill Climbing Metaheuristics
## 3 Methodology
### Building the Machine Learning Model  
A simple dense Deep Feedforward Neural Network was used for the study. Based on some preliminary tests, the architecture was defined as follows:  
\begin{table}
\begin{tabular}{l l l l l} \hline \hline  & **Average** & **StdDev** & **Min** & **Max** \\ \hline Iteration count & 19,635 & 17,547 & 1394 & 17,547 \\ Best fitness value & \(5.8\times 10^{-15}\) & \(6.5\times 10^{-16}\) & \(4.5\times 10^{-15}\) & \(7.4\times 10^{-15}\) \\ \hline \hline \end{tabular}
\end{table}
Table 1: Attributes of the training database.  
* Input layer--\(2k\) number of input neurons.
* Hidden layer--\(500\) dense neurons, using sigmoid activation.
* Output layer--\(1\) output neuron using sigmoid activation.  
The input layer is fed by the following vector: \([f_{i_{i}}e_{i},f_{i-1},e_{i-1},\dots,f_{i-k+1},e_{i-k+1}]\), where \(i\) is the number of the iteration, \(f_{j}\) is the fitness of the position of the \(j\)-th iteration, and \(e_{j}\) is the chosen \(e\) value at the \(j-1\)-th iteration.  
The output is the prediction of the best \(e\) value for the \(i\)-th iteration. As the \(e\) value can vary over a wide range (in this case, between \(10^{-9}\) and \(10\)), it was normalized by the following function:  
\[norm(e)=(log_{10}e+9)/10 \tag{8}\]  
A training dataset is built on the synthetic dataset presented in Section 3.3. The first \(k-1\) iterations of all HC executions were skipped, and after that, vectors containing the given input data format were built for each subsequent iteration. The training dataset consists of 392 M records. The chosen depth (\(k\)) value was 5.  
The network is trained using the Adam optimizer with a learning rate of 0.001 for 500 epochs. The entire database has been used for training, with no risk of over-fitting due to the massive size of the database (compared to the small capacity of the network). Using synthetically generated data without any noise also helps to avoid over-fitting.  
The limitation of the ML model trained in this way is that it is expected to work efficiently only in the search space used to generate the training database. This is the price to pay for replacing the otherwise general HC metaheuristic with a domain-specific HC algorithm, which is, in turn, expected to perform better in this narrower context.  
To evaluate the novel method proposed, the assembled hybrid ML-HC algorithm was examined in depth. The built-in FNN model itself was only evaluated with some preliminary tests. This resulted in the choice of three hidden layers. As can be seen in Figure 1, with one hidden layer, the network was not able to achieve good predictions, with two and three hidden layers; the networks showed similar results (slightly to the advantage of three); and with the introduction of the fourth layer, the network training slowed down considerably.  
Since the training of different network architectures required a lot of resources and time, finding the perfect architecture and optimizing the training hyperparameters are outside the scope of this paper. Obviously, there may be better architectures (for example, recurrent neural networks), and the size of the network may be smaller. But this network  
Figure 1: Loss by epoch for different numbers of hidden layers.  
was eligible for the goal of this paper--to prove that it is possible to predict the \(\epsilon\) value based on the data of previous steps--and this method can make the HC algorithm more efficient. A future research topic that can be considered is finding the appropriate ML model for this purpose.