## 4 Methodology  
The aim of this paper is to compare and analyse the hyperparameter optimisation process for the DQN algorithm using GAs with various selection, crossover, and mutation methods. By exploring different configurations of these GA methods, this study examines their impact on effectively optimising hyperparameters. This research seeks to assess how exploration in GAs influences the efficiency of hyperparameter optimisation, highlighting the role of specific hyperparameter values in improving the performance of the DQN model. The GA approach was chosen for its ability to perform a comprehensive search in the hyperparameter space, facilitating an in-depth analysis of configurations that enhance agent success. Additionally, GA was selected over other evolutionary algorithms due to its simplicity, well-documented performance, and ease of implementation. A robust set of operators supports effective exploration, enabling satisfactory results with relatively simple parameter settings, without the need for introducing more complex modifications. The research methodology involved analysing configurations of basic selection, crossover, and mutation techniques in genetic algorithms, with a particular focus on the level of exploration within hyperparameter value ranges and the performance achieved by the agent with optimised hyperparameters for a given configuration in the final generation of the population. In our approach, we compare different combinations of selection and genetic operators, evaluating how exploration of hyperparameter value ranges impacts the efficiency of the DQN agent in the optimisation process. The motivation for conducting this research stems from the lack of studies that offer a comprehensive comparison of multiple GA operator configurations; existing studies have typically focused on specific operators. This issue is particularly relevant to reinforcement learning (RL), as most studies have primarily focused on other machine learning paradigms. The proposed approach to this study can be divided into the following stages:  
* Selecting methods for selection, crossover, and mutation;
* Choosing the RL algorithm and the suitable environment;
* Selecting the hyperparameters to be optimised;Determining the appropriate metric for evaluating the final outcome of each configuration;
* Choosing techniques for analysing the obtained results. The selection of appropriate methods in GA is crucial for optimising their hyperparameters, and a well-chosen set of methods significantly affects the final outcome of the algorithm. Despite the existence of more advanced techniques, the decision was made to use basic selection, crossover, and mutation methods presented in Section 2.2. This choice is justified by the fact that, despite their simplicity, these techniques effectively maximise the fitness function and allow for an easy assessment of the exploration level within the search space. Furthermore, it was decided to apply the elitism approach, as described in Section 2.2, which allows the maintenance of a constant number of chromosomes across successive iterations of the GA. The choice of the RL algorithm was motivated by its complexity and the number of hyperparameters, which have an explainable impact on the result achieved. It was essential that its hyperparameters not only pertain to shallow reinforcement learning but also involve aspects of neural networks. Since the Q-Learning algorithm, discussed in Section 2.1.1, has a limited number of hyperparameters, the decision was made to optimise those of DQN, which enhances Q-Learning with neural networks. In addition, a replay buffer was employed, which introduces an extra hyperparameter related to its size. This allowed the resulting algorithm to include hyperparameters with values that were not limited to numeric values but also included discrete values, as outlined in Section 2.1.2. The CartPole environment was selected for its mechanisms that ensure consistent reproducibility.