## 1 Introduction  
In recent years, reinforcement learning (RL) has emerged as a powerful tool to solve complex decision-making problems in various domains, including robotics, finance, and autonomous systems [1]. Deep Q-learning (DQN), one of the prominent RL algorithms, has demonstrated significant success by combining neural networks with the classic Q-learning approach [2]. However, the performance of such algorithms is highly dependent on the careful tuning of their hyperparameters, such as the learning rate, discount factor, and network update frequency. These hyperparameters influence the balance between exploration and exploitation, convergence rates, and overall effectiveness of the model [3].  
Hyperparameters are an integral part of machine learning, differing from other parameters in that they cannot be automatically determined from data but must be set externally. They have a significant impact on both the efficiency and the outcome of an algorithm. This is especially pertinent in reinforcement learning, where hyperparameter optimisation directly affects an algorithm's success in applications like autonomous driving, roboticarm control, and enhancing the responsiveness of language models such as ChatGPT [4,5]. Depending on the specific task, optimal hyperparameter values can vary widely, and there is no universal scheme for assigning them to different problems.  
Hyperparameter optimisation (HPO) methods, such as grid search (GS) and random search (RS), often require significant computational resources and may still fail to identify optimal configurations [6]. To overcome these limitations, genetic algorithms (GAs) have gained attention for efficiently exploring large hyperparameter spaces by mimicking natural selection processes [4,7]. While HPO is well studied in supervised and unsupervised learning, reinforcement learning (RL) remains underexplored, particularly for deep neural network-based models. This gap likely stems from RL's unique challenges, especially in how learning data are gathered [4,8].  
The aim of this paper is to compare and analyse the hyperparameter optimisation process for the DQN algorithm using GAs with various selection, crossover, and mutation methods. This paper explores different configurations of these methods and their impact on the effective optimisation of hyperparameters. The research goal is to assess how exploration in genetic algorithms influences the efficiency of hyperparameter optimisation and to investigate the role of specific hyperparameter values in improving the performance of the DQN model.  
The novelty of this paper lies in the approach of comparing selection, crossover, and mutation methods in genetic algorithms, with a particular focus on how exploration influences the hyperparameter optimisation of the DQN algorithm, particularly across large hyperparameter value ranges. While other studies focus on comparing general optimisation methods or different machine learning paradigms, this study emphasises how various configurations can enhance exploration, providing a more robust approach to identifying optimal hyperparameter values and thereby improving the performance of the DQN algorithm.  
The remainder of this article is structured as follows: Section 2.1 provides an overview of RL, focusing on Q-learning and DQN, and the role of hyperparameters in these algorithms. Section 2.2 introduces GAs, explaining their key mechanisms. Section 3 reviews the related literature on RL, GA, and HPO. Section 4 outlines the methodology of the experiment. Section 5 presents the experimental results and analyses the impact of GA configurations on HPO, while also examining the influence of specific hyperparameter values on model performance. Section 6 provides a mini case study on stabilisation under incomplete observations, where the effects of sensor dropout on the CartPole environment are investigated using the best GA configuration. Finally, Section 7 concludes this paper and suggests directions for future work.