## 5 Experiment and Results
### Experiment Challenges and Issues  
Compiling the GA code revealed that there was a problem related to randomness, which prevented the proper comparison of the algorithms. This problem affected the functionalities that used mechanisms to generate random numbers and the agent training function. To ensure comparability of results, a random seed was defined for libraries such as NumPy, random, os, and PyTorch. This approach allowed the GA to obtain an identical initial population for each iteration by combining the selection, crossover, and mutation methods. Additionally, it reduced the instability related to rounding of weight values in the agent's neural network, so that the fitness function values were mostly identical for the same hyperparameters. The only drawback of the introduced change was the increase in the computational complexity of the function using neural networks. The last problem that prevented efficient research was the duration of the hyperparameter optimisation process. The test adaptation function allowed one to find optimal solutions in a very short time. This problem occurred after changing the test function to the agent training function, whereby each chromosome was evaluated by the function that trained the agent for 500 episodes. Table 1 shows the compilation time for 499 iterations of the genetic algorithm for two different fitness functions. The first column lists values for the fitness functions: one being the negative Euclidean distance, and the other the agent's average reward over the last 100 training episodes, computed as \(\frac{1}{100}\sum_{i=401}^{500}R_{i}\). The Selection column describes the selection method used, with values indicating either roulette selection or tournament selection. The Crossover column specifies the crossover method applied, where arithmetic crossover and one-point crossover are included. Similarly, the Mutation column details the mutation method used for individuals, with boundary mutation and non-uniform mutation represented. The next column shows the time taken, in seconds, for the algorithm to complete 499 generations. The final column represents the speed at which the algorithm processed each generation. The obtained data show that the problem of code execution time does not lie directly within the implemented GA, but in the RL agent training process. In order to reduce the algorithm's compilation time, the number of generations was reduced to 200. The biggest problem resulted from the construction of the agent training process, because most of the time was spent waiting for the environment response, over which the user has little control. This time also resulted from the way the environment was explored. Knowing that exploration limits the agent from improving the learned solutions, it was decided to reduce the value of the variable \(\epsilon\) to less than 1, so that decisions were primarily driven by the agent's learned policy. Figure 1 illustrates the trend in average reward over 500 episodes of training a DQN agent in the CartPole environment, using an \(\epsilon\)-soft exploration strategy. The initial exploration probability was set to 1, decaying at a rate of 0.99 per episode, with a final probability of 0.01 for taking suboptimal actions. Early episodes show low rewards, typical of intensive exploration, while rewards increase as the agent learns from experience. Fluctuations between episodes 100 and 300 reflect ongoing exploration, with a significant reward spike around episode 150 followed by a decline as the agent continued exploring. Toward episode 400, rewards stabilise, indicating the agent's shift to exploiting learned strategies.