## 6 Mini Case Study: Stabilisation Under Incomplete Observations  
Balancing is a fundamental control problem that arises in various engineering applications, including robotics, where systems must maintain stability despite external disturbances and sensor limitations. The CartPole environment serves as a simplified benchmark for such balancing tasks, providing a controlled setting to study decision-making under uncertainty. While CartPole typically offers complete state information, real-world systems are often subject to sensor malfunctions, communication failures, or noise, leading to incomplete or corrupted observations [38]. In this study, partial observability is introduced into the CartPole environment through sensor dropouts to simulate missing information. The objective is to assess the impact  
Figure 12: Scatter plots of hyperparameters vs. agent’s score, showing performance trends. (**a**) Learning rate vs. agent’s score. (**b**) Target update vs. agent’s score. (**c**) Replay buffer size vs. agent’s score. (**d**) Gamma vs. agent’s score. Figure 13: SHAP values showing the impact of hyperparameters on the model output. of these dropouts on the performance of an RL agent optimised using a GA. Specifically, the best-performing GA configuration--tournament selection, multi-point crossover, and boundary mutation, as identified in Section 5.4.1--is evaluated under conditions of partial observability. This section discusses the experimental setup, presentation of results, and evaluation of model performance under varying levels of sensor dropout. Table 16 summarises the modifications introduced to the standard CartPole-v1 environment using the gym.Wrapper to simulate missing sensor measurements. At each timestep, individual elements of the four-dimensional state vector were randomly nullified with a probability of 5% in the first scenario and 20% in the second scenario. The DQN algorithm was used for reinforcement learning, with the GA optimising four key hyperparameters: learning rate, target update frequency, replay buffer size, and gamma. The best-performing GA configuration, which achieved the highest agent performance in the fully observable environment, was directly applied to these altered conditions. All other hyperparameters and training settings were retained from the baseline experiments. Figure 14 illustrates the optimisation process of the GA across generations under different levels of sensor dropout: 0%, 5%, and 20%. The objective function value represents the agent's performance, with higher values indicating more effective optimisation. The GA configuration employed in this study was previously identified as the best-performing setup, achieving the highest agent score in the fully observable environment (0% dropout). However, as the probability of sensor dropout increases, the optimisation process exhibits a significant decline in performance. * P(loss) = 0% (blue line) The GA efficiently optimises the objective function, with stepwise improvements observed across generations. The function value increases \begin{table}
\begin{tabular}{c c} \hline \hline
**Parameter** & **Value** \\ \hline Environment & CartPole-v1 (modified using gym.Wrapper) \\ State vector dimension & 4 \\ Probability of nullifying each state element & 5\%, 20\% \\ Applied RL algorithm & DQN \\ Selection method & Tournament \\ Crossover method & Multi-point crossover \\ Mutation method & Boundary mutation \\ Optimised hyperparameters & Learning rate, target update frequency, replay buffer size, gamma \\ Baseline hyperparameters & Retained from original experiment \\ \hline \hline \end{tabular}
\end{table} 
Table 16: Summary of the CartPole environment modifications and applied GA configuration. Figure 14: Progression of the best score over generations for different probabilities of state information loss (0%, 5%, 20%). steadily, reaching a peak above 800 after approximately 125 generations.