## 2 System Model and Problem Formulation
### Proposed 6 DChOA-DRL  
In this paper, we propose utilizing DRL, which combines traditional RL with the power of DNNs, to solve complex decision-making tasks. The core concept of DRL is to leverage the ability of RL to learn optimal policies through interaction with an environment, while DNNs serve as function approximators to handle high-dimensional input spaces [44]. This combination allows the agent to make more sophisticated decisions in dynamic, uncertain environments. In a typical DRL framework, the RL component focuses on learning a policy that maximizes cumulative rewards. The agent interacts with the environment by observing its current state, selecting actions, receiving rewards, and updating its knowledge to improve future decision-making. The interaction loop continues until the agent converges on an optimal or near-optimal policy. The agent's policy, which determines the best action in a given state, is represented by a DNN. This DNN is trained to approximate a value function (Q-value). The role of the DNN in DRL is crucial for function approximation. In traditional RL methods, tabular approaches are often used, but these become inefficient in environments with large or continuous state spaces. Here, the DNN replaces the table by generalizing over the input space. Specifically, the DNN is used to approximate the Q-value function in a deep Q-Network (DQN), where the network takes the current state as input and outputs a Q-value for each possible action. The network is trained via backpropagation to minimize the error between predicted Q-values and target values, which are computed using the Bellman equation. By using a DNN, the agent can learn complex mappings between states and actions, making it scalable for large environments. Figure 5: Position update in the proposed 6 DChOA. The optimization of hyper-parameters in DRL is crucial for achieving optimal performance. Hyper-parameters such as the weights, biases, learning rate, discount factor (\(\gamma\)), \(\varepsilon\)-greedy, number of hidden layers, and number of neurons per layer directly influence the learning speed, stability, and effectiveness of the DRL agent. If these parameters are not properly calibrated, the agent may either fail to converge, converge to a suboptimal solution, or learn very slowly. For instance, inappropriate learning rates can cause the neural network to either overshoot the optimal solution or converge too slowly. Similarly, the discount factor controls the agent's balance between short-term and long-term rewards, and a poorly tuned value can lead to suboptimal decision-making. Another critical aspect is the exploration-exploitation trade-off, which is controlled by the \(\varepsilon\)-greedy policy. The exploration rate determines how much the agent explores new actions versus exploiting known, high-reward actions. If \(\varepsilon\) is not well optimized, the agent might explore too little or explore too much. Additionally, the architecture of the neural network (such as the number of hidden layers and the number of neurons per layer) affects the model's ability to approximate complex value functions and policies. Networks that are too shallow might not capture enough complexity, while overly deep networks can lead to overfitting or increased computational cost. Given the complexity of tuning these hyper-parameters manually, we propose using the 6 DChOA for automatic hyper-parameter optimization in DRL. This unique combination makes the 6 DChOA well suited for navigating the complex, high-dimensional space of hyper-parameters in DRL. In the proposed 6 DChOA-DRL algorithm, the 6 DChOA optimizes key hyper-parameters like the weights, biases, learning rate, and architecture of the DNN (hidden layers and neurons per layer). At each iteration, the 6 DChOA evaluates different configurations of these hyper-parameters by training the DRL agent with those values. The objective is to maximize the cumulative reward, which serves as the fitness function in the 6 DChOA.