## 2 Proposed Approach
### RL-Based Optimization Method
#### 2.2.1 Policy Generation
The reward function introduced in this research is specifically crafted to address the dual objectives of exploration and exploitation, crucial for the efficient navigation of complex search spaces in the quest to discover efficient optimization policies. This function is designed to modulate the behavior of the RL policy by incorporating both the improvement in objective values over iterations and a normalized measure of performance relative to a global benchmark. The adjusted logarithmic reward, \(\mathcal{R}_{1}\), plays a pivotal role in fine-tuning the particles' incentive to narrow down the search vicinity as they converge towards the region of optimal solutions. Defined as a logarithmic function of the normalized objective value at each iteration, \(\mathcal{R}_{1}\) effectively scales the reward based on the proximity to the optimum, which is always shifted to 0 in our case. This reward structure has a global effect since it relates to the improvements undertaken by a particle relative to the worst-known function value, \(W\). The second term in the logarithmic function of Equation (9), \(|f(\mathbf{x}_{i})^{k}|\), has a maximum value of 0 for the best agent and a value of 1 for the worst particle. The constant, \(\alpha\), in addition to controlling the precision of the algorithm, also prevents undefined reward functions when dealing with the globally best particle. The logarithmic nature ensures that the reward gradient becomes smoother near the optimum, thus preventing abrupt changes in particle behavior and facilitating a more controlled convergence. Overall, this reward structure encourages a guided exploration by discouraging the particles from exploring regions close to the globally worst areas of the design space. As the value of \(|f(\mathbf{x}_{i})^{k}|\) approaches very close to zero, the \(\mathcal{R}_{i,1}\) approaches a constant value of \(-\log\alpha\), suggesting that the overall reward structure (Equation (8)) becomes less influenced by the logarithmic term. The second component, \(R_{2}\), introduces a direct measure of improvement by comparing the objective values at successive iterations. This term inherently promotes a straightforward but effective incentive: particles are rewarded for actions that result in an immediate improvement in the objective value, with the magnitude of the reward being proportional to the relative improvement. This mechanism ensures that particles are inherently motivated to pursue trajectories that yield positive outcomes, aligning with the core objective of any optimization task. Moreover, this reward structure introduces a dynamic aspect to the reward structure, enabling a self-adjusting incentive system that naturally discourages stagnation and rewards continuous improvement. In computing the overall reward, these two rewards are summed and scaled by \(\frac{k}{L}\), thus ensuring that the value of the reward is contextualized within the frame of the entire optimization process. This scaling has the effect of tempering the reward in the early phases where exploratory actions are more prevalent and conversely amplifying it as the optimization episode matures. Furthermore, by including the two reward functions, \(\mathcal{R}_{1}\) and \(\mathcal{R}_{2}\), the policy learns to strike a critical balance between exploration and exploitation: \(\mathcal{R}_{1}\) encourages exploration, albeit in a targeted fashion, while \(\mathcal{R}_{2}\) encourages exploitation. In this way, policies that stimulate both aspects simultaneously are likely to obtain higher rewards and therefore are encouraged in future training episodes. For instance, policies that result in the particles being stuck in a locally optimal location will lead to high valuesof \(\mathcal{R}_{1}\) but suboptimal rewards from \(\mathcal{R}_{2}\). Therefore, \(\mathcal{R}_{1}\) can be thought of as encouraging policies that reach the vicinity of the global optimum, while \(\mathcal{R}_{2}\) encourages exploitation once the particles converge towards this region.