## 1 Introduction
For instance, searching a compositional parameter space for a new material requires following a complex set of steps, including the experimental setup, sample preparation, and material testing. In other contexts, computational fluid dynamics (and other simulation approaches) may be used to perform function evaluations, and, depending on the computational level of fidelity, a single simulation may require several hours or days to complete on several processors. Therefore, for such applications with expensive function evaluations, design optimizers must be designed to work with limited swarm sizes; otherwise, the optimization process becomes impractical due to excessive computing runtimes. For problems where a small swarm size is necessarily imposed due to the runtime-related considerations described above, swarm-based optimizers often suffer from premature convergence to local optima [12]. In cases where a limit is placed on the number of function evaluations due to a fixed time or computational budget, swarm-based optimizers may not have sufficient attempts to reach the vicinity of the global optimum. Another downside is that the performance of swarmed-based optimizers for various problems is often sensitive to the choice in the model parameters or constants (e.g., maximum velocity, communication topologies, inertia weights, etc.). These constants are required to be selected by the user a priori and do not necessarily generalize adequately across problems [11]. Thus, these parameters need to be carefully tuned to maintain acceptable performance for different optimization problems [3]. To address these limitations, researchers have explored various enhancements to swarm-based algorithms. One promising approach is surrogate-assisted or meta-model-based evolutionary computation. As discussed by [PERSON] [13], these methods use efficient computational models to approximate the fitness function in evolutionary algorithms, effectively reducing the number of expensive function evaluations required. Surrogate models have shown success in single- and multi-objective optimization, as well as in dynamic, constrained, and multimodal optimization problems [13]. Another innovative concept in stochastic search is the non-revisiting scheme, explored by [PERSON] et al. [14]. This approach uses search history to prevent duplicate solutions and enables adaptive parameter-less local search and fitness landscape estimation. Such methods can potentially improve the efficiency of swarm-based optimizers by avoiding redundant evaluations of the previously explored areas of the search space [14]. Given these advancements, numerous researchers have pursued adding enhancements to swarm-based algorithms in areas relating to their structural design, operational mechanisms, and search methodologies through the application of machine learning techniques and other methods. These enhancements have led to significant improvements in performance [15]. For instance, some past studies have delved into the indirect application of reinforcement learning (RL) for swarm optimization processes, including parameter tuning, operator selection, algorithm optimization, and solution development. In a notable study, [PERSON] et al. [16] presented a reinforcement-learning-based method for parameter adaptation in Particle Swarm Optimization (PSO), termed RLPSO, which boosts PSO's convergence rate by implementing a network that adjusts the PSO coefficients. This method utilizes the deep deterministic policy gradient (DDPG) technique to estimate the PSO operator values, considering particle diversity and the time spent. Similarly, [PERSON] et al. [17] introduced an RL-enhanced PSO approach to address the flexible job shop scheduling problem, employing Q-learning to dynamically modify the inertia weights and acceleration coefficients, thereby optimizing the balance between global exploration and local exploitation.