## 2 Related Work
### Application of Nature-Inspired Heuristic Algorithms  
In recent years, heuristic intelligent algorithms have been widely applied in various fields. These algorithms utilize optimization methods such as particle swarm optimization algorithm [27], genetic algorithm [28], and reinforcement learning [29] for the automatic design and optimization of neural network parameters.  
Ref. [30] proposes a hybrid algorithm, MPA-SCA, for parameter selection of hybrid active power filters (HAPF) in power plants. This algorithm combines the Marine Predator Algorithm (MPA) and Sine Cosine Algorithm (SCA) to optimize the performance of HAPF. The proposed algorithm is tested in various industrial power plants and the results show that it outperforms other algorithms in mitigating power harmonics. The study concludes that the proposed algorithm has the potential to be a valuable tool for designing and optimizing HAPFs in the power industry.  
Ref. [31] introduces a novel metaheuristic approach based on cuckoo search for deep learning-based depression prediction. This method is tested on a Twitter post dataset, achieving an accuracy of 87.5%. The results demonstrate the potential of this approach in utilizing social media data for predicting depression.  
Heuristic algorithms are widely applied in the field of microwave/RF as well. Due to the lack of analytical solutions or well-defined mathematical formulas in microwave circuit design optimization methods and the involvement of a highly complex optimization environment, these heuristic algorithms provide a flexible exploratory approach and have the ability to escape local optima. Therefore, they are well-suited to global optimization design of microwave circuits.  
Ref. [32] applied the particle swarm optimization algorithm to microwave circuit design, specifically optimizing microstrip couplers and single parallel short-circuit matching circuits. Simulation tests confirmed the effectiveness of this method. Ref. [33] proposed a universal precise design method for RF circuits based on the NSGA-II genetic algorithm, addressing the problem of multiple conflicting objectives in RF circuit design using a low noise amplifier as an example. Refs. [34; 35] proposed an automatic tuning framework for cavity filters based on reinforcement learning algorithms. This framework was tested in a simulation environment and verified its applicability under various custom tuning tasks. Ref. [36] utilized the deep deterministic policy gradient algorithm to automatically adjust and design the topology structure of high-sensitivity microwave microfluidic sensors, resulting in a significant improvement in sensor sensitivity.  
However, these heuristic intelligent search algorithms still require a large number of iterative optimizations to explore the entire design space. In microwave circuit design, each iteration of optimization requires evaluation based on electromagnetic simulations, resulting in significant computational overhead. In this work, we address this issue by utilizing neural network modeling techniques. Additionally, building upon previous research, this study explores the extension and application of the PPO reinforcement learning algorithm in microstrip filter design tasks. Furthermore, we accelerate the training process by incorporating a reward function based on engineer's expertise, thereby improving learning efficiency.