## 4 Machine Learning Methodology
### Model Training and Optimization  
#### 4.2.1 Hyperparameter Optimization  
Hyperparameter optimization is critical for enhancing the performance and generalizability of machine learning models. Unlike model parameters, which are learned during training, hyperparameters are preset and govern key aspects such as the learning rate, model complexity, and regularization. Appropriate tuning of these hyperparameters can effectively prevent overfitting, accelerate convergence, and improve the model's robustness across diverse datasets. Consequently, through hyperparameter optimization, models can achieve more accurate and stable performance on complex tasks, thereby enhancing their applicability in practical scenarios.  
A common approach to hyperparameter optimization involves the use of the particle swarm optimization (PSO) algorithm. Compared to directly employing machine learning models for prediction, the integration of PSO significantly enhances the global search capability for optimal model parameters, thereby improving prediction accuracy and reducing the risk of overfitting. Moreover, the PSO optimization process is model-independent, making it broadly applicable to various machine learning algorithms with strong generalizability. PSO mimics the search behavior of particles within the solution space to identify the optimal combination of hyperparameters. In the context of machine learning, hyperparameter optimization is a crucial step for improving model performance; PSO is particularly effective for global optimization over a broad search space, thereby avoiding the local optima often encountered with traditional grid or random search methods. PSO is an optimization technique inspired by the foraging behavior of bird flocks, where a population of particles collaborates by flying through the solution space to locate the global optimum. In this algorithm, each particle represents a potential solution and explores the space by updating its position and velocity. The velocity update is determined by a combination of inertia weight, the particle's individual best-known position (pBest), and the global best-known position (gBest). Through iterative adjustments of particle velocities and positions, the swarm gradually converges to the optimal solution, as described in Equations (3) and (4).  
\[v_{i}(t+1)=w\cdot v_{i}(t)+c_{1}\cdot r_{1}\cdot(pBest_{i}-x_{i}(t))+c_{2}\cdot r _{2}\cdot(gBest_{i}-x_{i}(t)) \tag{3}\]  
\[x_{i}(t+1)=x_{i}(t)+v_{i}(t+1) \tag{4}\]  
where \(v_{i}(t)\) represents the velocity of particle \(i\) at time \(t\), while \(x_{i}(t)\) denotes the corresponding position; \(pBest_{i}\) is the individual best position achieved by particle \(i\), and \(gBest\) is the global best position found across the entire swarm; the inertia weight \(w\) controls the influence of the previous velocity on the current update, whereas the learning factors \(c_{1}\) and \(c_{2}\) regulate the contributions of the individual best and global best positions, respectively; \(r_{1}\) and \(r_{2}\) are random numbers drawn from a uniform distribution over the interval [0, 1].  
The variation in fitness values during the hyperparameter optimization of each machine learning model using the PSO algorithm is illustrated in Figure 6. As shown, the fitness values of all models decrease significantly through iterative optimization and tend to stabilize before the 13 th generation. Among them, the BPNN model exhibits the lowest fitness value, suggesting its potential superiority over the other models. Upon completion of the iterations, the optimal hyperparameters for each machine learning model are retained and reintroduced into the respective models for subsequent selection of the optimal machine learning model.