## 4 Machine Learning Techniques
### Combined Adaptive-Network-Based Fuzzy Inference System and Particle Swarm Optimization (ANFIS-PSO)  
ANFIS is known to be a useful ML approach to solve function approximation problems [73]. ANFIS, which uses ANN standard training algorithms, is structured with if-else rules and input-output fuzzy joined data [74]. To simulate complex nonlinear mapping, a trained ANN and fuzzy logic work together in different imprecise and unpredictable noisy spaces [75]. Further aspects about ANFIS are reported in [27]. The PSO is based on competition and collaboration among particles [76; 77]. It performs a local search and leads to fast convergence in minimizing the objective function,  
Figure 10: GMDH: dependence of the RMSE at the end of training on the maximum number of layers in the ANN, and on data proportioning. Figure 9: GMDH: dependence of the RMSE at the end of training on the maximum number of layers in the ANN, and on the value of the selection pressure. see [78; 79]. The most striking advantage of this optimization algorithm over others is that the use of swarming particles makes the process robust against the solution getting entrapped into local minima of the handled objective function, since each particle renews its position and pathway based on its and others' information. In this work, PSO has been used to provide the membership functions of ANFIS, assumed to be Gaussian ones, thereby decreasing the computation costs for a given topology of ANFIS itself. A thorough explanation on the combined ANFIS-PSO algorithm can be found in [80]. In order to enhance the performance of this algorithm, the critical hyperparameters that need to be set are the number of iterations, or in other words, the stopping criterion, and the number of particles in the population handled at each iteration. Although exact rules for the swarm size selection are not reported in the literature, the main issue linked to it is that, when the problem dimension increases, the swarm size does increase as well, and may quickly become unmanageable [81]. By setting the maximum number of iterations to 5000 and the number of particles varying in a range between five and 95, Figure 11 shows the evolution of RMSE. It can be seen that a number of iterations equal to 5000 looks appropriate to converge to the final solution, independently of the number of particles. As depicted in the inset of Figure 11, after 5000 iterations, the solution linked to 65 particles has shown the best performance. The initial and final values \(W_{1}\) and \(W_{2}\) of the inertia weight also need to be set. This weight provides a trade-off between the concentration of particles in their neighborhoods, and the exploration of new points in the parameter space. Higher values of the weight allow more new points to be explored, while lower values allow the concentration capability of PSO to be boosted. The value of this hyperparameter has been linearly reduced during the run, so that the search effort is essentially focused on the exploration at the initial steps (governed by \(W_{1}\)), while it is focused more on exploitation at the latter phases (governed by \(W_{2}\)) of the run. Figure 12 shows the sensitivity of the PSO performance to \(W_{1}\) and \(W_{2}\). In the graph, different colors are related to different values of \(W_{1}\), whereas different line formats are associated to different values of \(W_{2}\). It clearly emerges that the red lines associated to \(W_{1}=0.8\) provide the best performance. Furthermore, the continuous lines linked to  
Figure 11: ANFIS-PSO: evolution of the RMSE during the optimization by PSO, as a function of the number of particles. \(W_{2}=0.2\) give rise to the minimum final value of RMSE and, in all the cases, to the steepest descents in its evolution.