## 3 Methodology
### The Proposed Metaheuristic-Based Feature and Hyperparameter Optimization
#### 3.4.2 HSMO Optimizer
As the algorithm progresses, the focus shifts more towards exploitation rather than exploration, allowing the algorithm to converge towards promising solutions and avoid excessive exploration. In the local leader phase, the initial selection of monkeys is random. However, it is important to understand that at the beginning of this phase, when \(pr\) is small, monkeys are selected for position updates with high perturbation rates in all dimensions, which promotes exploration. But over time as the pr value increases, the frequency of picked monkeys reduces, resulting in fewer and fewer updates, and the algorithm focuses more on exploitation to refine and improve the quality of the solutions. During the local leader decision phase, the pr value, which increases in each successive iteration, influences the probability in determining different update strategies for the monkeys in a group. When the pr value is smaller (specifically, if the generated random number is greater than or equal to pr), there is a possibility that a non-optimal minima hasbeen reached, and the solution may be stuck there. In such cases, random initialization is used to introduce exploration and to prevent the solution becoming stuck by using Equation (9). On the other hand, when the \(pr\) value is large, it suggests that the group has exceeded the local leader limit (threshold), indicating that the global and local leaders have likely converged to a desirable solution, so their parameters have more of a chance of being used for the updates, promoting exploitation of the current best solutions using Equation (14). Particle Probability:  
Particle probability controls the update of a position value during the optimization process and then generates a trial vector by mutating and recombining the population vectors. The probability is a value between 0 and 1, where a higher value indicates a higher probability of changing a parameter value. Binary SMO:  
As previously stated, because the new monkey locations are values in continuous form, the continuous values were translated to binary values that correspond to each other. A Boolean operator is used to accomplish this task of transferring continuous solutions in each dimension, which forces the monkey to discretize their movement. For Boolean parameters, the basic equations of the continuous SMO algorithm are modified using the logical operators AND, OR, and XOR. 1. Initialize:  
- Initialize the particle positions and velocities. - Set parameters like inertia weight (w), cognitive parameter (c1), social parameter (c2), and the number of particles (N). - Randomly generate initial positions and velocities for particles. - Create a DataFrame to store particle information: ID of particle, objective function value, global best position g, and parameters (analogic and continuous). - Calculate the initial objective function values for particles. - Set the best positions for each particle and global best position. 2. Iterative Optimization:  
- For a specified number of iterations or until an iteration is met:  
a. Update Velocities:  
- Calculate new velocities for each particle using the formula in Equation (8)  
b. Update Positions:  
- Update the positions of particles using the new velocities. - For continuous parameters, add the velocity to the position, as defined in Equation (2). - For binary parameters, use a sigmoid function to determine whether to flip the bit based on the velocity as defined in Equation (6). c. Evaluate Objective Function:  
- Calculate the objective function values for the updated positions of the particles. - Update the DataFrame with the new positions and objective function values. - Update the best positions for each particle if the new position is better. d. Update Global Best:  
- Determine if any particle's current position is better than the global best position. - Update the global best position if necessary, as defined in Equation (8). 3. Final Output:  
- Return the best solution found and its corresponding objective function value. ```
1.