## 2 Materials and Methods
The algorithm is computationally inexpensive compared to other swarm intelligence algorithms, does not require storing or computing large matrices or finding solutions to complicated equations. However, due to the heuristic nature of the algorithm, the estimated optimum can sometimes not be as accurate, as the optimum that classical optimization algorithms can find. The drawback of the classical gradient-based or Hessian-based algorithms is that they often converge to local minima or maxima, although various techniques exist, that can prevent the premature convergence of such algorithms. To take the best from both worlds, we propose the combination of the evolutionary-inspired FSS algorithm and the two classical optimization methods. The first proposed algorithm consists of the population-based FSS described above in detail and of the [PERSON]'s method in optimization. The latter classical optimization algorithm is based on the [PERSON]'s method for finding roots of a differentiable function \(f\) by solving the \(f\Big{(}\vvet{x}\Big{)}=0\) equation. Such solutions are the stationary points of the considered function \(f\), which can either be minimum, maximum or saddle points [30]. In general, the iterative scheme for the [PERSON]'s classical optimization method is given by:  
\[\vvet{x}_{t+1}=\vvet{x}_{t}-\Big{[}\mathbf{H}f\Big{(}\vvet{x}_{t}\Big{)} \Big{]}^{-1}\
abla f\Big{(}\vvet{x}_{t}\Big{)}. \tag{7}\]  
where, \(t\) is the iteration number, \(\vvet{x}_{t}\) vector represents the solution on the \(t\)-th iteration, \(\Big{[}\mathbf{H}f\Big{(}\vvet{x}_{t}\Big{)}\Big{]}^{-1}\) denotes the inversion of the Hessian matrix, \(\
abla f\Big{(}\vvet{x}_{t}\Big{)}\) represents the gradient of the function \(f\) at the given point \(\vvet{x}_{t}\). The derivatives can be computed either symbolically or numerically, using finite difference approximation [31]. The iterative optimization process continues until the specified iteration limit is not achieved, or while \(\left|x_{i,t+1}-x_{i,t}\right|>\varepsilon\), where \(x_{i,t}\) represents the \(i\)-th component of the \(\overrightarrow{x}_{t}\) vector on iteration \(t\). In the proposed hybrid approach, we run \(iter_{max}\) iterations of the FSS algorithm first, and then, using the discovered \(\overrightarrow{x}_{best}\) solution as the starting point, we apply the [PERSON]'s method in optimization. The [PERSON]'s method converges to the closest to \(\overrightarrow{x}_{best}\) stationary point. Notably, in some cases the [PERSON]'s method is not applicable, for example, when the Hessian matrix is singular, and, consequently, not invertible. The second proposed algorithm is based on gradient descent methods. Gradient descent is an iterative optimization technique for finding the minimum of a function, and gradient ascent is a technique for solving the dual maximization problem. To minimize a given function \(f\), one takes steps proportional to the negative of the gradient \(\
abla f\); to maximize a given function, one takes steps proportional to the positive of the gradient:  
\[\overrightarrow{x}_{t+1}=\overrightarrow{x}_{t}\mp\gamma\
abla f\big{(} \overrightarrow{x}_{t}\big{)}. \tag{8}\]  
Gradient descent formula is given by Equation (8) with the '\(-\)' sign used, gradient ascent formula is given by Equation (8) with the '\(+\)' sign used. The \(\overrightarrow{x}_{t}\) vector represents the solution on the \(t\)-th iteration, \(\gamma\) is the learning rate, \(\
abla f\big{(}\overrightarrow{x}_{t}\big{)}\) is the gradient of the optimized function \(f\) at the point \(\overrightarrow{x}\).