# A Direct Search Algorithm for Global Optimization
## 2 Direct Search Methods and Global Optimization  
A direct search method solves optimization problems without requiring any information about the gradient of the cost function [1; 2]. Unlike many traditional optimization algorithms that use information about the gradient or higher derivatives to search for an optimal point, a direct search algorithm searches a set of points around the current point, looking for one where the value of the cost function is lower than the value at the current point. Direct search methods solve problems for which the cost function is not differentiable or not even continuous. Direct search optimization has been receiving increasing attention over recent years within the optimization community, including the establishment of solid mathematical foundations for many of the methods considered in practice [1; 3; 4; 5; 6; 7]. These techniques do not attempt to compute approximations to the gradient, but rather, the improvement is derived from a model of the cost function or using geometric strategies.  
There exists a great number of problems in different fields, such as engineering, mathematics, physics, chemistry, economics, medicine, computer science or operational research, where derivatives are not available, but there is a need for optimization. Some relevant examples are: tuning of algorithmic parameters, automatic error analysis, structural design, circuit design, molecular geometry or dynamic pricing. See Chapter 1 of [6] for a detailed description of the applications.  
Initially, direct search methods have been dismissed for several reasons [8]. Some of them are the following: they are based on heuristics, but not on rigorous mathematical theory; they are slow to converge and only appropriate for small-sized problems; and finally, there were no mathematical analysis tools to accompany them. However, these drawbacks can be refuted today. Regarding slow convergence, in practice, one only needs improvement rather than optimality. In such a case, direct methods can be a good option because they can obtain an improved solution even faster than local methods that require gradient information. In addition, direct search methods are straightforward to implement. If we not only consider the elapsed time for a computer to run, but the total time needed to formulate the problem, program the algorithm and obtain an answer, then a direct search is also a compelling alternative. Regarding the problem size, it is usually considered that direct search methods are best suited for problems with a small number of variables. However, they have been successfully applied to problems with a few hundred variables. Finally, regarding analytical tools, several convergence analysis studies for a direct search algorithm were published starting in the early 1970s [9; 10; 11; 12]. These results prove that it is possible to provide rigorous guarantees of convergence for a large number of direct search methods [4], even in the presence of heuristics. Consequently, direct search methods are regarded today as a respectable choice, and sometimes the only option, for solving certain classes of difficult and practical optimization problems, and they should not be dismissed as a compelling alternative in these cases. Direct search methods are broadly classified into local and global optimization approaches.