A framework is developed to explore the connection between e ective optimization algorithms and the problems they are solving. A number of \no free lunch" (NFL) theorems are presented that establish that for any algorithm, any elevated performance over one class of problems is exactly paid for in performance over another class. These theorems result in a geometric interpretation of what it means for an algorithm to be well suited to an optimization problem. Applications of the NFL theorems to information theoretic aspects of optimization and benchmark measures of performance are also presented. Other issues addressed are time-varying optimization problems and a priori \head-to-head" minimax distinctions between optimization algorithms, distinctions that can obtain despite the NFL theorems' enforcing of a type of uniformity over all algorithms.
The No Free Lunch theorem in optimization REF states that there is and will never be an optimization algorithm to solve all optimization problems.
No free lunch theorems for optimization
IEEE TRANSACTIONS ON EVOLUTIONARY COMPUTATION
Computer Science Mathematics