No Free Lunch Theorems

{{mathematics}}
'''No Free Lunch Theorems for Search'''<ref name=WM95>Wolpert, D.H., Macready, W.G. (1995), ''No Free Lunch Theorems for Search'', Technical Report SFI-TR-95-02-010 (Santa Fe Institute).</ref> is the title of a 1995 paper of David H. Wolpert and William G. Macready, and '''No Free Lunch Theorems for Optimization'''<ref name=WM97>Wolpert, D.H., Macready, W.G. (1997), ''No Free Lunch Theorems for Optimization'', IEEE Transactions on Evolutionary Computation 1, 67. http://ic.arc.nasa.gov/people/dhw/papers/78.pdf</ref> the title of a follow-up from 1997.

In these papers, Wolpert and Macready show that ''for any algorithm, any elevated performance over one class of problems is offset by performance over another class''<ref name=WM97/>, i.e. that on average each algorithm works as well as a blind search.

[[Intelligent Design]]ers love these ''No Free Lunch Theorems'' (often abbreviated to '''NFL'''), as they seem to indicate that - for instance - evolutionary algorithms have to get some additional ''information'' from somewhere to work.

Usually they neglect the fact that an algorithm has to work only on a certain class of functions, and not on all functions. Wolpert himself has repudiated [[William Dembski]]'s interpretation of the theorems as lending support to intelligent design.

==Mathematical formulation of the  ''Sharpened No Free Lunch Theorem''==

While the original NFL theorems looked at the class of all functions, the ''Sharpened NFL'' theorem is formulated for smaller classes with a special property, i.e., being ''closed under permutation''.

First, some  definitions have to be introduced in the way as they seem popular in the literature when talking about the NFL and such. This set of definition can be found - for instance - in a paper<ref>Andrea Valsecchi and Leonardo Vanneschi (2008), ''A study of Some Implications of the No Free Lunch Theorem'', Applications of Evolutionary Computing, Volume 4974/2008, Springer Berlin/Heidelberg,  pp. 634-635</ref> of  Andrea Valsecchi and Leonardo Vanneschi:
*<math>\mathcal{X},\mathcal{Y}</math> are finite sets. 
*<math>f:\quad \mathcal{X} \rightarrow \mathcal{Y}</math> is a function
*a ''trace'' of length ''m'' over <math>\mathcal{X}</math> and <math>\mathcal{Y}</math> is a sequence of couples <math>\langle (x_1,y_1), \dots, (x_m,y_m) \rangle</math> such that <math>x_i \in \mathcal{X}</math> and <math>y_i = f(x_i)\,</math> <math> \forall i = 1,2,\dots,m</math>
*a trace is called ''simple'' if an element of <math>\mathcal{X}</math> appears at most once in it
*''T'' is the set of all possible traces over the sets <math>\mathcal{X}</math> and <math>\mathcal{Y}</math>
*a ''search operator'' is a function <math>g: T \rightarrow \mathcal{X}</math>
*''g'' is ''non-repeating'' if <math>\forall t \in T, g(t) \notin t^\mathcal{X}</math> (here, <math>t^\mathcal{X} = \{x_1,x_2,\dots,x_m\}</math> for the trace <math>\langle (x_1,y_1), \dots, (x_m,y_m) \rangle</math>, and <math>t^\mathcal{Y} = \{y_1,y_2,\dots,y_m\}</math>)
*if <math>t \in T</math> is simple and <math>g</math> is non-repeating, then also <math>t' = t \| (g(t),f \circ g(t))</math> is a simple trace. Here <math>\|</math> is the concatenation operator
*a deterministic search algorithm <math>A_g\,</math> is an application <math>A_g: ((\mathcal{X} \rightarrow \mathcal{Y}) \times T) \rightarrow T</math> with <math>\forall t \in T: \quad  A_g(f,t)= t \| (g(t),f \circ g(t))</math>, where ''g'' is a search operator.
*<math>A_g\,</math> is called ''non-repeating'' if ''g'' is non-repeating
*for convenience, we'll drop the ''g'' and speak of a search-algorithm ''A''. 
*a deterministic non-repeating search algorithm will be called ''algorithm'' for short
*<math>A^m(f,t)</math> is the ''m''-th iteration of ''A'' to trace ''t'', defined by <math>A^0(f,t) = t</math>, <math>A^{m+1}(f,t) = A(f,A^m(f,t))</math>
*<math>A^m(f) = A^m(f,\langle \rangle)</math>, where <math>\langle \rangle</math> is the empty trace.
*A set of functions <math>F \subset \mathcal{Y}^\mathcal{X}</math> is called ''closed under permutation (c.u.p)'' if <math>\forall f \in F</math> and for each permutation <math>\sigma</math> of <math>\mathcal{X}</math> (<math>\sigma \in \mathcal{S}_\mathcal{X}</math>), we have <math>(f\circ \sigma) \in F</math>. 

Now, at last, the ''Sharpened NFL'':

'''Sharpened NFL''': ''Let <math>A, B</math> be two deterministic non-repeating search algorithms and let <math>F \subset \mathcal{Y}^\mathcal{X}</math> be c.u.p. Then <math>\forall m \in \{1,..., |\mathcal{X}| \}:</math><math>\{(A^m(f))^\mathcal{Y}:f \in F\}</math><math>=\{(B^m(f))^\mathcal{Y}:f \in F\}</math>

[[Category:Mathematics]]

==References==
{{reflist}}

Categories: Mathematics