Talk:No free lunch in search and optimization

message: When I checked the link at Most of these references can be obtained from... not only were the files not available, but the citations were incorrect and/or incomplete. 22:34, 21 November 2006 (UTC)
thread_title: older entries 

message: The figure is incorrect, and has to go. In fact, when all cost functions are equally likely, each algorithm observes each possible sequence of cost values with equal likelihood, so there is no specialist / generalist trade-off of the sort depicted in the plot. To state that more precisely, if there are N domain elements and M codomain elements (distinct costs), then the probability that an arbitrarily chosen algorithm will observe values in an arbitrarily chosen sequence is M-N.
At the risk of seeming like I'm engaged in self-promotion, I'm going to take a shot at rewriting this article. The touted "simple explanation" is not freely accessible online, and it happens that I gave a very simple explanation during a student's thesis defense in 1994. I will try to respond to the "request / challenge" above, but part of me says that into every life a little math must fall. NFL is inherently about probability distributions on functions and sequences.
ThomHImself 00:54, 11 March 2007 (UTC)
thread_title:  Factually incorrect figure 

message: Section 3 of Wolpert and Macready (1997) gives two NFL theorems, not one. The 1995 tech report and the 1997 article of W&M both refer to theorems, plural, in the titles. This makes reference to the theorem something of an embarrassment. Beyond that, people who state the theorem rarely, if ever, state the first NFL theorem (the one they seem to have in mind). Instead they state their favorite interpretations.
The fact is that the actual NFL theorems are not as interesting as their immediate implications. That is why I am moving this article to "No Free Lunch in Search and Optimization," with "No-free-lunch theorem" redirecting here.
ThomHImself 21:10, 11 March 2007 (UTC)
replies: text: Note that double redirects do not work, so that if you do move this page, you will need to update any double redirects that are created as a result of the move.  The What Links To No-free-lunch_theorem special page will help in locating them.  -- Cat Whisperer 21:18, 11 March 2007 (UTC)
thread_title:  "''The'' No Free Lunch Theorem" is not the point 

message: A figure will clarify the introduction.
Watch for sections giving a gentle introduction to NFL and formalizing NFL.
I made a large number of changes without logging in. Sorry, but I'm a neophyte, and didn't know I wasn't logged in. ThomHImself 05:25, 29 March 2007 (UTC)
thread_title: More changes to come

message: It is somewhat humorous that someone added to the introduction precisely what I document as mathematical folklore in a later section on "the" NFL theorem:
:Put simply, the theorem says that over the average of all possible solution landscapes, no search algorithm is on average better than any other.
No one who makes this claim ever cites both an article and a theorem number. There is no such theorem in the writings of Wolpert and Macready. Furthermore, at the point where the sentence was added, there had been no mention whatsoever of a theorem. The article is now about NFL in search and optimization, and the folkloric theorem is just one aspect of that. ThomHImself 06:16, 29 March 2007 (UTC)
thread_title: Deleted folkloric theorem from introduction

message: I took this out, and I would be interested in knowing what's going on:
:==Additional Notes==
:This is also referenced by the infamous Pascal. If this is heard, please disbelieve him.
ThomHImself 23:34, 26 April 2007 (UTC)
thread_title: Deleted "Additional Notes" message

message: Thanks to whomever cleaned up the math typesetting in "Synopsis of NFL in mathematical terms." I will see later if I can't use <math> to improve some other stuff. —The preceding unsigned comment was added by ThomHImself (talk • contribs) 05:05, 29 April 2007 (UTC)
thread_title: Math typesetting

message: The treatment of Baylor's Evolutionary Informatics Laboratory as part of the "great intelligent design conspiracy" was unjustified. We have to go on what the lab actually gives evidence of doing. What I see is a generalization of Dembski's ideas to what could be an entirely legitimate field of study. It is certainly the case that practitioners of evolutionary computation supply domain knowledge when they can, and that it is important to understand how this information affects the results obtained.
The text entered in the article reduced ultimately to argumentum ad hominem -- against Dembski, actually. Bob Marks has made outstanding contributions to engineering that not even an opponent of intelligent design like me questions. When he says he is engaged in legitimate scholarship, one has to suppose he is in the absence of evidence to the contrary. ThomHImself 23:21, 19 June 2007 (UTC)
thread_title: Moved Info on Baylor Lab

message: NFL is in fact what I prefer to call conservation of search performance, in analogy with  Schaffer's conservation of generalization accuracy (nonexistent article) in machine learning (no reference to conservation). This article could use a section relating NFL to conservation of generalization accuracy, but only when the main article exists.
Instead we got "Cullen Schaffer did it first, only it was a little different" editing that put his name in the introduction and at the head of the list of references. I have given positive suggestions, but to what entered the article, I will simply say no.
A little story: I sketched a proof of the first NFL theorem in 1994, during a public thesis defense. I asked the student what he hoped to accomplish by tweaking optimizers to get better performance on test functions. He said, of course, that he hoped to obtain a generally superior optimizer. After a brief pause, I responded that optimizing a uniformly drawn function was equivalent to optimizing the output of a uniform random number generator. The silence that fell over the room was remarkable. I was never so arrogant as to think I was the only person to complete such a shallow line of reasoning. Over the years, I have met various people who realized there was NFL before Wolpert and Macready called it that.
The substantive part of Wolpert and Macready's work is the information-geometric analysis of "alignment" of algorithms and problems. ThomHImself 23:59, 26 June 2007 (UTC)
thread_title: Cullen Schaffer and conservation of generalization accuracy

message: Perhaps images should be added to this. In the [Criterea], it says that articles should be illustrated. --Dan LeveilleTALK 08:54, 28 December 2007 (UTC)
replies: text: Thanks, Dan. There used to be an illustration, but it was based on a common misunderstanding of NFL. The only clear illustration of how NFL "works" I have seen is a collection of three figures in the English (2004) paper. They convey technical detail, and adapting them for the article is not easy. I plan to decorate the introduction with something like Figure 4 – it fits with the final paragraph – though I worry about discouraging the general reader. The illustrations of a decision tree and the operation of the decision tree on objective functions would enhance the "formal synopsis" section. I would very much appreciate advice from anyone on this matter.ThomHImself (talk) 23:38, 15 January 2008 (UTC)
replies: text: Good point. It is a hard situation. --Dan LeveilleTALK 01:24, 16 January 2008 (UTC)

text: See "Illustration" section below.ThomHImself (talk) 07:48, 18 January 2008 (UTC)
thread_title:  GA Tip 

message: Regarding "Specified complexity, active information, and NFL": The unpublished papers coauthored by Dembski and Marks make no reference to intelligent design or to specified complexity. The notion of active information is very different from specified complexity, and is in itself reasonable. Marks seems to me an honest creationist who shares IDists' interest in information. The use of "active information" in arguments dismissing the success of computational simulations of evolutionary processes is undeniably analogous to the use of "specified complexity" in Dembski's earlier arguments. There is certainly the possibility that ID has again morphed. Thus I have opted to keep discussion of the two concepts together, but to avoid the suggestion that Marks is a member of the intelligent design movement.ThomHImself (talk) 20:24, 15 January 2008 (UTC)
thread_title: New section title

message: I think a simple illustration to go with the introduction would be two menus, one offering low prices to the vegan, and the other offering low prices to the carnivore. The prices on each menu would simply be those on the other, reordered. I would appreciate hearing from any passersby whether this would clarify.ThomHImself (talk) 07:48, 18 January 2008 (UTC)
thread_title: Illustration

message: Treatment of a topic in the encyclopedia is not endorsement. user:Odd Nature has been unable to provide a reliable source at Robert J. Marks II for his claim that Marks is an ID advocate, so he has come here to assert that Marks is an ID advocate. He obliterated a section here documenting the fact that Marks' active information is not equivalent to Dembski's specified complexity. The content was sourced. He also obliterated a section of Evolutionary Informatics Lab that included Mark Chu-Carroll's criticism of specified complexity and acknowledgment that active information might be useful. This looks like systematic removal of a balanced response, not a NPOV. Any human being, including Dembski, is capable of producing both bogus and useful ideas. We have to report on the ideas as ideas, and not engage in thinly veiled ad hominem attacks.
Odd Nature has a habit of deleting without discussing. I look forward to having my credulousness demonstrated to me in detail. BTW, I have a chapter highly critical of ID theory to appear in an edited volume this June. ThomHImself (talk) 09:17, 18 April 2008 (UTC)
thread_title: "Paring this down per WP:NPOV. Size and credulousness of section gave undue weight to ID proponents views."

message: I had little knowledge of Wiki policy when I began editing this article. There is a great deal of my original research here. Having since held others to WP:NOR in other articles, I am going to police myself here. ThomHImself (talk) 16:47, 24 April 2008 (UTC)
thread_title:  Removing original research 

message: The second paragraph elaborating the NFL metaphor was something that I have never seen in the literature, and was my WP:Synthesis. I thought long and hard about the figure, because I had neither contributed nor seen a good, simple figure in the NFL literature. Obviously this was my synthesis. It did not follow immediately from anything in the literature, and was supported only by my own authority. ThomHImself (talk) 17:07, 24 April 2008 (UTC)
thread_title:  Introduction 

message: When the article passed GA review, a reviewer expressed vague doubts about this section, and I didn't understand what s/he was driving at. The example was entirely novel, reflecting how I understand my own formal results in the source I gave. To my knowledge, here is nothing like the example in the NFL literature. ThomHImself (talk) 17:21, 24 April 2008 (UTC)
thread_title: Example: NFL in a roulette game

message: The "functionally equivalent" algorithm was my invention. To my knowledge, neither it nor its functional equivalence to anything appears in the NFL literature. The nice concept of Phi-NFL distributions on objective functions was something I came by working on this article. I have not published it, and to my knowledge no one has published an equivalent concept. ThomHImself (talk) 17:41, 24 April 2008 (UTC)
thread_title: Formal synopsis of NFL

message: My statement in plain language of what Theorem 1 "says" was novel, and supported only by my own "authority." Similarly, my claim that the theorem implies much more than the "folkloric theorem" is unsupported, and furthermore reflects my bias. I will remove this bias throughout the article later. ThomHImself (talk) 17:51, 24 April 2008 (UTC)
thread_title: Original NFL theorems

message: This section was full of common misinterpretation of the NFL results when I began editing the article. I replaced them with my own "wise" interpretation, synthesizing statements from published sources, and adding two longish paragraphs with no sources whatsoever. I have pruned this section to the interpretation that was in the article when I first came on the scene, plus qualifying remarks directly supported by sources. ThomHImself (talk) 18:12, 24 April 2008 (UTC)
thread_title: Interpretation of NFL results

message: That people substitute a plain-language statement for Wolpert and Macready's Theorem 1, and that they reduce the entire body of NFL results to that folkloric theorem, is a pet peeve of mine. This probably owes in part to the fact that it deemphasizes my own contributions. Characterization of the plain-language statement as a folkloric theorem occurs nowhere but here to my knowledge, and emphasis of the characterization definitely reflects my bias. Thus I have removed it from the article. ThomHImself (talk) 18:40, 24 April 2008 (UTC)
thread_title: Folkloric NFL theorem

message: Omission of the (A)NFL theorem was originally oversight on my part. When I came upon it again, six years after first reading the paper proving it, I did not agree with authors' interpretation, and pondered how to treat their interpretation without contradicting different claims in the article. I should never have considered this. What is published is what should be reported here. My unpublished reservations have no place here. 
The (A)NFL theorem deserves mention in the introduction. ThomHImself (talk) 19:26, 24 April 2008 (UTC)
thread_title: Overview and (A)NFL theorem 

message: Please do not repeat the claim that Marks is a proponent of a certain ideology (see this diff) unless you can provide a source allowed by policy on reliable sources for living persons.
WP:BLP specifies immediate removal of possibly defamatory or otherwise damaging claims about living persons that are not sourced according to policy. This applies to all of Wikipedia -- talk pages etc., and not just articles. The three revert rule does not apply to defense of WP:BLP. Note in the diff above that I've done nothing but remove the possibly defamatory or otherwise damaging claim. ThomHImself (talk) 22:30, 24 April 2008 (UTC)
thread_title:  Possibly defamatory claim about [[Robert J. Marks II]] 

message: ===Figure===
I have twice removed factually incorrect figures from this article. Anyone who adds a figure should adapt it from a figure in the NFL literature (peer-reviewed article or published book), and not the one at the NFL website (it has already been in this article). ThomHImself (talk) 23:43, 24 April 2008 (UTC)
thread_title:  Suggestions for making this a GA again 

message: I have twice removed factually incorrect figures from this article. Anyone who adds a figure should adapt it from a figure in the NFL literature (peer-reviewed article or published book), and not the one at the NFL website (it has already been in this article). ThomHImself (talk) 23:43, 24 April 2008 (UTC)
thread_title: Figure

message: With my removal of a massive amount of exposition that was my original research or reflected my personal opinion, some sections of the article are very thin. They need additional material that is not made up by the editor (don't repeat my violations of Wiki policy), but is directly supported by books or peer-reviewed papers on NFL. ThomHImself (talk) 23:43, 24 April 2008 (UTC)
thread_title: Elaboration and elucidation

message: This section has been appropriated for polemics by anti-ID editors. I have been accused of conflict of interest in the matter, so I will do nothing with the section but defend Wikipedia policy on claims about living persons. There should be discussion of invocations of NFL by Dembski and Marks at Evolutionary Informatics Lab or in an article on active information. I recommend that someone who wants to make this a maintainable technical article simply delete the section. ThomHImself (talk) 23:43, 24 April 2008 (UTC)
thread_title: Intelligent design and NFL

message: I think this article could benefit from another sentence in the introduction explaining the topic to laypeople.  As a computer science major unfamiliar with the topic, I found it confusing.  —Preceding unsigned comment added by 69.143.161.239 (talk) 03:13, 8 May 2008 (UTC)
thread_title: A More Friendly Introductory Paragraph

message: The search in constraint satisfaction is of a fundamentally different type than addressed here. The problem instance is decomposed in constraint satisfaction. This article addresses "search" as iterative sampling of the objective function. That is, the objective function is treated more or less as a black box. Schaffer's result in machine learning is relevant because it, too, treats the function as a black box. ThomHImself (talk) 21:52, 3 September 2008 (UTC)
thread_title:  Deletion of constraint-satisfaction content from introduction 

message: From the current article:
:In the "no free lunch" metaphor, each "restaurant" (problem-solving procedure) has a "menu" associating each "lunch plate" (problem) with a "price" (the performance of the procedure in solving the problem). The menus of restaurants are identical except in one regard — the prices are shuffled from one restaurant to the next. For an omnivore who is as likely to order each plate as any other, the average cost of lunch does not depend on the choice of restaurant. But a vegan who goes to lunch regularly with a carnivore who seeks economy pays a high average cost for lunch. To methodically reduce the average cost, one must use advance knowledge of a) what one will order and b) what the order will cost at various restaurants. That is, improvement of performance in problem-solving hinges on using prior information to match procedures to problems.
I don't think this is a good metaphor — or maybe I'm just missing something here. Clearly neither the vegan nor the carnivore will ever get a "free lunch" if each "lunch plate" has a non-zero "price"! So the "No Free Lunch" theorem doesn't seem to have anything to do with the cost of lunch in our allegory — which seems unnecessarily confusing to me. Either the mathematical "Free Lunch" should be represented in the allegory by a lunch which is free, or else the allegory shouldn't be talking about the allegorical price of allegorical lunches at all.
Lastly, the last sentence of the paragraph quoted above seems to be trying to explain that "No Free Lunch" is rarely relevant in the real world, but it would IMHO be a good idea to state more forcefully that yes, some search algorithms are better than others in the real world (e.g., don't use a linear search if a binary search would be faster; GIF images are usually smaller than BMPs; and so on) precisely because the real world is not uniformly distributed, and is full of "advance knowledge" and "prior information". The current article seems to be aimed at an audience of experts to the bafflement of non-experts, which I found weird given that it's an article about a theorem with a "folksy" name. --Quuxplusone (talk) 19:12, 8 October 2008 (UTC)
replies: text: Indeed. I read the "dumbed down" version and was thoroughly confused. The technical explanations were much more palatable (a bad sign)!  —Preceding unsigned comment added by 65.183.135.231 (talk) 22:02, 15 October 2008 (UTC)
thread_title:  Confusing illustration 

message: Should this article, or perhaps a related article, mention Herb Sutter's "The Free Lunch is Over" essay (http://www.gotw.ca/publications/concurrency-ddj.htm), which describes another way in which there is no "free lunch" in computing?  —Preceding unsigned comment added by DestroyerofDreams (talk • contribs) 01:53, 27 February 2010 (UTC)
thread_title:  Herb Sutter essay 

message: The introduction to the article is writen using too much jargon, and is generally unclear. It should be re-written so that people not already familiar with the topic can understand what it is talking about. The word "procedures" seems to have a special meaning, which is not a priori obvious. Perhaps the wording in the "Overview" section should be moved here, as it is written much more clearly.  — Preceding unsigned comment added by 68.108.237.62 (talk) 17:43, 20 April 2012 (UTC)
thread_title:  Simplify/Rewrite Introduction for non-specialists