Beyond NFL : A few tentative steps

The No Free Lunch theorem (NFL) asks some serious questions to researchers interested in search processes. In an attempt to start answering these questions this paper examines the close link between local landscape structure and the performance of the genetic operators hill-climbing, mutation and uniform cross-over. Three abstract landscapes are described, each of which is based on the simple Mt. Fuji landscape. Each landscape has been constructed to have a local structure that is easy for only one of the three operators to traverse. The performance of various hybrid algorithms are also compared. It is shown that the tight coupling between operators and local structure holds even when the operators are combined. The conclusions drawn here are argued to have their equivalent in ‘real-world’ problems. 1 No Free Lunch (NFL) The No Free Lunch theorem (Wolpert & Macready, 1995) (NFL from now on) states that all search algorithms perform equally well when performance is averaged over all possible search spaces (fitness functions). Put it another way, if sear ch algorithm A outperforms algorithm B on a set of problems then there must exist problems on which algorithm B outperforms algorithm A, so that when averaged over all possible functions their performance is the same. This theorem has se rious implications for anyone trying to search a search spac e or equivalently optimise a fitness function. A lot of previous work in the field of search algorithms has compared the performance of various different search algorithms on test suite problems to demonstrate which performs best (Baluja, 1995; Davis, 1991; Craighurst & Martin, 1995; Whitley, Mathias, Rana, & Dzubera, 1995). The underlying assumption of most of this work was that there is one search algorithm that is best in most, or even all, search spaces. It i only with this assumption that it makes sense to compare the performance of search algorithms on abstract search spaces , and then say that improved performance on these spaces is good evidence that the search algorithm should be the algorithm of choice on real problems. NFL completely undermines this approach. Instead it suggests that good performance of an algorithm on a search space only tells us that that algorithm performs well on that searc h space. This strong interpretation of the consequences of NF L suggests that a new problem should be considered separately from all others solved so far. If this is the case then the only area of useful research is how to sensibly search a completel y unknown search space. The weaker interpretation of the consequences of NFL suggests that there are classes of search space problems and tha t for each class there would be a best algorithm that searches the instances of that class most efficiently. Also, cruciall y, this interpretation is only usefully different from the str ong interpretation if there is a way of knowing that a new problem is actually an instance, or not, of one of the classes for which a best algorithm is known without performing a significant number of evaluations. If the weak interpretations holds then identifying the salient features of such classes i one of the few ways in which search algorithm research could progress beyond NFL. There is a growing body of work that tries to categories the nature of different problems rather t an promote a particular algorithm (Soule & Foster, 1997; Vassilev, 1997; Rose, Ebeling, & Asselmeyer, 1996). Similarly there is work that tries to understand the necessary relatio nship between landscape structure and search algorithm that will give rise to good performance (Sakamoto & Goldberg, 1997; Kargupta, 1995; Muehlenbein & Schlierkamp-Voosen, 1995; Altenberg, 1997; Hordijk & Manderick, 1995; Harvey, 1994). 1.1 Real World Problems One of the strongest hints that the classification of search spaces is possible is the fact that problems so far encounter ed in real world applications are, in general, searched most ef ficiently by a number of algorithms other than random search (McIlhagga, Husbands, & Ives, 1996) (Kargupta & Goldberg, 1997). These algorithms include the genetic algorith m (GA) and multi-start hill-climbing. This has lead to a much used, but still fairly vague, notion of a meta class of ‘real world problems’ which contains all of the search spaces that are likely to be encountered in real applications. NFL only holds for performance averaged over the entire set of possib le search spaces. So, in this sub-set of the entire set of search spaces, it isnot true that all algorithms will perform as well as all others, and it is this set that people are really intere s d in. The question now becomes: Is it possible to say that particular sub-classes of the real world problems are always be st searched by particular algorithms ? If so, is it possible to i dentify the class of a particular instance (problem) without ha ving to search it ? Only if both of these are true do we actually gain anything beyond the strong interpretation of NFL. 2 The Assumptions of Search This paper takes a step towards understanding the relations hip between local structure and the success of genetic algorith ms. There are a number of underlying assumptions about the nature of a real world search space that drive the search bias of these algorithms. The assumptions are what enable the algorithms to perform better than random search, and relate to the way in which the evaluation of points in the search space can indicate where next to choose candidate solutions. Some typical assumptions are: Points in the search space with high fitness are indicators of the location of points in the space with higher fitness. There are local correlations in the fitness landscape. There are global correlations in the fitness landscape. There are sufficiently many points that represent successful solutions that a search strategy can find them, but not so many that random search will find one faster. A real world fitness landscape is likely to have neutral networks (Forst & Reidys, 1995; Hoshino & Tsuchida, 1996). Out of these the assumptions the two that are almost always used are that good points lead to better points and that there are correlations in the fitness landscape that can be exploit ed by different search operators. These assumptions are at the heart of most people’s intuition about the nature of real-wo rld problems. NFL shows that in general they do not always hold, as hill-descending will on average perform just as well as hi llascending, showing how even when there are correlations in the landscape, it can be that bad points lead to the location o f better points or even the optimum. However, for real-world problems these two assumptions appear to hold. This paper will look at how three particular search operators that derive from these assumptions behave differently on three abstract search spaces that generally hold true to these assumptions. The operators to be examined are Fit ne ss Genome Space Parent B Parent A Sub-space of disagreement Loci of disagreement between parents 1 1 1 0 1 0 0 1 0 0 0 1 1 0 1 1 0 0 1 1 1 0 1 0 1 1 1 1 0 0 0 1 0 0 1 0 1 0 1 1 1 * 1 0 1 * * 1 0 0 0 1 * 0 1 * * 0 1 1 Parent A