Review of "Adaptation in Natural and Artificial Systems by John H. Holland", The U. of Michigan Press, 1975

This book is an unquestionably important work for researchers in learning. |t provides a novel and thorough analysis of a fundamental learning problem: given a previous sample of behaviors which have been differentially rewarded, what behaviors should next be generated to optimize expected rewards? Holland's approach, in overview, is to establish the relevance of this question for both artificial systems and natural populations in adapting to their environments, to show how genetic operators provide a means for searching and evaluating a number of possibilities which is t exponential compared to the fixed number which would be searched by a corresponding enumerative technique and, subsequently, to develop a representation for data and simple computer programs and an associated learning algorithm which exploits the advantageous genetic operators for training from samples of such representations. The broad generality of the learning problem to which this book is addressed suggests the obvious advantages and potential pitfalls of such investigations. Any significant success in treating such an unrestricted problem is certain to have wide applicability. Conversely, if learr~drcR' is actually a common rubric for many different and essentially distinct processes, the consideration of their commonalities may lead to useless discoveries and research. White the truth of these opposing possibilities is unknown, Holland's book presents a persuasive argument for considering learning as a general problem of adaptation to the environment and, more specifically, for developing an appreciation of the mechanisms of natural adaptation apparently evolved in living organisms in the arena of natural selection. To introduce the important aspects of such methods Holland begins by comparing artificial and natural learning problems to the two-armed bandit problem: Given two games g l and g2 having payoffs 0 or 1 depending on unknown probabilities p! and P2, respectively, and corresponding histories of n I and n 2 successes (payoff 1) out of Nj and N 2 trials, which of the two Rames should next be played? Although a particular plan is shown for this problem which is optimal in the limit (Corollary 5.2), the important point of this exercise is the demonstration that this best plan allocates a number of plays to the currently "better" of the two games which is an exponential multiple of that allocated to the currently "worse" game. From this, Holland concludes that good plans should incorporate mechanisms which sample from alternative behaviors in proportion to exponential functions of their expected payoffs. Subsequently, he shows that just such weighted samplings are intrinsic to "reproductive plans" using the genetic operators of crossing-overs inversion, and mutation to develop new members of the population from the current stock. Here, the proposed representation of each individual is a string of feature values (alleles) and the operators have the following definitions: crossingover occurs when two mated individuals produce a new one with a description which is the concatenation of one substring from each of the parents; inversion occurs when an individual with description SjS2'~ 3 where S 2 = s2! s22,. ,S2k produces a new individual with description S|S4S 3 where S 4 = S2k.s22s21~ mutation occurs when Review of HOLLAND