Jay Scott has review ed this paper for his archive on Machine Learning in Games . We present the information-theoretic derivation of a learning algorithm that clusters unlabelled data with linear discriminants. In contrast to methods that try to preserve information about the input patterns, we maximize the information gained from observing the output of robust binary discriminators implemented with sigmoid nodes. We derive a local weight adaptation rule via gradient ascent in this objective, demonstrate its dynamics on some simple data sets, relate our approach to previous work and suggest directions in which it may be extended. This document describes the GAucsd system for function optimization developed by Nici Schraudolph at UCSD on the basis of Genesis 4.5, a genetic algorithm package written by John J. Grefenstette.
Genetic algorithms appear to hold a lot of promise as general purpose adaptive search procedures.
However, the authors disclaim any warranties of fitness for a particular problem. The purpose of making this system available is to encourage the experimental use of genetic algorithms on realistic optimization problems, and thereby to identify the their strengths and weaknesses. It is appealing to consider hybrids of neural network learnning algorithms with evolutionary search procedures simply because nature has successfully done so, suggesting that such hybrids may be more efficient than either technique applied in isolation. We survey recent work in this area and report our own experiments on using the GA to search the space of initial conditions for backpropagation networks. We find that use of the GA provides much greater confidence in the face of the dependence on initial conditions that plague gradient techniques, and allows a reduction of individual training time by as much as two orders of magnitude.