Co-Evolutionary Learning by Automatic Modularisation with Speciation

This thesis discovers and describes some problems in both co-evolutionary learning and genetic algorithm (GA) speciation, and uses this understanding to improve co-evolutionary learning. Ever since Art Samuel's early checkers program, a seductive idea for creating expertise arti cially is to have a collection of individuals compete against each other. This can cause an escalating \arms race" of innovation and increasing sophistication. Co-evolutionary genetic algorithms are a recent version of this concept. Co-evolutionary learning is suitable for a range of management and control problems, such as controlling a stock portfolio, and numerous military tasks. We study the most intuitive of such problems, learning to play a simple game of con ict, but our results are relevant to many important real-world problems. We demonstrate why co-evolution can sometimes fail (and fail spectacularly) to cause the desired escalation of expertise. In most games, and in related problems of management and control, there is more than one good strategy. Genetic drift causes a GA to nd only one high-quality strategy. In a co-evolutionary environment, it then over-specialises to that one strategy, creating expertise that is na vely vulnerable to other strategies. This can cause mass extinctions, when a random mutation is novel enough to exploit its overspecialised opponents. With this explanation in mind, we then show that some plausible modi cations to the co-evolutionary GA do not greatly improve this poor generalisation ability. To induce a smoothly escalating level of expertise, more radical modi cation is needed. Speciation o ers a way to improve the generalisation ability of co-evolutionary learning: a speciated co-evolving GA can automatically nd all the high-quality strategies for a game of con ict, preventing convergence to only one strategy and the over-specialisation that causes. We scrutinise a pair of sharing-based GA speciation methods, and discover some hitherto unknown problems which can cause poor results. Knowing these aws, we implement and validate some improvements. Some previous researchers have studied speciated co-evolutionary systems, but failed to utilise the diverse expertise embodied in the various species. Instead, they have used only the best individual, and discarded the rest of the population. This is a waste of learning: for tasks (such as learning a game) where generalisation to a wide range of possible opponents is desirable, a diverse ensemble of specialised strategies is usually more versatile than a single individual. We extend co-evolutionary learning beyond that level. Many real-world problems are too di cult to solve with a monolithic solution. The standard solution is a modular approach, where di erent sub-solutions deal with di erent aspects of the problem. For a game of con ict (and numerous management and control problems), Australian Defence Force Academy Page v di erent contingency strategies must be separately re ned and specialised to suit di erent possible opponents or situations. A committee of human experts usually decides how to manually modularise the system into sub-solutions. Their choice can be less than optimal, and does not always re ect the underlying structure of the problem. Manual modularisation relies on a priori knowledge of the problem. However, such knowledge is often di cult and costly to obtain for real-world problems. Speciation can divide responsibilities automatically. In the example problem of learning a game, a speciated co-evolutionary genetic algorithm can produce a repertoire of complementary high-quality strategies, where each species is a good opponent for another species. We describe a simple gating algorithm for combining these di erent strategies into a single high-level strategy. This gating algorithm uses majority voting. This approach generalises better than the best individual in the nal generation of the GA, with or without speciation and co-evolution. This validates our modular approach over the earlier method of simply using the best individual. It also demonstrates that speciation as automatic modularisation is a promising method of realising the potential of co-evolutionary learning. Page vi University College, The University of New South Wales Australian Defence Force Academy Page vii Acknowledgements First and foremost, many thanks are due to my supervisor, Dr. Xin Yao. His advice, support, and encouragement have been absolutely rst-class, and he has been the best supervisor any student could possibly have wanted. Dr. Bob McKay has been an excellent co-supervisor, especially in nding ambiguities and loopholes in my writing. Thanks are also due to those who answered my unsolicited questions over the years, including Samir Mahfoud, Simon Ronald, Rik Belew, Risto Miikulainen, David Moriarty, Greg Werner, Liane Gabora, David Fogel, Zbigniew Michalewicz, Steve Hubbard, Paul Donnelly, Robert Smith, Steven Salzberg, Robert Congleton, Peter Angeline, Stephen Hood, Mark Nelson, Alan Meyrowitz, Mitchell Potter, and last but not least my namesakes Hugh Darwen and Jamie Darwen. I would also like to thank the many people at ADFA's School of Computer Science who have helped me over the years. The group of people working on evolutionary computation at ADFA have provided many productive discussions and exchanges, including Peter Whigham, Yong Liu, Guangmin Lin, and Jason Bobbin. The postgraduate conveners during my time here, rst Dr. Bob McKay and lately Mrs. Miriam Fergusson, have provided excellent support mechanisms to encourage postgraduate research. I would also like to thank our secretariat, Eileen Trott, Helen Foot, Pam Giannakakis (n ee Trianta llopoulos), Maxine Kelly, and Alison McMaster. I would also like to thank our head of department, Professor Charles Newton, who has encouraged postgraduate research by nding the funds to let us present papers at conferences whenever possible. I also thank ATERB for funding to travel to a conference Several excellent public-domain genetic algorithm packages are available. This thesis has used and modi ed John Grefenstette's excellent GENESIS 5.0 package. Australian Defence Force Academy Page ix