Learning Logical Exceptions In Chess

This thesis is about inductive learning, or learning from examples. The goal has been to investigate ways of improving learning algorithms. The chess end-game \King and Rook against King" (KRK) was chosen, and a number of benchmark learning tasks were de ned within this domain, su cient to over-challenge stateof-the-art learning algorithms. The tasks comprised learning rules to distinguish (1) illegal positions and (2) legal positions won optimally in a xed number of moves. From our experimental results with task (1) the best-performing algorithm was selected and a number of improvements were made. The principal extension to this generalisation method was to alter its representation from classical logic to a non-monotonic formalism. A novel algorithm was developed in this framework to implement rule specialisation, relying on the invention of new predicates. When experimentally tested this combined approach did not at rst deliver the expected performance gains due to restrictions on the size of training samples. With larger samples and a more e cient generalisation method, together with the new specialisation algorithm, a successful result was obtained. The outcome of this test was a new inductively generated solution to the KRK illegality problem. Unlike other solutions using similar formulations of this problem in the Machine Learning literature the new result was complete and correct. The key step was the invention of new predicates during the specialisation stage of the inductive process. iii This new method was then applied to task (2), i.e. learning rules to conjecture the value of n for positions won optimally in n moves. Positions were extracted from an exhaustive database containing all canonical positions in the KRK end-game. Prolog rules induced from these are presented for positions won in 0 and 1. These are complete and correct when tested against the database, and have been validated by a human expert. iv Acknowledgements I owe a debt of gratitude to many who have helped me along the way to completing this thesis. Without them, it would not exist. My thanks are speci cally due as follows. To Donald Michie, who rst employed me at the Turing Institute to work on the US Army chess project. Beyond giving me my rst job in Arti cial Intelligence, by his consistent interest in and many excellent suggestions concerning this work his involvement has proved invaluable. I will always feel privileged to have worked with him. To Stephen Muggleton, who has been both colleague and friend. His inspiration, technical brilliance and communicable insight are by now well known throughout the Machine Learning community, as they have been to me for many years. He proposed the chess problems studied in the thesis, and suggested ways in which they might be approached. It is largely due to his encouragement, foresight and patience that I have stayed the course. Together with Donald Michie, he has taught me that AI is of relevance and importance to the heart of the scienti c enterprise. To Bob Henery, for very generously agreeing to undertake the supervision of this thesis. To Jean Hayes-Michie, for her collaboration on the human learning experiments, and her empathy as a fellow PhD candidate. To Thirza Castello-Cortes, for her kindness and forbearance throughout many hours of detailed supervision. v To Ivan Bratko, for his constructive criticism at all stages of this work and in particular his help in validating the induced chess concepts. To Arthur van Ho for his work on the KRK database. To Ashwin Srinivasan for many fruitful conversations and his moderating in uence on my ideas. I would like to thank the sta , especially Mary Ritchie, and the pupils of Glenwood School, Glasgow, for their help in carrying out the human learning experiments. Thanks also to David Aha, Robin Boswell, Wray Buntine, Jason Catlett, Peter Cheeseman, Peter Clark, Saso Dzeroski, Cao Feng, Boonserm Kijisikurul, Ross King, Eduardo Morales, Tim Niblett, Ross Quinlan, Stuart Russell, Claude Sammut, Barry Shepherd, Sia Tangkitvanich, Stefan Wrobel and any others not mentioned above with whom I have had discussions regarding the material in this thesis. I am especially grateful to all the sta of the Turing Institute for providing the facilities allowing the conduct and completion of this research. I would also like to thank Knowledgelink and Attar Software for facilities and help. All drawings were produced using Tgif Version 2.9 by William Chia-Wei Cheng. This work was supported in part by Alvey project IKBS/092 and IED project 4/1/1320. Finally I thank those of my family and friends who, with varying degrees of wit or seriousness, expressed such scepticism as to the eventual completion of this thesis that my determination to nish the work was thereby increased. vi Previous Publication Parts of Chapter 4 appeared in [85]. Parts of Chapter 5 and Appendix B appeared in [5]. A version of Chapter 6 appeared as [3]. Parts of Chapter 7 appeared in [4]. Authorship The work in this thesis is my own. The composition of the thesis is my own. However, with respect to the inclusion of material published previously in collaboration with di erent authors, I acknowledge the following contributions. Jean Hayes-Michie advised me on the format of the human learning experiment in Chapter 4 and Donald Michie assisted in calculating the statistics. Stephen Muggleton assisted me in proving Theorem 5.7 in Appendix B and ran Golem on the even-numbered KRK depth of win examples in Chapter 7. vii