REPRESENTING IMAGERY

Glasgow has made a powerful case for the importance of imagery for understanding mind and intelligence. Moreover, her knowledge representation scheme for imagery is by far the best currently available. Nevertheless, it has clear limitations which show the need for richer and more flexible representation schemes. Glasgow’s admirably clear and incisive article convincingly argues for the importance of imagery to artificial intelligence (AI). While its message may be unappealing to those who think that logical formalisms provide all the representational tools that A1 needs, the paper shows that there is more to some kinds of intelligence than is dreamed of in predicate calculus. My own interest in imagery stems from a concern to develop a computational understanding of scientific discovery. Many scientists have reported that images played a crucial role in their most creative thinking: the most eminent include Bohr, Boltzmann, Einstein, Faraday, Feynman, Heisenberg, Helmholtz, Herschel, KekulC, Maxwell, Poincare, Tesla, Watson, and Watt. Such reports cannot be taken at face value, since it is possible that the scientists’ introspections were erroneous and that what seemed to be use of images was actually at the computational level a different kind of process. Nevertheless, the reports are suggestive and gain credibility from the substantial body of support from psychological experiments €or the creative potential of imagery (Finke, Ward, and Smith 1992). Recognition of the importance of imagery is only the beginning, however. A1 needs new computational tools to be able to model the richness and power of imagistic reasoning. The kinds of programming languages we are used to working with are ill-suited for the kinds of pictorial representations that imagery requires. While developing computational models of analogical thinking, I had been struck by the visual character of some of the analogies being analyzed (Holyoak and Thagard 1989; Thagard 1993; Thagard, Holyoak, Nelson, and Gochfeld 1990). But Glasgow’s representational scheme was the first I encountered that seemed rich enough to be applicable to visual analogies. Treating spatial representations as embedded symbolic multidimensional arrays provides a convenient way to represent pictorial information and supports very useful functions for generating, transforming, and inspecting images. Using Glasgow’s representational scheme, Susan Hardy and I developed VAMP. 1 (Visual Analogical Mapping Program, One), which takes spatial representations of two analogs and computes the appropriate correspondences between them (Thagard and Hardy 1992). However, in the course of developing and applying VAMP. 1, several limitations of Glasgow’s representational scheme became apparent, most of them due to its use of arrays as the fundamental data structure. Arrays provide an attractive beginning point, since they are mathematically well understood and are capable of capturing much spatial information. But not all images fit naturally into the “boxy” arrangement assumed by arrays. For example, we can use a pictorial representation of the solar system, with the planets orbiting the sun, to suggest a simple pictorial model of the atom, with electrons orbiting the nucleus. But neither the orbits nor the different locations of the planets as they orbit are naturally encoded using arrays. Moreover, it is not clear how to represent the spatial fact that one object is contained in another, for exzmple a heart in a body. Of course, we might use a