Source: http://cogprints.org/1555/1/im-im-cp.htm
Timestamp: 2019-04-21 15:11:37+00:00

Document:
California State Universities, Fullerton & Los Angeles.
Can theories of mental imagery, conscious mental contents, developed within cognitive science throw light on the obscure (but culturally very significant) concept of imagination? Three extant views of mental imagery are considered: quasi-pictorial, description, and perceptual activity theories. The first two face serious theoretical and empirical difficulties. The third is (for historically contingent reasons) little known, theoretically underdeveloped, and empirically untried, but has real explanatory potential. It rejects the "traditional" symbolic computational view of mental contents, but is compatible with recent situated cognition and active vision approaches in robotics. This theory is developed and elucidated. Three related key aspects of imagination (non-discursiveness, creativity, and seeing as) raise difficulties for the other theories. Perceptual activity theory presents imagery as non-discursive and relates it closely to seeing as. It is thus well placed to be the basis for a general theory of imagination and its role in creative thought.
Can extant scientific theories of mental imagery (quasi-perceptual experience) account for, or accommodate, the much wider functions of imagination as traditionally conceived? Such functions range from ordinary supposal to the most valued aspects of human insight and creativity (Thomas, 1997a, 1999a), but it is the latter that will be of particular concern here. The issue is raised not only to bring a fresh perspective to the notoriously recalcitrant scientific debate about the nature and cognitive importance of imagery, but also to try to illuminate the obscure topic of imagination itself. Prima facie imagery and imagination are intimately related--certainly "imagination" is often used to name the faculty of image production (or the mental arena in which images appear)--and this fact itself demands explanation. I will argue that neither of the better known theories of imagery, the "quasi-pictorial" theory (Kosslyn, 1980, 1994) and the "description" theory (Pylyshyn, 1973, 1981), are capable of meeting this challenge. I will develop and defend a theory that can, I believe, meet it: a version of the little known "perceptual activity" theory, related to recent "situated" approaches to cognition (Clancey, 1997).
It seems likely that the association between imagery and creative imagination is not unrelated to the unusual passion (Dennett, 1978) surrounding the debates about imagery of the past 30 years or so. Except to the aficionado, mere visualization is surely not obviously, in itself, a terribly exciting matter; "imagination", by contrast is a highly value laden and culturally charged term (Kearney, 1988). If the relationship between imagery and imagination (in the relevant sense) can be clarified, perhaps the constraints which scientific results lay upon theories of imagery may help to bring some discipline to the more extreme claims made on behalf of the imagination.
In particular, the concept of imagination has played a key role in anti-scientific attitudes and ideologies, at least since the writers and artists of the Romantic movement gave it a central place in their ideology (Daston, 1998). Imagination is presented as supremely valuable and forever beyond the reach of scientific understanding; it is the guarantor and the embodiment of the (alleged) fact that science will never be able to illuminate "what really matters" in life. Romanticism was by no means unequivocally hostile to science (Abrams, 1953), but the strong anti-scientific strand of the 1960s "counter culture" movement clearly derived from it (Roszak, 1970), and although Gross and Levitt (1994) try to blame the contemporary prevalence of anti-science on the "academic left", they acknowledge that its intellectual roots are in Romanticism rather than Marx or the labor movement. Furthermore, despite the fact that the vital role of imagination in scientifically thinking has been urged repeatedly and convincingly (e.g. Tyndall, 1872; Van't Hoff, 1878/1967; Brown, 1991; Holton, 1996) the apologists of science have frequently demonstrated a hostility to the claims of imagination that is quite comparable to the hostility to science shown by the ideologues of imagination (Daston, 1998). Nevertheless, the Romantic conception of imagination (which was heir to a long tradition) has had an enormous and ineradicable intellectual influence, and is deeply embedded in our folk psychology (Brann, 1991; Sutherland, 1971). There is every reason to think that it contains genuine and important insights, and that it addresses real socio-cultural, or even spiritual, needs. For science to dismiss the issues it raises would amount to the acceptance of a major limitation on the scope of the scientific approach to reality. In order to address these issues rationally and effectively we need a scientific theory of imagination that pays due respect to the folk/Romantic conception.
In what follows I will first outline the extant scientific theories of the nature and mechanisms of imagery, and will attempt to assess them purely as such. I will then try to delineate the relevant key components of the folk/Romantic conception of imagination, and finally I will try to assess to what degree the respective scientific theories of imagery are capable of throwing light upon them.
Let us provisionally define mental imagery as quasi-perceptual experience, experience that significantly resembles perceptual experience (in any sense mode), but which occurs in the absence of appropriate external stimuli for the relevant perception (for discussion and defence of this definition see Thomas, 1999b). There are normally important experiential differences between imagery and perception--without them imagery may slide into hallucination--but these need not concern us here; it is the similarities which are definitive.
"Experience", of course, implies consciousness, and this may raise a problem because there are reasons to think that imagery can play a role in cognition even when subjects are not consciously aware of it (Paivio, 1971, 1986; Baars, 1996). However, people often are consciously aware of quasi-perceptual experiences, and it is upon this phenomenological fact that the major cognitive research programs into imagery have been built (Thomas, 1987). I submit that we should think of occurrent mental images as, at the least, potential contents of consciousness (in something like the sense of Searle, 1992), even if it is the case that they may sometimes occur without the subject being explicitly aware of them. Imagery may, conceivably, play an important role in cognition even when people deny consciously experiencing it (Thomas, 1989, 1999b).
Notoriously, many psychologists of the behaviorist era did deny if not the experience certainly the psychological significance of imagery. However, from the late 1950s interest in imagery gradually increased (Holt, 1964), and from the later 1960s a number of striking, now classic, experimental effects (for reviews see Morris and Hampson, 1983, and Finke, 1989) convinced many of its cognitive significance (Thomas, 1987). Imagery proved to have powerful mnemonic effects (Paivio, 1971, 1986, 1991), and it appeared that an image could be smoothly rotated (Shepard & Metzler, 1971; Shepard & Cooper, 1982), and scanned across (Kosslyn, 1973, 1980). It also appeared that when inspecting their images, subjects could find subjectively large details more quickly than subjectively small ones (Kosslyn, 1975, 1980), and that imagery and perceptual tasks in the same mode would often mutually interfere with one another (Brooks, 1968; Segal, 1971; Craver-Lemley & Reeves, 1992). A theory of the nature of imagery was clearly needed.
With Morris and Hampson (1983), I take there to be three types of imagery theory extant in the Cognitive Science literature. I shall refer to them as "description theory", "picture theory", and "perceptual activity [or PA] theory". However, we might better understand these as paradigms (Kuhn, 1970), or, better still, as research programs (in the sense of Lakatos, 1978), rather than as static and unitary theories. Although research programs are characterized by certain core commitments, which suggest particular lines of investigation and explanatory strategies, different theoretical elaborations can be constructed around these cores to account (with varying degrees of strain) for almost any empirical evidence that is likely to arise. Thus, in the course of its development a single vigorous research program may encompass a number of well elaborated and quite diverse theoretical positions, whose detailed empirical commitments may differ considerably. In the case of the pictorialist research program we see this even within its relatively recent evolution: Contrast the very different versions of picture theory given by Kosslyn in his two major statements on the subject (Kosslyn 1980, 1994). In the light of this, we should not expect to be able to make a definitive choice between the three imagery research programs solely on the basis of empirical evidence. Rational choice between programs (or paradigms) depends not just upon empirical adequacy, but also on judgements about factors like the elegance and consistency of the theoretical elaborations needed to assimilate the evidence, and the scope and promise of the core ideas (Lakatos, 1978).
Picture theory has a very long history, going back to Plato or even Democritus (Thomas, 1987), and until quite recently it was almost universally accepted (White, 1990). All versions hold that having visual imagery involves having entities, in the head or in the mind, which are like, or functionally equivalent to, inner pictures. These pictures are thought of as being composed of copies or remnants of earlier sense impressions, complexes of visual sensations, which were themselves picture like. "Picture" is, of course, strictly appropriate only to copies in the visual mode (upon which most discussions of imagery concentrate), but, if this sort of account really works for the visual, then it seems at least initially plausible that similar "copy" theories could be devised for the other sensory modes (Matthews, 1969).
Picture theory came under severe philosophical attack in the middle years of this century (e.g. Sartre, 1936/1962; Ryle, 1949; Wittgenstein, 1953) for being committed to an implausible, Cartesian view of the mind. However, in the opinion of many, Kosslyn (1975, 1980, 1994) has succeeded, in the teeth of trenchant and sophisticated objections from such description theorists as Pylyshyn (1973, 1978, 1979, 1981) and Hinton (1979), in showing that his computational version of picture theory is both coherent and empirically credible.
At least, Tye (1991) seems to have convincingly demonstrated that it is coherent given the assumption that computational data structures of some type are a proper model for conscious and intentionalistic mental contents. Kosslyn implicitly shares this assumption--let us call it computational mentalism, or CM--with the descriptionists. Indeed, most discussions of cognitive theory conflate the notion of a mental (intentional) content with that of a computational representation, failing to distinguish the CM dogma from the view that brain function may best be understood and simulated computationally1. However, one can consistently accept the latter position (and even a form of "strong AI") whilst rejecting CM, and there may be good reasons to do so (Slezak, 1999; Cummins 1989, 1996). CM faces the symbol grounding problem, the problem of understanding how computational states can acquire intrinsic intentional content (Harnad, 1990), and it is not at all clear that this problem is soluble2. If CM is false (as I believe) then both the description theory of imagery and the computational version of picture theory defended by Kosslyn and Tye are profoundly compromised. However, the critiques of these theories that follow below will not depend upon prior rejection of CM. Many people accept CM, and it is at the foundations of the information processing paradigm, which views the task of a theory of perception as, essentially, to give an account of how the information implicit in states of the sense organs is converted into a canonically "mental", consciously available, data format or formats. Kosslyn's fundamental disagreement with the descriptionists is over what sorts of format might be able to fill this role.
The quasi-pictorial theory of imagery [Based on Kosslyn (1980 p. 6)]. The system is considering whether a fox has pointed ears. A "quasi-picture" is constructed in the visual buffer, on the basis of a description in LTM, and the "mind's eye" analyses it to extract the required information.
Kosslyn draws his theory from an analogy with computer graphics, and its basic form is illustrated in figure 1. Data from which images may be constructed are stored in long term memory in the form of "deep representations", structural descriptions not fundamentally dissimilar to those envisaged by description theorists3. But these are not directly available to consciousness. They are analogous to the files in which data is saved by a computer graphics program, and on the basis of which actual, viewable pictures are constructed on the computer's CRT monitor. Kosslyn's (1980) theory is explicitly based on this "CRT metaphor". He holds that "quasi-pictures" or "surface representations" are constructed on the basis of the information in deep representations. This construction takes place at a functionally defined neural locus that he calls the "visual buffer". Once the quasi-picture is established it is available to consciousness as an image, and, furthermore, information that was merely implicit in the deep representation (such as the pointedness of the fox's ears, in the depicted example) can be extracted from it by the postulated "mind's eye function". We should not think of this "function" as literally seeing the image, but it is needed to read and interpret the buffer's "surface display".
Clearly Kosslyn thinks of the visual buffer as also being a stage in perceptual information processing, and in more recent work he has explicitly identified it as composed out of the several retinotopic maps of the brain's occipital cortex (Kosslyn, 1994). The evidence regarding this claim, however, is conflicting. Although Kosslyn and others (Kosslyn et al., 1993; Damasio et al., 1993; Kosslyn, Thompson, Kim, & Alpert, 1995) have detected activity in the relevant brain areas during visual imagery, other researchers (Roland & Gulyás, 1994; Mellet et al., 1996) find no such activity, and argue that imagery is more consistently associated with activity in other, non-retinotopically organized regions. Neurological patients who have lost the retinotopically mapped regions in one cerebral hemisphere, leaving them blind in the corresponding half of their visual field, show certain impaired imagery abilities in the blinded hemifield (Butter, Kosslyn, Mijovic-Prelec, & Riffle, 1997; Farah, Soso, & Dasheif, 1992). However, other patients suffering from cortical blindness due to damage in these areas seem to have relatively normal imagery (Chatterjee & Southwood, 1995). Furthermore, some patients with localized damage in the retinotopically mapped areas experience vivid, well-formed "visual hallucinations" (i.e. imagery that is outside of conscious control--they do not typically mistake it for reality) precisely in the affected (blind or "blindsighted") parts of their visual fields (Ramachandran & Hirstein, 1997; Weiskrantz, Warrington, Sanders, & Marshall, 1974). This suggests that these brain areas cannot be essential for visual imagery.
However, even if Kosslyn proves to be mistaken in equating the occipital retinotopic maps with his imagery buffer, this will not in itself entail the wholesale rejection of his research program. It could still quite reasonably be argued that the buffer may be located elsewhere in the brain, mapped in a less obvious way. No particular claim about the buffer's neurological implementation should be taken as among pictorialism's core commitments.
Kosslyn and Shwartz (1977) developed a computer model of Kosslyn's theory, and its instantiation of the "surface representation", the model for the mental image itself, is revealing. Figure 2 is produced by selectively filling cells in a rectangular array or matrix, which models the visual buffer of the underlying theory. Note that certain cells contain different letters. Kosslyn plausibly assumes that images fade over time, and successive letters of the alphabet represent older, successively more faded portions of the image. The cells, then, are not simply filled or unfilled, they contain symbols representing qualitative features of the image at that point. The theory implies that each cell might contain multiple symbols, representing such things as color, 3-D depth and the presence of edges (Tye, 1991; Kosslyn, 1994) (although these aspects have not been computationally implemented).
Simulated image ("surface representation") of a car, as printed out by the Kosslyn & Shwartz (1977) program.
Following figure 1, the array representation is constructed by following a stored description, and can thus readily be manipulated in various ways: being redrawn at various different sizes, positions and orientations on the buffer, and with varying amounts of detail included. This allows Kosslyn to account for the rotation, scanning, and size/inspection time effects mentioned above (§2).
It is important to note that the images of Kosslyn's theory are "quasi-pictures" or "functional pictures" rather than pictures in a literal sense (Tye, 1991). After all, the array representation in the computer (and, if it exists, in the brain) is not actually visible. The real model for the mental image is not the screen display but the underlying array representation in the computer's memory, physically instantiated as electronic states in RAM chips. Image manipulations are carried out on this array, and are merely mirrored on the screen or printout. Furthermore, the elements of the array in the memory need not be laid out in the actual spatial arrangement seen in figure 2. The representations of adjacent cells in the printout need not be in physically adjacent sections of RAM. All that matters is that they be treated as if they were adjacent by the computational routines which form, transform and inspect the array (Kosslyn, 1980).
Moreover, if, as suggested, each cell in the array holds several symbols, for various represented qualities, there is no need for even the symbols within a single cell to be represented in physically close memory elements. We might have multiple arrays, one for each sort of quality, so long as the accessing routines treat them as a single, superposed, array with multiple symbols in each cell. This is relevant, because in Kosslyn's (1994) neuropsychological version of the theory the visual buffer is taken to be composed of multiple arrays, instantiated in the multiple, specialized, retinotopically mapped areas of visual cortex.
However, standard philosophical objections to picture theories have not necessarily been circumvented by all this (Slezak, 1993, 1995). Pace Kosslyn (1994), the theory (and, indeed, any perceptual theory of similar form) still relies on an "undischarged homunculus" (Dennett, 1978), the "mind's eye function", to interpret the surface representation. No amount of neurophysiological detail is, in itself, going to solve this problem. In fact, the basic functional organization of Kosslyn's theory (figure 1) does not differ from that of Descartes' theory of imagery, which also involves a merely quasi-picture (see Treatise on Man and Optics in Cottingham, Stoothoff, & Murdoch, 1985). Of course, the theories do differ when we move beyond the functional level. Descartes, for example, put his "visual buffer" on the surface of the pineal gland instead of in the occipital cortex. More significantly, Descartes ascribed his mind's eye function to an immaterial, scientifically inaccessible soul, while Kosslyn clearly believes that his will eventually be explicable in purely physicalist terms. Kosslyn, to borrow Dennett's (1991) term, is a "Cartesian materialist"4. This is not an indefensible position (Attneave, 1961), but it is certainly problematic.
The specific ways in which deep representations and surface representations are stored in the Kosslyn-Shwartz program, and how various transformations of the quasi-picture are generated, need not detain us. There are many possibilities. Several different ways of implementing (symbol filled) array theories of imagery have been proposed by AI researchers since the original Kosslyn and Shwartz work; some connectionist (e.g. Julstrom & Baron, 1985; Mel, 1986, 1990; Stucki & Pollack, 1992) and some not (e.g. Glasgow & Papadias, 1992; Glasgow, 1993). These all have interesting features, but none attempt the psychological range of Kosslyn's model. The core commitment of the quasi-pictorialist research program is not to any particular details of implementation, but to the (symbol filled) array representation for the image itself, and to something resembling the functional organization of figure 1.
Description theory is also known amongst psychologists as "propositional theory", because the data structures that it takes to embody mental images (and percepts too) are regarded as expressing propositions descriptive of the relevant perceptual scenes. When such structures are the end product of perceptual processes, we are perceiving; when they are constructed inventively, or retrieved from memory, we experience imagery. In effect, the data structures are sentences in an inner language, "mentalese" (Fodor, 1975), and the core commitment of description theory is that such language-like representations are sufficient to account for all genuinely cognitive processes, including the experimental effects attributed to imagery. Pylyshyn, the principal advocate of description theory, explicitly and forcefully urges the view that this conception of an inner "notation" through which thoughts are "written in the mind" should be understood quite literally and realistically (Pylyshyn, 1991). Of course, description theorists consider the content of the representations involved in imagery to be perceptual, but they consider that there is nothing specifically perceptual about their format. This format does not differ from that of the representations underlying other cognitive processes. Furthermore, no specifically perceptual processing mechanisms need be active when imagery is experienced (Pylyshyn, 1978, 1984).
The earliest attempts to implement computer models of imagery phenomena were made in the Carnegie-Mellon school of Artificial Intelligence research, where the established approach was to examine people's strategies in solving logical and mathematical puzzles (largely through the analysis of protocols produced by subjects talking through their puzzle solving out loud) and then to attempt to program a computer to solve the same puzzles by a similar strategy (Newell & Simon, 1972). Baylor (1972) and Moran (1973) attempted to apply this approach to certain puzzles that people seem to solve using visual imagery. Baylor (1972) took on Guilford's (1967) "block slicing" puzzles, and devised a way of describing rectangular blocks using LISP data structures. Blocks were represented in terms of nested lists of faces, edges, and vertices (V1 through V8), and property lists holding information like edge length and face color. Using such representations, he wrote a program that solved certain of the puzzles using a sequence of steps similar to that followed by a human solver (according the protocol analysis). Moran (1973) achieved comparable results applying similar techniques to another type of imagery puzzle.
Neither Baylor nor Moran took any significant account of the experiential, quasi-perceptual nature of imagery. Neither did they give much heed to the (then very new) experimental results on rotation, scanning, etc. Later attempts to simulate imagery computationally do take account of these things (see §2.1.2), but, simultaneously, abandon treating the image representation as a sentential description. Nevertheless, description theory has continued to play a prominent role in discussions of imagery. This is very largely due to the vigorous advocacy of Pylyshyn, who initially seems to have seized on Baylor's and Moran's work as an alternative to what he took to be the naïve and unworkable pictorialist views implicit in early cognitivist research on imagery (Pylyshyn, 1973). Subsequently he turned his efforts to attacking the more explicit conception of images as "analog" representations then being developed by Kosslyn (1975, 1980), Shepard (1975), and others. In this work Pylyshyn (e.g. 1978, 1981) recommends description theory as greatly theoretically preferable to picture theory, but he does little to develop its details.
Versions of a PA approach to imagery were outlined by several psychologists in the 1960s and 70s, including Hebb (1968); Hochberg (1968); Sarbin and Juhasz (1970); Janssen (1976); Morgan (1979); and Neisser (1976, 1978). More recently, Ellis (1995), and Ramachandran and Hirstein (1997 p. 442) have sketched similar views. (See also Dennett  on hallucination.) PA theory does not involve CM or the information processing framework. However, that does not mean that it cannot be modelled computationally. Indeed, Farley (1974) produced a computer model based on Hochberg's version of the theory (like the Baylor and Moran models, however, it has very limited psychological scope). Otherwise, there has been very little cross-fertilization between the theorists, little directly relevant experimental research (what there is being inconclusive), and little critical discussion (and consequent defense and elaboration) of any of the versions (Thomas, 1987, 1997b). As a consequence, even the best developed published exposition (Neisser, 1976) remains very sketchy.
This neglect of PA theory, however, may be mainly due to historical accident. In the 1970s, information processing theory was decidedly in the ascendant, and there was a very high profile battle between the description theorists and the quasi-pictorial theorists and their allies (the "analog-propositional dispute") over the best way to understand imagery within this framework. Consequently, most subsequent discussions treat these two approaches (plus, perhaps, some hybrid of them [e.g. Chambers, 1993)] as exhausting the alternatives. Such opposition as there was to information processing theory (and CM) mostly coalesced around the work of Gibson (1966, 1979). Unfortunately, Gibson's metatheoretical rhetoric (if not, perhaps, his positive theory--see Bickhard & Richie, 1983) emphatically rejects using the concept of mental representation (explicitly including mental imagery) within the context of perceptual theory. Thus, although Neisser's PA theory was deeply influenced by Gibson (and Gibson himself [1970, 1979 p. 256] hints at a PA view), the study of imagery has not flourished in this tradition.
The information processing approach to perception originated in computer vision research, where it was long taken as axiomatic that the task of a machine vision system was to process the signal produced by its TV-camera sensor into an internal descriptive model of the world before the camera's lens. However, this orthodoxy has recently been challenged by the so called active vision (or active perception) approach (Bajcsy, 1988; Ballard, 1991; Bajcsy & Campos, 1992; Blake & Yuille, 1992; Aloimonos, 1992, 1993; Swain & Stricker, 1993; Landy, Maloney, & Pavel, 1996; Scassellati, 1998). This approach is clearly influenced by Gibson, but may be seen as part of the wider movement toward understanding cognitive systems in terms of their environmental situatedness (Clancey, 1997). Active vision systems are typically robotic rather than merely computational, and instead of being used to build a comprehensive inner model of its surroundings, the robot's perceptual capacities are simply used to obtain whatever specific pieces of information are currently necessary for the ongoing control of the its behavior in the world.
Visual sensory data is analyzed purposefully in order to answer specific queries posed by the [robot] observer. The observer constantly adjusts its vantage point in order to uncover the piece of information that is immediately most pressing. (Blake & Yuille, 1992--emphasis added).
"Activity" should here be understood in a strong, literal sense that goes beyond and subsumes views that invoke it merely in the sense of top-down control of information processing within the computational architecture of a vision system (or a brain). Not only does the robot actually move its TV-camera sensors to point at whatever it presently needs to know about; different specialized algorithms are purposefully applied to extract the specific sort of information (e.g. shape, distance, velocity, texture) that the system currently requires. Often, these algorithms themselves may involve purposefully moving the sensor relative to what is being examined (Aloimonos, Weiss, & Bandyopadhyay, 1988), actively picking up information (c.f. Gibson, 1966, 1979) rather than passively transducing.
Although they have not (to my knowledge) applied their ideas to imagery, active vision researchers seem to have revived the general outlook on perception formerly taken by PA theorists. With certain more neuroscientifically oriented thinkers also moving in a similar direction (Churchland, Ramachandran, & Sejnowski, 1994; Cotterill, 1997), perhaps the time is ripe to revive PA imagery theory itself.
In consideration of the underdeveloped state of extant versions of PA theory, and the failure of most Gibsonians and Situated Cognition researchers to apply their insights to imagery, in this section I give my own tentative synthesis. Conjectures about matters of detail are intended as merely illustrative. I do not aim to provide a finished, detailed theory, but, rather, some concrete sense of how such a theory might work.
In the spirit of the proceduralist approach to memory (Kolers & Roediger, 1984), perceptual learning is not viewed as a matter of storing descriptions (or pictures) of perceived scenes or objects, but as the continual updating and refining of procedures (or "schemata" [Neisser, 1976]) that specify how to direct our attention most effectively in particular situations: how to efficiently examine and explore, and thus interpret, a scene or object of a certain type (Stark & Ellis, 1981). Through such processes of controlled perceptual exploration we collect the information that takes us from a vague, preattentive appreciation that something is out there, to a detailed understanding of just what it is. We engage in "a rapid sequence of microperceptions and microreactions, almost simultaneous as far as consciousness is concerned" (Damasio & Damasio, 1992), and it is through this attentive process of searching out the distinctive features and feature complexes of the things before us, that we come to recognize and categorize them, to perceive them as whatever they may be.
On this view, no end-product of perception, no inner picture or description is ever created. No thing in the brain is the percept or image. Rather, perceptual experience consists in the ongoing activity of schema-guided perceptual exploration of the environment. Imagery is experienced when a schema that is not directly relevant to the exploration of the current environment is allowed at least partial control of the exploratory apparatus. We imagine, say, a cat, by going through (some of) the motions of examining something and finding that it is a cat, even though there is no cat (and perhaps nothing relevant at all) there to be examined. Imagining a cat is seeing nothing-in-particular as a cat (Ishiguro, 1967).
impulses in the optic nerve fibres at each moment of scanning a scene are the answers, in code, to the "questions" that had been asked at the previous moment. . . . [What] goes on in the retina is not like the recording of a "picture", but the detection of a series of items, which are reported to the brain (Young, 1978).
Information gathering is a dynamic process that responds at once to events in the visual world, to the system's evolving understanding of that world, and to changing requirements of the vision task (Swain & Stricker, 1993).
The properties tested may sometimes be complex, relational ones. In robotic active vision systems the TV-camera sensor may function at one moment as part of an instrument for determining shape from shading, at another as part of an instrument for estimating relative distance from parallax, and so on, according to which particular algorithm currently has control of it (control of both its motion and the analysis of its output). Ballard (1991) calls this capacity of a single sensor to function as needed as a component of different perceptual instruments "sensor fission". The PA theorist will hold that human vision, indeed perception in general, may be usefully thought of in a similar way: We do not so much have five, general purpose senses, as a large array of anatomically overlapping perceptual instruments, a capacious "box of tricks" (Ramachandran, 1990).
A Perceptual Activity theory of imagery.
Dotted arrows represent processes operative during perception, but which may be suppressed or attenuated during imagery.
The right-hand box in figure 3 represents the array of perceptual instruments available to an organism. I take it that higher organisms are endowed (perhaps innately) with many such instruments, each specialized for making a particular type of perceptual test. In many cases different instruments may share many of the same anatomical resources: not only receptor cells and muscles, but even some of the same neural structures. Which particular instrument is in play at any one time will depend not so much on which anatomical structures are being used, as on which neural "algorithm" is in control of them. Switching between such algorithms may be possible "on a rapid (~100 ms) time scale" (Van Essen, Anderson, & Felleman, 1992). The seeming immediate presence of the visual world to consciousness does not arise because we have built a detailed internal representation of it, rather it is (like the ever shining fridge light) a product of the "immanence illusion" (Minsky, 1986): For the most part, the visual perceptual instruments ask and answer their questions so quickly and effortlessly that it seems as though all the answers are already, and contemporaneously, in our minds.
Haptic tests involve applying the perceptual instrument directly to an object. Other sense modes, however, apply their instruments to causal products of the real objects of interest: products that are reasonably reliably correlated with properties of the objects themselves. Thus we detect the presence of the gorgonzola by sniffing at the foetid air it produces. Likewise, our ears interact not with the speaking person but with the vibrating air, and our eyes interact directly not with the things we see but with the "optic array" (Gibson, 1966, 1979), the ambient structured light that is scattered from them. Figure 3 attempts to depict this situation by the shaded area around the object of perception, and by the fact that the double headed arrows that represent the perceptual tests reach into this area, but not necessarily right to the perceived object itself.
However, things are yet further complicated because it seems quite likely that some perceptual instruments may be applied to causal products within the perceiver's body (arrows within the right hand box in figure 3). Although some auditory tests (for example for locating a sound source) involve moving the head, and thus the ears, relative to the vibrating air, it may be that other tests are applied within the ear itself, to the vibrational state of the cochlear fluid (itself, of course, a causal product of the vibration of the sound source). The cochlea receives a considerable degree of efferent innervation, which seems to control movements of "hair cells" within its fluid-filled interior. These movements are thought to dynamically regulate the cochlea's sensitivity and frequency tuning, and possibly underlie active discrimination of specific speech components (Dallos, 1992, 1997).
Similar considerations may even apply to the retina of the eye. There is a good deal of evidence for efferent fibers in the optic nerve, fibers that carry signals from the brain to the retina (Repérant, Miceli, Vesselkin, & Molotchnikoff, 1989). According to Pribram (1991) these comprise about 8% of the fibers in mammalian optic nerve, and serve to modulate the afferent, sensory signals carried by the other fibers, bringing them under directionally specific attentional control. Although some of the evidence for this has been disputed (Mangun, Hansen, & Hillyard, 1986), it is certainly consistent with the notion that the optical image on the retina may have various different tests applied to it by selective tuning of receptor arrays under central control.
It may also remain useful to think of some perceptual instruments as working completely within the brain, making their tests upon "representations" which are, in effect, "bottom-up", passive causal products of the stimulus object. I think it is a serious mistake to think of such neural representations as "mental" merely because they occur in the brain, but they do carry information about their causative objects, much as a shadow, a photograph, or a TV signal does. Some visual tests (notably those involving eye movements) seem to be applied directly to the optic array (Gaarder, 1966; Gibson, 1966, 1979), and there is reason to think that these are, evolutionarily, more basic (Horridge, 1987). However, considerable information can be obtained from brief tachistoscopic displays, and this, together with the related phenomenon of "iconic memory" (Neisser, 1967), suggests that other tests are applied to bottom-up neural (as opposed to "mental") representations in retinotopically mapped cortex.
It would not contravene the core commitments of the PA research program if all perceptual tests were like this (although it seems empirically unlikely, from the PA perspective, to truly be the case). If it actually were so, then we would be returned to an understanding of visual perception consistent with Kosslyn's view as expressed in figure 1, with a visual buffer subject to examination by an active mind's eye. Note, however, that this would not amount to embracing a quasi-pictorialist imagery theory. Kosslyn's theory requires that an image be established in the visual buffer before the mind's eye can go to work and examine it. PA theory, even in this completely "internalized" form, makes no such demand: PA imagery no more requires an image in the buffer than it requires an object before the eyes; it is the testing activity that matters, rather than to what (if anything) the tests are applied.
Nevertheless, the likely fact that some perceptual tests are directed at neural, bottom-up representations may help to make sense of the otherwise contradictory findings, previously discussed (at the end of §2.1.1), concerning activity in retinotopically mapped brain regions during imagery. Such perceptual tests would involve sending efferent signals to retinotopic cortex (the pathways for this certainly exist [Felleman & Van Essen, 1991]), and we should thus expect some activity in such cortex if these tests are deployed during imagery. Any such activation, however, would be an effect, a by-product, of the process of imaging rather than its necessary causal antecedent. This would explain why, although visual imagery may often be accompanied by activation in the retinotopically mapped areas, it can sometimes occur without such activation (the relevant tests are not in play) (Mellet et al., 1996), or even when the relevant parts of these areas have been destroyed (Ramachandran & Hirstein, 1997; Weiskrantz et al., 1974).
Considering the possibility of perceptual tests directed at the retinotopic areas should also help to make it clear how the PA view of imagery and perception fits with theories of perceptual consciousness of the sort proposed by Marcel (1983) and Gray (1995). These authors assemble a considerable range of empirical evidence supporting the view that we are not conscious of "automatically" produced (i.e. bottom-up, neural) perceptual representations, but, rather, that "conscious percepts" arise from "a constructive act of fitting a perceptual hypothesis to its sensory source" (Marcel, 1983). Marcel explicitly identifies his concept of perceptual hypotheses with the concept of schema that we have been using, and, except for the fact that he only considers an entirely intra-cranial process (i.e. testing hypotheses only against neural representations, rather than, say, the optic array, or the object itself), "fitting a perceptual hypothesis to its sensory source" would seem to mean much the same as carrying out a set of perceptual tests under the direction of a schema. Gray (1995) explicitly notes the similarity of his account of conscious perception with the PA theory of Neisser (1976).
The suggestion is that during imagery, the schema is active in much the same way that it is during perception. It still sends out at least some of its "orders" to the perceptual instruments, and selects procedural branches to follow. However, the reciprocal control of the schema's activity by the perceptual instruments is lost, or at least much attenuated. Hypotheses are still being put forward, as it were, giving rise to (potentially) conscious experiences, but they are not subject to testing against reality. This might either be because the instruments themselves (or some of them) are inhibited from carrying out their tests (Marks, 1973; Jacobson, 1932), and so fail to return any meaningful result, or it might be that results that are returned are disregarded, or not given full weight, by the schema. Either or both these processes may well occur in particular cases. However this may be, the processes that do occur will still display the intentionality of perceptual consciousness, being still directed towards an object, albeit a merely intentional ("intentionally inexistent") object rather than a material one (Anscombe, 1965; Ishiguro, 1967). It is important to stress that, on this theory, no thing or state in the mind or brain corresponds to the percept or image. Although the schema may be considered the repository for what we learn about the world through perception, and although it controls our conscious perceptual activity, it should not be identified with that activity. As has been independently argued (Rabb, 1975; Heil, 1982; Tye, 1984), strictly speaking there is no percept or mental image, only the activity of perceiving or imagining, which takes different forms according to what is being perceived or imagined.
The above account of PA theory admittedly needs further development. Little has been said about the internal structure of perceptual schemata, how they interact with more general aspects of behavioral control, or how they are modified in response to new information that conflicts with the perceptual expectations they embody. In this regard, however, we can appeal to standard accounts of schemata (e.g. Rumelhart, Smolensky, McClelland, & Hinton, 1986), which will serve no worse (if no better) than they serve the numerous other psychological theories which rely on the concept. We also lack much detail about what perceptual instruments there are, and how they might work (and details that are given are speculative). Such facts will only be ascertained through detailed empirical investigations from a PA perspective (although there are surely many suggestive leads in the existing, especially the Gibsonian, literature). But these details do not, in any case, belong to the core commitments of the PA research program, any more than specific claims about neural realization or coding schemes are constitutive of picture or description theory. The core commitments are: the view of categorical perception (perceiving as) as the active exploration and interrogation of the environment; the view of imagery as the (partial) commission of such an exploration in the absence of the appropriate object (going through the motions of an interrogation without regard, or full regard, to answers received); and the rejection of CM (i.e. of the identification of conscious contents with neurally realized data structures). A schema may very well be best understood as a computational data structure, but we are not conscious of our schemata. It is not the schema but the activity it supports that carries intentionality and embodies our perceptual and imaginal awareness.
The following sub-sections will review certain issues for the three theories that arise from experimental findings. They do not aim at comprehensiveness; neither do they aim definitively to refute or establish any of the theories. Research programs are not susceptible to refutation or verification. The aim is merely to raise some questions about the explanatory power of two the better known approaches, and to suggest that PA theory may have at least a comparable explanatory potential. The fact is that the methods of experimental psychology have not, as yet, been sufficient to bring about a resolution of the imagery debate (Pylyshyn, 1994), and some authors have gone so far as to suggest that they cannot be (Anderson, 1978; Palmer, 1978). Thus, in §3 and §4, I will turn to broader considerations of theoretical scope and context.
Kosslyn's picture theory was designed to explain the "classic" imagery effects (rotation, scanning, mnemonics, etc.) outlined in §2, and, although doubts have been raised (e.g. Pylyshyn, 1979; Yuille, 1983), it does a generally creditable job (Kosslyn, 1994; Kosslyn, Holyoak, & Huffman, 1976; Kosslyn & Pomerantz, 1977). However, results on the reinterpretation of images, and on imagery in congenitally blind subjects may be less easily accommodated.
The Necker cube and the duck-rabbit.
Chambers and Reisberg (1985) briefly showed subjects examples of single "reversible figures", such as the Necker cube or the duck-rabbit (figure 4), with instructions to "form a mental picture" of the figure, so they could later draw it. Next, the subjects were asked to visualize the figure and attempt to "see" the alternative interpretation in their image. (It had been determined that the 5 second exposure time was too short for subjects to see more than one of the two possible interpretations whilst actually viewing the figure.) Subjects were unable to find a second interpretation from their image in any of the 55 trials made. This was despite being allowed as much time as they liked; despite having previously been familiarized with reversible figures of a similar type; and despite being given hints as to how to get a figure to reverse by spatial attention shifting. Nevertheless, when they subsequently made their drawings, subjects were able to recognize the alternative interpretation in their own drawings in all but four trials. Similar failings at image reinterpretation tasks have been reported by many other experimenters, including: Reed (1974); Palmer (1977); Hinton (1979); Reisberg, Smith, Baxter, and Sonenshine (1989); Reisberg and Chambers (1991); and Slezak (1991, 1995).
Picture theory would lead us to expect that, provided sufficient detail has been encoded (and the success with the drawings seems to confirm this), we ought to be able to reinterpret a mental image quite as easily as we can reinterpret the equivalent real picture. It has been shown that reinterpreting the "reversible figures" depends on shifting the focus of visual attention across the picture (Tsal & Kolbert, 1985), and Kosslyn (1994) explicitly holds that we can spatially shift our attention across the visual buffer; nevertheless, it is easy to reinterpret the duck-rabbit picture and very hard to reinterpret its image. True, it has now been shown that images of certain rather different figures can be reinterpreted somewhat more easily, and that under special conditions even images of the duck-rabbit and its ilk can sometimes be reversed (Finke, Pinker, & Farah, 1989; Peterson, Kihlstrom, Rose, & Glisky, 1992; Brandimonte & Gerbino, 1993; Cornoldi, Logie, Brandimonte, Kaufmann, & Reisberg, 1996). Nevertheless, this work does nothing to undermine the integrity of the original results, and others like them. An important disanalogy between imagery and pictures remains (Slezak, 1993, 1995). People clearly experience significant difficulties in reinterpreting images under conditions where they have very little trouble with the equivalent picture.
Kosslyn (1980, 1994) attempts to account for these results in terms of the differential fading of parts of the image. According to his theory, a complex image is built up in the visual buffer by the sequential placing of its various sub-parts, and a part will begin to fade as soon as it is written to the buffer. Thus, by the time a complex image is fully assembled the earlier parts will be considerably fainter than the newer. Although the fading parts can be successively "refreshed", the whole image can never attain a uniform vividness. He then argues that if reinterpreting an image involves reconstruing the relationships between parts, or the way that it is segmented into parts, (as is plausibly the case with the experiments in question) then the differential vividness of the relevant parts explains the difficulties of such reinterpretations.
This seems coherent, but is hardly without problems. For one thing, it is not clear that it accords with the phenomenology of imagery. Folk psychology does acknowledge that images fade, but I, at least, can never recall experiencing an image distinctly more faded in some structural parts than in others. I may focus in on one part, so that the rest fades from my attention, but this does not seem to be what Kosslyn is describing. Another, less subjectively based, objection is that many experimental tasks that subjects can do well seem to involve simultaneously attending to, and even manipulating, multiple parts of images that are at least as complex as those used in the reinterpretation experiments. For example, some of the foundational experiments of Kosslyn's own research program call for the simultaneous visualization of two animals (Kosslyn, 1975), and many of the mental rotation experiments of Shepard and his students (Shepard & Cooper, 1982) involve manipulation of images of complex irregular polygons or three-dimensional assemblages. The duck-rabbit and the Necker cube seem simple by comparison.
Chambers and Reisberg themselves (1992; Reisberg, 1994) provide an alternative pictorialist explanation of their results. They suggest that pictorial images occur paired with a second form of representation, a "reference frame", which biases either how the image is read by the "mind's eye", or else helps to determine which of the telling details specified in the "deep representation" actually gets a detailed instantiation in the "surface display". However, this epicycle only seems to push the problem back a step. Instead of wondering why it is hard to reinterpret images we are left to wonder why it is hard to switch reference frames appropriately during imagery, but not during perception.
It is also surely a considerable embarrassment for picture theory that almost all the "classic" effects that originally served to convince the psychological community of the reality and cognitive significance of visual imagery (§2) have since been demonstrated in the congenitally blind, who can hardly be capable of forming mental reconstructions of former visual sense impressions. Mnemonic effects of imagery in the blind were found by Craig (1973); Jonides, Kahn, and Rozin (1975); Zimler and Keenan (1983); and Kerr (1983). Kerr also reports mental scanning, and size/inspection time effects. Mental rotation in the blind is reported by Marmor and Zaback (1976), and Carpenter and Eisenberg (1978). Hampson and Duffy (1984) found interference effects. Effects may be weaker, or reaction times slower, than in the sighted, but formally the results are very similar.
No doubt the blind subjects rely mainly on "tactile/kinaesthetic" (i.e. haptic) imagery, as is evidenced by the way that they fail at more purely visual imagery tasks, such as perspectival judgements (Arditi, Holtzman, & Kosslyn, 1988). However, quasi-pictorialist forms of explanation do not readily carry over to the haptic mode. Although a "passive-touch buffer" (mapping the skin much as the visual buffer is supposed to map the retina) seems conceivable, it would not be remotely adequate to the tasks in question. Haptic knowledge of objects and spatial relations is mediated by active, exploratory movements, and involves a complex coordination between tactile sense, proprioception, and motor control. There would seem to be little place for any equivalent to the sort of bottom-up, relatively peripheral object representations that are supposed to inhabit the visual buffer. This does not, of course, prove that quasi-pictorialist explanations will not work for visual imagery. However, it would be very remarkable for two quite disparate forms of imagery mechanism to engender such formally similar experimental results.
According to Intons-Peterson and Roskos-Ewoldsen (1989) the results on imagery in the blind helped to persuade many that "images are amodally spatial, and perhaps not visual in nature" (presumably an allusion to description theory). However, as Kosslyn and Pomerantz (1977) have argued, description theory does not deal very satisfactorily with the "classic" imagery effects (§2). Although it is possible to devise explanations for them in terms which are consistent with basic descriptionist tenets, these explanations generally require otherwise quite unmotivated, ad hoc assumptions. None of the effects would have been predicted from the theory's core principles. Of course, we would have to tolerate such ad hoc explanations if there were no coherent alternatives to description theory, and Pylyshyn (1978, 1984) tries to establish that there are none by bringing conceptual objections against the very notion of fundamentally non-language-like mental representation. However, these arguments do not appear to be sound (Tye, 1991). With the demolition of these metatheoretical arguments of Pylyshyn's, much of the motivation for taking description theory seriously would seem to have dissipated.
If such illusions truly can be induced through imagery, it raises a serious problem for description theory. The "cognitive impenetrability" of all these illusions (the fact that the illusion persists even when we "know better") is surely, from a computationalist perspective, excellent evidence that they arise in modularized perceptual input systems (Fodor, 1983). But Pylyshyn (1978) was determined to deny that imagery effects involve such systems. His whole commitment to description theory hung on the view that imagery arises entirely from "propositional" mentalese representations in the "central systems", and is thoroughly "cognitively penetrable".
Neuropsychological findings seem to reinforce this point. There is now much evidence that visual imagery depends on activity in parts of the brain otherwise considered to be dedicated to visual perception (Farah, 1988; Kosslyn, 1994). Such parts would naturally be construed, on the Pylyshyn-Fodor view, as embodying just the sort of "cognitively impenetrable" (Pylyshyn, 1978), "informationally encapsulated" (Fodor, 1983) input modules that are supposed not to participate in imagery. Furthermore, Bisiach and Berti (1990) argue that (quite apart from the conceptual problems alluded to in §2.1.1) the very notion of language-like, "mentalese" representation is neuropsychologically implausible.
Given the undeveloped state of PA theory, the explanations given here must be speculative. The intent is merely to show that PA theory is not obviously inadequate to the empirical facts against which the other theories measure themselves.
PA theory should have little difficulty with evidence suggesting the involvement of specifically perceptual mechanisms in imagery, as presented in the previous sub-section. Schemata and perceptual instruments are supposed to subserve both perception and imagery. It is entirely to be expected that quirks and defects of their functioning would show up in both modes, and that at least some of the same brain areas would be involved.
Kosslyn et al. (1976) explain imagery's mnemonic effects in terms of Paivio's "Dual Coding" theory (1971, 1986, 1991), which posits two interacting systems of memory, one linguistic/conceptual and the other imaginal. As neither picture theory nor PA theory assimilates imagery to linguistic/conceptual representation, both may appeal to dual coding theory in this regard.
Mental scanning is readily explained by PA theory's claim that imagery consists in covertly going through the motions of the equivalent perceptual process. If we steadily scan our gaze across a real scene, it will take longer to move through a large visual angle than a smaller one. It is thus to be expected that if subjects covertly go through the motions of such scanning (as they are constrained to do in the relevant experiments, either by explicit instructions or experimental conditions) they will also take longer the larger the angle. Even if actual head and eye movements are suppressed, the schema's abortive attempt to initiate and control them should follow a comparable temporal course to real visual scanning. Similar considerations apply to size/inspection time effects. Other things being equal, it takes longer to pick out small details of a scene or object before us than it does to pick out large ones; we may have to peer, or move closer. Likewise, it will take longer to covertly go through the motions of picking out smaller details8.
PA theory also seems well adapted to explaining interference between imagery and perceptual tasks: Interference occurs when each task simultaneously makes conflicting demands upon the same perceptual instrument, or upon different instruments that share some of the same anatomical resources. It has long been recognized that it is not adequate to analyze the complex findings in this area in terms of a simple categorization of tasks according to sense mode. The PA account suggests the possibility of a finer grained, more explanatorily powerful categorization. However, the details must await advances in our knowledge of the perceptual instruments.
Mental rotation is more problematic. Neisser (1976) suggests that it should be understood as the product of schemata specialized for the perception of rotating objects; or perhaps we might better say, schemata that invoke perceptual instruments specialized for estimation of rotational motion. These suggestions, however, fail to explain why a subject's mental rotations (within a particular experimental paradigm) always seem to go at a specific preferred speed (Hampson & Morris, 1978). It may be better to follow Kosslyn's recent suggestion (1994) that the effect is controlled by the motor system, and that what we are imagining is successive orientations of the object as if we were physically turning it. Of course, Kosslyn interprets this in terms of the creation of a succession of differently oriented pictures on the visual buffer, but PA theory can equally well invoke such a succession of imaginings.
Unlike picture theory, PA theory addresses both haptic and visual imagery in formally similar explanatory terms. Indeed, our account of PA theory in §2.3.1 developed the example of haptics before extending the approach to the other modes. Thus the homology between experimental findings on the haptic imagery of the blind and on the mainly visual imagery of the sighted is unsurprising for the PA theorist, and presents none of the special problems that it raises for picture theory.
The difficulties that subjects have in reinterpreting images also make sense in PA terms. We know that the interpretation that we see depends on the way in which we look at the figures; in particular, on how we direct our attention spatially (Stark & Ellis, 1981; Tsal & Kolbert, 1985). For PA theory it is precisely such ways of looking that get stored in visual memory and are reactivated as imagery. If we are only exposed to an unfamiliar figure long enough for us to successfully undertake one of the possible ways of looking at and so interpreting it, only that one way will be stored, and our subsequent imagery will involve recapitulating only that one way, giving us access only to the one interpretation9.
Many diverse ideas are associated with the idea of imagination. My purpose here is merely to document the centrality of three of them to the folk/Romantic conception: non-rationality (or non-discursive thinking), creativity, and imagery. I will particularly consider the views of the best known English Romantic poets: Blake, Wordsworth, Coleridge, Shelley, and Keats. It is these writers, rather than any philosophers or psychologists, who have done most to shape the "folk" conception in its ideologically significant form.
During most of the history of Christendom, imagination was generally considered negatively, a dangerous faculty, liable to lead us into sin and error, and opposed to reason, which would lead us to God (Kearney, 1988). This attitude persisted into the early modern period, even as reason was coming to be linked with science rather than religion. Imagination was associated with the kind of magical thinking which the new "mechanical philosophy" was struggling to overcome (Pagel, 1958; Yates, 1966). Blaise Pascal (1670/1973), mathematician, physicist, and devout Christian, was particularly vehement in this regard, characterizing imagination as "that mistress of error and falsehood," an "arrogant faculty, the enemy of reason".
"To take off his filthy garments & clothe him with Imagination,10.
Wordsworth was presumably using "reason" as a generic term for cognitive power, of which imagination is one species. Pascal and Blake, however, were contrasting imagination with another species of cognitive power, discursive or "mathematical" reason. John Keats made the contrast explicit when he compared imagination favorably with "consequetive reasoning"11, and Shelley (1840/1954), in similar vein, distinguished reason, the "principle of analysis", from imagination, the "principle of synthesis". In fact, the distinction between imagination and discursive reason, phantasia and dianoia, is as old as Aristotle (De Anima 427b 15-16), and, perhaps because satisfying positive characterizations of imagination have always been hard to come by (Brann, 1991), the contrast between them has long been a key constituent of ordinary understandings of the concept of imagination (Kearney, 1988). The distinctive Romantic achievement was not to establish this contrast, but to sharply invert the relative valuation placed on its two terms.
But the Romantics valued imagination so highly not because of its non-discursiveness per se, but because they held it responsible for artistic creativity ("inspiration"). The idea of creative imagination did not originate with the Romantics, in fact it has a very long history (Kearney, 1988; Brann, 1991; Cocking, 1991), but it was the Romantics who gave it its modern prominence and significance. Largely because of their continuing influence, creativity remains one of our culture's most important values, in science as much as in the arts.
There has, however, been a tendency amongst 20th century analytical philosophers to deny the connection between imagination as creativity and imagination as the production of imagery. A widely used Dictionary of Philosophy (Flew, 1979) warns us firmly against confusing them. Philosophers routinely treat "imagine" as polysemous, as if its usages were a collection of homonyms, sometimes equivalent to "form (or experience) imagery", but, in other cases, to "suppose", or "pretend", "believe", "think creatively", and so on (Thomas, 1997a). This orthodoxy, however, has been powerfully criticized by White (1990). White, however, holds that although there is only one sense of "imagination", it has nothing whatsoever to do with imagery! As I have argued in detail elsewhere (Thomas, 1997a; see also Thomas, 1999a), White makes a persuasive case against the polysemy view, but he fails to make the case for dissevering imagination from imagery. A more plausible view is that "imagination" is a "family resemblance" term (Wittgenstein, 1953) whose prototypical (but not exclusive) application is to the production of imagery.
Anyway, there can be little doubt that the Romantic writers who so valorized the creative imagination still, like others before and since, thought that they were talking about a unitary faculty deriving its name from its responsibility for image production; they clearly thought mental imagery and creativity were intimately bound up with one another (Warnock, 1976; Robson, 1986; Brann, 1991). This opinion should be taken seriously. It is surely no accident that these views struck such a lasting chord in the minds of the Romantics' contemporaries and successors. Even recent psychologists, seemingly little burdened with Romantic ideology, still assume imagery must be a key factor in creative thought (Finke, Ward, & Smith, 1992; Weisberg, 1986).
Where it goes beyond mere distaste for Romantic posturing and metaphysical extravagance, the motivation behind the prevalent philosophical uneasiness with the Romantic notion of creative imagination arises, I suspect, from a not unreasonable suspicion that the theory of imagery which the Romantics inherited from their forebears, and which remains the "default" conception today (i.e. the picture/copy theory), is quite inadequate to explaining creativity.
Admittedly, picture theorists of the associationist tradition had an account of how we might imagine things never seen; through the separation and recombination of parts of pictorial images drawn from memory. Matthews (1969) disparagingly calls this "mental carpentry". We can, so the story goes, imagine a sphinx, for example, by taking the head from some memory image of a woman and sticking it onto the body from an image of a lion. Likewise, more modern technology suggests that we might also be able to "morph" our mental pictures, stretching, squashing, and bending their components into new forms. No doubt we really can in some way separate, shuffle, distort, and recombine features of our imagery, and no doubt the ability has its cognitive uses (Finke, Pinker, & Farah, 1989; Finke, Ward, & Smith, 1992). However, this is hardly plausible as the fundamental basis of artistic and scientific creativity. Artistic creativity is not mere inventiveness, and it certainly does not amount to the reshuffling (or distortion) of bits of copies of former sense impressions. Perhaps because picture theory seems to have little more to offer, and because until very recently (on the philosophic time scale) picture theory was the only serious imagery theory around, æstheticians have grown skeptical about making any deep linkage between imagery and creativity.
But the Romantics clearly did not equate creativity with some capacity to shuffle and recombine image parts. Coleridge (1817/1975) called this latter ability "fancy", sharply distinguishing it from true imagination. Imagination was the initial capacity to form images, and, indeed percepts. Its role in artistic creativity was "secondary". In the "primary" sense of the term, imagination was "the living Power and prime Agent of all human Perception" (Coleridge, 1817/1975), and the "principle of synthesis" (Shelley, 1840/1954): the capacity to comprehend the meaningless welter of incoming sensation, synthesizing it into a coherent, meaningful whole: the secret ingredient that turns mere mechanical receptivity into mental apprehension. This, in fact, was essentially the received, traditional philosophical understanding of imagination, deriving most directly from Kant, but ultimately from Neoplatonic expositions of Aristotle (which Coleridge knew well [Baker, 1957]), or, very arguably, from Aristotle himself (Nussbaum, 1978; Thomas, 1987, 1999a).
Unfortunately, nobody, from Aristotle onwards, had ever been able to give any naturalistic account of how this wonderful synthetic, meaning-creating capacity might work. Kant, at least, admitted the fact. He described it as "an art concealed in the depths of the human soul, whose real modes of activity nature is hardly likely ever to allow us to discover" (Kant, 1787/1929). Coleridge and the other poets left us instead with a mixture of warmed-over associationist notions, clearly inadequate to the demands being placed upon them, and fragments of grandiose idealist metaphysics, ripe for mystificatory appropriation. Perhaps in the age of cognitive science we can at last do better.
Let us now consider how well the three contemporary theories of imagery fit together with the traditional/Romantic conception of imagination just outlined.
Whatever its merits might be as a theory of imagery, description theory holds very little promise of providing an account of imagination in the traditional and significant sense just outlined. Its view of mental processes, even imagery processes, as consisting exclusively of the manipulation of sentential, mentalese representations, could hardly be more rationalistic. The home ground of the theory of mentalese representation is a picture of behavioral control as a process of discursive rational deliberation, the practical syllogism in modern dress: belief plus desire imply executable intention. Whether or not the mentalese sentences which are supposed to constitute imagery enter into these deliberations, it is axiomatic for description theory that they are part of the same representational system, encoded in the same format. If we embrace an account of imagination based on sentential representation we lose our grip on it as a distinctive mental faculty. Imagination would be entirely assimilated to discursive, "consequetive" reason, and thereby would essentially disappear from our understanding of the mind.
We do, of course, sometimes use "imagine" to mean something like "suppose", or "entertain the possibility of" [the truth of some proposition]. But this is not what "the folk" generally take imagination to be (Sutherland, 1971). To reduce it entirely to these bloodless terms is simply to duck the most significant issues that the concept raises. They are perfectly proper usages, but only distantly related to the idea of creative imagination (Thomas, 1997a).
PA theory can accommodate this example, because the perceptual processes that it holds to be involved in imagery are precisely those processes that enable us, in perception, to see things as whatever they are or might be taken to be--to see the duck-rabbit as a duck, for example, or a pikestaff as a pikestaff--by discovering (and perhaps selectively attending to) defining features. But there can be a continuum of cases between the extremes of veridical seeing and "pure" imagery (where the imaginal experience incorporates no aspect of what is before us). In the former, all the perceptual tests ordered by the schema are actually carried out, and all results returned by the perceptual instruments are given their full weight in determining the course of subsequent testing. In "pure" imagery, either the ordered tests are not carried out at all, or else any results that are returned by the instruments are completely ignored. Ryle's example falls between: some of the tests are carried through and responded to appropriately, and some are not. When the child determinedly scrutinizes her doll's unsmiling lips for a smile then relevant positive results, such as might come from a test for, say, pinkness, may be accepted (i.e. allowed to influence the course of further testing), but any relevant negative results (e.g. from a test for curvature) will, as in "pure" imaging, be ignored. She will continue her examination just as if she actually had confirmed the presence of a cheery upward curve.
§4.2.1 Can Picture Theory Explain Seeing As?
On Ryle's reading of picture theory, the imagined smile would exist not on the doll's face but in some non-physical, Cartesian mental space, and, as he says, this is absurd. However, as it stands, this argument fails as a refutation of the real theory. For Kosslyn (and, come to that, Descartes) the "quasi-picture" is not anywhere non-physical, but in the brain, and at just the same locus where similar pictures are supposed to form during normal, veridical seeing. Indeed, the primary function of the visual buffer in this theory is, surely, the registration of images coming in from the eyes. Imagery occurs when this function is hi-jacked by information from some internal source. Since the perceptual process gives rise to perceptions of things as "out there", in the world, picture theory would seem to predict, and certainly to allow, that imagined things will also seem "out there".
But if we consider in more detail how Kosslyn might try to deal with Ryle's example I think we will see that it still raises serious problems. The only approach I can think of that does not involve ad hoc postulation of quite new, otherwise unmotivated, mechanisms is to suggest that although most of the image of the doll on the visual buffer is a percept, formed from information coming directly from the eyes, a small portion of the buffer might be reserved for an image of smiling lips generated out of information from internal sources: a mental image pasted over a part of a perceptual image, as it were.
Because our visual input is in constant flux (not least because our eyes are in constant and irregular motion) we must assume that the perceptually generated contents of the visual buffer will be being constantly updated. It might seem that a "pasted on" image of the type just suggested would be liable to be disrupted by this process12, and the smile would very quickly disappear. However, that conclusion can be avoided if we assume that images are also constantly updated, repeatedly re-written to the appropriate region of the buffer. Kosslyn already holds (1980, 1991) that such regular re-writing of images is necessary if they are not to fade away.
Now, however, suppose the doll were suddenly and unexpectedly snatched away, or the room plunged into darkness. How could the child (or her brain) know to cease writing the smile image until after she had registered that the doll percept had gone? The theory would surely predict that a vision of smiling lips would remain momentarily floating before the child's eyes. I have never experienced any such effect, nor come across anything like it in all my extensive reading of the imagery literature. In similar vein, Hamlet imagined the appearance of a weasel, a whale and a camel in a cloud: if the cloud had suddenly blown away would a faint weasel image have briefly remained in the sky?13 Indeed, is the notion of an image, even a diaphanous one, "pasted over" a percept even a superficially plausible explanation here? Rather than some internally generated picture being superimposed upon the percept, should we not rather say that the cloud and the lips themselves were being actually seen, but in a peculiar manner. Hamlet saw the cloud itself as weasel-like (he detected, or feigned to detect, certain weaselish features in it), the doll's actual lips were seen as smiling.
§4.2.2 Can Seeing As Be Set Aside?
Faced with this argument, pictorialists could retreat, and say that quasi-pictorial theory is only intended to deal with "pure" mental imagery. Examples like the smiling lips and the weasel cloud would therefore be irrelevant to imagery theory. This move is logically unassailable, but, besides flouting Occam's razor, it leaves the smile and the weasel quite unexplained. What might a pictorialist say is going on in such cases?
Kosslyn (1991, 1994) has recently argued that there are in fact two types of imagery. In addition to quasi-pictorial imagery he also now postulates "attention based imagery", which occurs, for example "when you look at a tiled floor (. . .) and "see" a letter by attending to specific tiles" (Kosslyn, 1994)14. Except for the fact that he seems to be thinking of attention entirely in spatial terms (as a matter only of where we look rather than also of what we may be looking for), the sketchy account he gives of the mechanism of "attention based imagery" looks very much like an embryonic version of PA theory.
If this interpretation is correct, this move of Kosslyn's will certainly solve the problem at hand--indeed, picture theory will become immune to any considerations that seem to favor PA theory--but the cost in theoretical parsimony seems considerable. We might legitimately wonder whether, if anything even approaching the amount of energy and ingenuity that Kosslyn and his allies have lavished on the elaboration and defense of quasi-pictorial theory were to be devoted to developing the theory of "attention based imagery", there might be anything left for quasi-pictorial theory to explain. But for the purposes of this article, Kosslyn's move to embrace two theories of imagery need not be rejected. I would merely want to insist that it is PA (or "attention based") imagery, and not pictorial imagery, that is related to the process of seeing as, and thus, as we shall see, to creative imagination.
I want to suggest that it is precisely the mundane sense of imagination as the capacity for seeing as which forms the missing link between imagination as mental imagery and creative imagination in the arts and sciences (or, come to that, in practical affairs) (Thomas, 1997a). A plausible view of the value of works of art--where they have any beyond the mere amusement or sensuous pleasure they afford--finds it in their capacity to induce us to see things as we would not otherwise have seen them: to see fresh (or forgotten) aspects or meanings in an otherwise seemingly banal reality. This is the key, I suggest, to the association made between imagination and the artistic, and the creative in general.
The point is hardly revolutionary. As Passmore (1991 p. 54) says, "The [conceptual] boundaries between 'seeing', 'seeing as' and 'imagining' are . . . notoriously insecure". A recent philosophical reference work informs us that "what seems crucial to the imagination is that it involves . . . perspectives, new ways of seeing things, in a sense of "seeing" that need not be literal" (Hamlyn, 1994). This non-literal sense of "seeing" is, nevertheless, perfectly conventional. To say that one sees Polonius as pompous, or the Dodgers as winners, is not to say that these things are literally visible or even perceptible. The application of "imagine" and its cognates to the arts is the result of a parallel, and equally conventional, metaphorical extension of meaning. To put things crudely, "imagination", in one important sense at least, just is our name for the faculty of seeing as, and its metaphorical extensions cover a similar range. It is not an identical range, of course: We do not normally apply "see as" to cases of pure imagery, where there is no obvious admixture of current reality, neither would we use "imagine" when we want to imply that we are seeing things as they truly are. However, there are significant cases between these extremes where either expression is appropriate. Just as saying the child sees the doll as smiling is equivalent to saying that she imagines a smile on its face, so we regard the facts that Van Gough induces us to see nature as infused with dynamism, or Kafka makes us see ordinary life as absurd and terrifying, as being effects upon our imaginations.
Thus we say that imagination is necessary for the appreciation of art works. Their creation is a closely related matter. For one thing, producing worthwhile art surely depends on a sort of critical connoisseurship towards one's own productions; artists must be able to appreciate whether what they are producing, or thinking of producing, is any good, whether it conveys what is wanted. That is properly said to be a function of their imaginations. Also, of course, the artist's own vision of reality (which may be achieved in the very process of creating the art work that conveys it) is itself a matter of seeing as, and thereby imagination. These brief remarks do not, of course, constitute a fully worked out account of the place of imagination in the arts, but they are surely an improvement on the jejune default notion that creative imagination is simply a capacity for producing an abundance of unusual ideas.
Perhaps even more obviously, scientific creativity also involves seeing as, seeing the world in a particular new way, and, thereby, imagination. Scientific theories induce us to see (in literal or extended sense) aspects of the world as we would not otherwise have seen them. Only when they do this are we able to apply them, to use them to get to grips with reality. Consider Hanson's (1958) famous example of Tycho Brahe seeing the sun as rising over the horizon, whereas Kepler, the Copernican, saw the situation as Earth turning towards the sun. This initial, and, at the time, eccentric revisioning is what made Kepler's further discoveries possible. As in the arts, scientific understanding involves coming to see some aspect of reality in a particular way, and creativity depends on coming to see things in a new way. Where nature had long been seen as the harmonious creation of a benevolent God, Darwin came to see it as an arena of fierce struggle, and eventually persuaded others to see it that way too. Faraday, perhaps almost literally (Koestler, 1964), saw magnets and electrical charges as surrounded by fields of force. Of course, Kepler did not have to be actually looking at the rising sun, or at Mars up in the sky, as he calculated his orbits. Neither did Darwin have to be watching animals, or Faraday have to stare at a magnet as they worked out the details and implications of their theories (no more do painters necessarily have to have their model constantly before them as they work). But imagery is ubiquitous in scientific thinking, innovative and otherwise, even in the most abstract and mathematical fields (Shepard, 1978; Tweney, Doherty, & Mynatt, 1981; Miller, 1984). The creative scientist's distinctive ways of seeing things undoubtedly infuse this imagery, and it is in such circumstances that we speak most naturally and literally of imagination being in play.
Of course, the sophisticated imaginative abilities that are said to go into scientific discovery, or the production of works of art, even minor ones, are a long way from mundane capacities for seeing squiggles as ducks, weasels in clouds, or smiles on dolls' faces. However, if, as I have argued, a picture theory of imagery cannot explain even the latter, mundane, cases, what hope has it of illuminating the former?
In his seminal work on Romanticism, Abrams (1953) argues that a distinctive attitude to perception lies at the philosophical core of the movement. Whereas their 18th century predecessors regarded consciousness as a mirror, passively receiving the images projected into it from the outside world (Rorty, 1980), the Romantics saw it like a lamp, throwing its beams outward to illuminate the world and to constitute experience. Clearly, within these broad metaphorical terms, the information processing approach to perception, from which both pictorial and description theories of imagery derive, is a mirror theory--perceptual data flows inward to a passive, receptive consciousness--whereas PA theory and associated approaches to perception, where conscious experience arises rather from the activity of a mind reaching out into the world, fall under the lamp metaphor. Unsurprisingly, then, PA theory does a much better job of accounting for the imagination as the Romantics conceived it. They bequeathed us a concept of imagination as a faculty of mental imagery that is non-discursive and responsible for the most profound creative insights. Description theory treats all mental processes as discursive through and through, and quasi-pictorial theory does not seem to help us towards a true understanding of creativity.
I have suggested that the understanding of creative imagination might be approached through the investigation of our capacity for seeing as, but picture theory has nothing to offer us here. If it were the correct approach to imagery, it would require supplementation by a quite separate theory of seeing as, of categorical perception, (presumably to be carried out by Kosslyn's "mind's eye function"). There would be no fundamental connection between image formation and seeing as, and, therefore, no particular connection between imagery and imagination!
Perceptual activity theory accounts for imagery and seeing as via a common mechanism, but without assimilating imagery to discursive thought. Thus, if correct, it might allow us to do full scientific justice to the concept of imagination in its culturally most significant form. For historically contingent reasons, PA theory remains theoretically underdeveloped and empirically untried, but its better known rivals still face unresolved empirical problems themselves. It deserves more attention.
Ancestral versions of this article were read at conferences of The British Psychological Society, History and Philosophy Section (Lincoln, U.K., 1989), and The Society for Philosophy and Psychology (College Park, MD, 1990). Its development was shaped by comments and information from many people, particularly (in roughly chronological order): Jonathan Hodge, Daniel Reisberg, David Hilbert, Lee Rowen, Peter Slezak, Robert Cummins, Susan Volman, several anonymous referees, David Salter, John Heil, Glenn English, Muhammad Ali Khalidi, Stephen Kosslyn, William Clancey, and Dennis McClain-Furmanski. Portions of earlier drafts were written whilst I was Mellon Postdoctoral Instructor in Philosophy at the California Institute of Technology, and during a 1993 National Endowment for the Humanities summer seminar at the University of Arizona.
Contact the author, Nigel J.T. Thomas, E-mail: n.j.thomas70@members.leeds.ac.uk.
1. The deliberate embrace of this conflation gives us the metaphysical doctrine of functionalism that has dominated philosophy of mind for decades.
2. There has been an enormous amount of philosophical effort expended on it, in various guises, over the past couple of decades, but there is very little sign of the discussions converging on any generally acceptable solution. For effective critiques of the leading candidate approaches, see Cummins (1989, 1996, 1997). Cummins (1989, 1996) offers his own theories of the content of computational cognitive representation, but he rejects the goal of using these to ground intentionality or consciousness. In effect, he cuts the Gordian knot of representation theory by rejecting CM.
3. Kosslyn is not claiming psychological reality for the specific, LISP-like, format of deep representation illustratively shown in figure 1.
4. Dennett (1991) apparently denies this, but the elaborate analogy he constructs to elucidate quasi-pictorialism (the CADBLIND system) does not show that Kosslyn avoids a homunculus. CADBLIND, after all, is envisaged as feeding information to a conscious human user. CADBLIND is a fair analogy for Kosslyn's quasi-pictorial image, but Dennett overlooks the problematic role of the mind's eye function.
5. PA schemata differ from Rumelhart's, however, and from related notions like Minsky's (1975) "frames", in that they are not so much "places to put information" as "plans for obtaining more of it" (Neisser, 1976).
6. There seems to be no reason to think that the results of tests involving one particular sense mode must always give rise to further testing through the same mode. Thus for PA theory perception (and imagery) is inherently multi-modal. Of course, any particular perceptual (or imagery) episode may more or less strongly emphasize one mode or another.
7. I regard this is not as a nomological but a conceptual fact. "Intentionality" is a philosophical term of art introduced to tease out a central strand from the otherwise complex and confused "folk" notions of mind and consciousness (Thomas, 1998).
8. The explanations in this paragraph may very justly be seen as close to the "tacit knowledge" explanation offered by Pylyshyn (1981), and recently defended by Currie (1995). However, Pylyshyn seems to view subjects' "tacit knowledge" as, at some level, explicitly represented and deliberatively applied to the experimental tasks. This lays him open to being interpreted as suggesting that subjects are effectively faking the relevant response times--they (tacitly) know roughly how long a certain perceptual task would take, and adjust their response times on the equivalent imagery task in accordance with what they think the experimenter is asking of them. Kosslyn (e.g. 1981) has objected to this sort of explanation on the grounds that, in many of the relevant experiments, subjects can be shown to have no explicit awareness of the relevant facts about either perception or experimenter expectations. The PA account avoids such objections. Although PA theory might be construed as appealing to "tacit knowledge" about perception, this knowledge must here be understood not as explicitly represented and used by free decision, but rather as being implicit in the functional architecture of our perceptual systems, and thus as necessarily involved in their deployment. If it were not in play, we would not be using imagery at all.
9. On the other hand, with very familiar shapes, such as letters, that we have had the opportunity to look at in many different ways and in many different circumstances (in different orientations, against different backgrounds etc.), it is to be expected that we will find little difficulty in imagining them in different ways. This may explain results such as those of Finke, Pinker, and Farah (1989), whose subjects had relatively little difficulty in reinterpreting the emergent shapes produced when they were asked to imagine letters and mentally rotate and juxtapose them in various ways (e.g. a B rotated 90º anticlockwise and placed on top of a V, can be reconstrued as a heart). Of course, few shapes that an experimenter is likely to get us to imagine are likely to be utterly unfamiliar (such that we have never seen anything in the least like them) and thus totally resistant to reconstrual.
10. Blake's Milton, written 1804-1808 (Keynes, 1966).
11. Letter 43 (written 1817) in Rollins (1958).
12. Conceivably such images might remain stable despite any changes going on in other regions of the buffer, but if so the argument of the next paragraph applies a fortiori.
13. Conceivably a solution to this problem could be gerrymandered by suitable rigging of the workings of the mind's eye function. It might be suggested that reports of changes in the percept are routinely sent to the image generating mechanisms first, and the passing on of information to consciousness is delayed until any image data left in the buffer has been cleared. However, this is not only sharply ad hoc, but biologically very implausible. Why should the efficiency of perception be compromised for the sake of maintaining an insignificant illusion? It would not matter if the lips, or the weasel, lingered for a moment. But they don't.
14. His reasons for this surprising move seem to be unrelated to the present argument. Rather it stems from the discovery of the so called "two cortical visual systems": anatomically separate neural complexes, one apparently devoted to recognizing things ("what") and the other to determining their location ("where") (Mishkin, Ungerleider, & Macko, 1983). Differential damage to these structures in human patients leads to the appropriate differential impairments in certain experimental imagery tasks (Levine, Warach, J., & Farah, 1985; Farah, Hammond, Levine, & Calvanio, 1988). But surely the proper interpretation is not that there are two distinct types of image (sharply differentiated in mechanism despite the fact that they are not sharply differentiated in experience), but that normal vision (and normal imagery) involves these two complexes (and, surely, others) working together, such that different aspects of vision (or imagery) are impaired by different lesions. Ordinary visual experience integrates "what" and "where" information. The fact that it depends on more than one neural structure is not a reason for saying that it is "really" more than one experience.

References: §2
 §2
 §3
 §4
 §2
 §2
 §2

§4

§4