Can images be rotated and inspected? A test of the pictorial medium theory

Since the “equivalence” of imagery and perception has been one of the central tenets of the pictorial theory, the negative results of Chambers and Reisberg (1985) on an image reinterpretation task may be seen as posing a fundamental challenge for the pictorial account. Finke, Pinker and Farah’s (1989) claimed refutation of these negative results may be questioned on a number of methodological grounds. In addition to examining these issues, we report results of an experiment which tests what is seemingly another direct prediction of pictorial theories. Our investigation employs newly devised imagery tasks whose success depends on being able to “rotate”, “inspect” and reinterpret images. Our negative results add further weight to a tacit knowledge account of images as intrinsically interpreted, abstract symbols. New Focus for Imagery Debate After twenty years of the recent controversy, the ‘imagery debate’ is widely regarded as having become stalled, with no new ideas on how to break the impasse by bringing some decisive evidence to bear. It is in this regard that the possibility of reinterpreting visual patterns in mental imagery has recently emerged as new focus for the longstanding controversy. The question of whether, and under what conditions, novel information may be discovered from images may shed new light on the debate, since it provides a new means for testing the claimed parallel between imagery and perception and the properties of the conjectured pictorial medium. “Equivalence” of Imagery and Perception Specifically, the possibility of reinterpreting an image follows as a direct implication of the pictorial theory which posits an “equivalence” between imagery and perception. On the pictorial view, a mental image is conceived to be a “surrogate percept, allowing people to detect some pattern or property in a remembered scene that they did not encode explicitly when they saw the scene initially” (Pinker and Finke 1980, p. 246). It is in this sense that the uninterpreted images in a spatial medium are themselves “functionally equivalent to physical objects or events” (Finke, 1980 p. 113), and cause the same mechanisms to be activated as in actual visual perception itself (ibid, p. 130). Kosslyn (1987, p. 149) explains, one purpose of imagery involves “recognition processes” to discover information which is not stored explicitly in memory and thus we “look” at our images in a way which is analogous to the way we look at external objects in order to inspect them. Divergent Predictions By assimilating imagery so closely with vision, indeed by claiming their “equivalence”, pictorialism is committed to predicting closely similar “perceptual” phenomena in imagery to those found in perception itself. It is this deep commitment to the perceptual character of imagery which is the source of its vulnerability to such asymmetries as those of Chambers and Reisberg (1985) and our own results. Thus, Kosslyn suggests that “image interpretation is at the heart of the role of imagery in cognition (if one cannot inspect imaged patterns, they are useless)” (1988, p. 249). In line with the remarks we have seen by Pinker and Finke, Kosslyn too suggests, “The recognition mechanisms [of vision] can be used in imagery as a way of accessing stored information” (1988, p. 264) and his model entails that “images depict visual information, and that this information is interpreted by some of the same sorts of classificatory procedures used in classifying sensory input during vision” (1980, p. 32). Kosslyn explains further that “the purposes of imagery, in large part, parallel those of vision” and “one may ‘recognize’ parts and properties of imaged objects that had not been previously considered” through the “use of recognition processes” (1987, p. 149). Thus, the significance of reinterpreting images as a crucial test of the pictorial theory is evident in Kosslyn’s explicit predictions: The image is formed by forcing a change of state in the visual buffer in the attended region, which can then be reprocessed as if it were perceptual input (e.g., the shape could be recategorized), thereby accomplishing the purposes of imagery that parallel those of perception. (1987, p. 155; emphasis added) By contrast with these direct implications of the quasi-perceptual, pictorial-medium theory, the ‘tacit knowledge’ account would predict that the reinterpretation of images is difficult because it construes the mental representations as highly abstract entities which are the output of ‘higher’ cognitive processes; these are encodings of conceptualizations or beliefs and, in this sense, already meaningful and not requiring interpretation, nor susceptible of easy re-interpretation (Pylyshyn 1973, 1978). New Paradigm: Non-Chronometric Tests At least part of the reason for the persistence of the imagery debate has been the fact that the dispute has involved alternative explanations of the same chronometric evidence. Accordingly, experiments are needed on which the contending accounts deliver different predictions. In particular, more directly illuminating would be data concerning a task whose success is directly dependent upon the operation of the postulated process. Julesz’s (1971) random-dot stereograms served in this way as an unambiguous and unfakeable test for stereopsis. Divergent predictions of the contending theories on an ‘all-or-none’ task is a significant feature of the experiment which we report here. Our own evidence concerns a perceptual organisation task which provides unequivocal criteria of the successful rotation, inspection and re-interpretation of images using "recognition processes" and "shape classification" procedures. Despite the demonstrated ease of our task under perceptual conditions, naive subjects have been uniformly unable to succeed in the task under imagery conditions as would be predicted on the pictorial theory. Chambers & Reisberg Negative Results This question of reinterpreting visual images had been brought into sharp relief with the work of Chambers and Reisberg (1985) who found that subjects were uniformly unable to reverse their mental images of the familiar ambiguous figures such as the duck/rabbit and Necker cube. Chambers and Reisberg see their results as supporting the “philosophical” arguments for taking imagery to be more conceptual and cognitive, in the sense that they are intrinsically interpreted symbols which do not need, and do not easily permit, further interpretation. Since the close parallel, indeed “equivalence” of imagery with the mechanisms of perception has been one of the central tenets of the pictorialist theory (Podgorny and Shepard 1978, Finke 1980, Farah 1988), the results of Chambers and Reisberg are surprising and may be seen as posing a fundamental challenge for the pictorial theory. Of course, the negative results of Chambers and Reisberg provide experimental support for the specific claims made over a decade ago by Pylyshyn (1973, 1978) in his critiques of pictorial theories. Despite these earlier skeptical claims, however, in the intervening period there has been experimental evidence of just such abilities of people to detect novel properties in imagined scenes. Pinker and Finke (1980) report subjects’ ability to “see” novel properties which “can be ‘read off’ the display” and should emerge from images after mental rotation. Finke and Slayton (1988) have extended this work, providing further evidence “that people are capable of making unexpected discoveries in imagery” and that novel patterns can “emerge” from within imaged patterns. Response by Finke, Pinker and Farah. Most recently, Finke, Pinker and Farah (1989) have sought to reinforce these claims with new experiments which also purport to show that subjects can inspect and reinterpret their images by “applying shape classification procedures to the information in imagery” (1989, p. 51). This latter work is of particular interest because it has been specifically designed to counter the skeptical conclusions warranted by the negative results of Chambers and Reisberg and sets the scene for our own investigation. Thus, it is in the light of this clash of experimental results and theoretical claims that our own experiment is to be understood: Our new experiment avoids the specific objections to Chambers and Reisberg results while also addressing potential problems of their own procedures. In this way, by falsifying central entailments of the pictorial medium theory in a different manner, our results can be seen as illuminating the precise conditions under which such seemingly contradictory results can be obtained. Methodology of Falsification Discounting “philosophical” considerations, Finke et al. claim to "refute" Chambers and Reisberg by showing them to have been mistaken on the “strictly empirical” claim that mental images are not subject to reconstrual. Construing the question at stake in this way as a “strictly empirical” one, Finke et al. assert “our experiments show that the answer to it is that such reinterpretation is possible” (1989, p. 74). But this notion of refutation uncharitably construes the Chambers and Reisberg case in its most implausible and most uninteresting form. The logic of their claim is not merely that images cannot be interpreted in some unqualified and completely general sense, and consequently refuted by finding a contrary case. The Chambers and Reisberg investigation derives its significance from showing the inability of image reconstrual precisely under conditions in which we would have expected it according to the quasi-perceptual account. By demonstrating the possibility or reinterpretation under entirely different conditions, Finke et al. have merely avoided the issue. It is not the “strictly empirical question” of whether or not images can be reconstrued simpliciter. To be sure, this is an empirical question in an unproblematic sense, but it is not the question raised by the investigation of Cha