Motor Affordances in Mental Rotation: When minds reflect the world and when they go beyond

Motor Affordances in Mental Rotation: When minds reflect the world and when they go beyond Stephen J. Flusberg (sflus@stanford.edu) Gavin W. Jenkins (gjenkins@stanford.edu) Lera Boroditsky (lera@stanford.edu) Stanford University, Department of Psychology Jordan Hall, 450 Serra Mall, Building 420, Stanford, CA 94305 USA Abstract A vivid imagination is one of the defining features of human mental life. A large body of research has shown that mental imagery is supported by some of the same cognitive systems that underlie perception and action (Decety, 1996; Kosslyn, Thompson, & Ganis, 2006). At the same time, our imaginations can go far beyond what is currently available in our physical environment; we can conjure up new worlds in art and literature, design new technologies, and conduct thought experiments that lead to major scientific discoveries. In this paper we investigate the extent to which the physical constraints and affordances of objects in the real world impinge on our ability to represent and manipulate those objects in our imagination. In particular we ask: are objects that are more difficult to physically manipulate also more difficult to mentally manipulate? Participants interacted with two wooden objects modeled after the figures from Shepard and Metzler’s (1971) classic mental rotation study. One object was easy to physically rotate while the other was difficult to rotate. They then completed a mirror-image mental rotation task consisting of images of the manipulated objects. Participants were slower to solve the mental rotation task for trials consisting of images of the hard-to-rotate object, but only when they used a motor strategy in the task. Keywords: Mental representation; Mental imagery; mental rotation; motor affordances Background The capacity to vividly imagine scenes, events, and actions is one of the hallmarks of human mental life, and researchers have been investigating the nature of mental imagery for the better part of four decades (Kosslyn, Ganis, & Thompson, 2001; Shepard & Metzler, 1971). A great deal of research has suggested that mental imagery is supported by some of the same cognitive systems that underlie perception and action; brain areas known to process visual information are also engaged during imagined visual activity (Kosslyn, Ganis, & Thompson, 2001; Kosslyn, Thompson, & Ganis, 2006; O’Craven & Kanwisher, 2000; Winawer, Huk, & Boroditsky, under review) and brain areas responsible for planning and executing motor movements are also activated during imagined motor movements (e.g. Decety, 1996; Jeannerod & Frak, 1999; Porro et al., 1996). It seems that the imagination is intimately tied to the processes that mediate our perceptual and motor interactions in the world. At the same time, however, our imaginations often appear to go far beyond the information that is available in our physical environment: we can conjure up new worlds in art and literature, design new technologies, and even conduct thought experiments that lead to major scientific advances. In fact, what often seems to separate our imagination from the everyday world is that mental imagery appears free from the constraints imposed by the physical environment. Indeed, people often imagine actions or events that they have never seen or never could experience themselves. We can imagine ourselves riding on a beam of light, leaping tall buildings in a single bound, or making a hook shot from half court. On the one hand, then, mental imagery is supported by perception and action systems in the brain, but on the other hand our imaginations seem to go above and beyond our possible perceptual and motor worlds. This begs the question, to what extent do the constraints and affordances of objects in the real world impinge on our ability to represent and manipulate those objects in our imagination? Consider that you are redecorating your office and you are imagining what the room would look like if you moved a few items around. When you imagine moving the couch to the far wall, does the bulk of the couch, which constrains your physical motor interactions, automatically affect your mental imagery for the couch? Or can you just as easily imagine moving the heavy couch as the small reading lamp? Here we ask this very question: are objects that are more difficult to physically manipulate also more difficult to mentally manipulate? We address this issue by utilizing a novel adaptation of the classic Shepard and Metzler (1971) mental rotation task. In their original study, participants were presented with images of two three-dimensional block figures and had to indicate whether the objects were identical or were mirror images of one another. Interestingly, reaction times were linearly proportional to the angular disparity between the two objects. These findings have since been replicated many times, though several researchers have argued that there are actually several possible mental strategies for solving this task (e.g. Kosslyn et al., 2001b). For example, it is possible to use a motor strategy and imagine manually rotating the objects (Kosslyn et al., 2001b; Wexler, Kosslyn, & Berthoz, 1998) or to use a visual strategy and imagine the objects rotating on their own (Kosslyn et al., 2001b). Kosslyn and colleagues (2001b) used PET imaging to demonstrate that participants were able to consciously adopt one of these strategies and flexibly switch to a different