Cognitive neuroscientists have discovered through various experiments that our body representation is surprisingly flexible. Multisensory body illusions work well in immersive virtual reality, and recent findings suggest that they offer both a powerful tool for neuroscience and a new path for future exploration. I mmersive virtual reality (IVR) can give participants the illusion of having a different body: when wearing a head-mounted display with head-tracked glasses and looking down toward their feet, they see a virtual body spatially coincident with their own. Through real-time motion capture, participants can move their real body and see the virtual version move correspondingly; they can also see this movement in a virtual mirror reflection. 1 Because virtual reality is entirely programmed, the form or type of virtual body can be quite different from the participant's actual body, which can impact perception, attitudes, and behavior. A dramatic example of this is when adults inhabiting a virtual child's body overestimate the size of objects and demonstrate implicit attitude and behavioral changes that seem more child-like, but in an adult body the same size as that of a child, they do not exhibit such changes. 2 Here we review the emerging field of body representation and its implications for a new, powerful virtual reality paradigm. A standard IVR system has two critical components. The first is a display device that delivers high-resolution stereovision and sound to participants by substituting their real-world sensory input with computer-generated input, including haptic feedback. The second is the use of realtime head tracking with six degrees of freedom. The result is that the world is displayed to the participant based on head-gaze direction. IVR has two major forms of delivery. The first is a headmounted display (HMD) that offers wide field-of-view stereoscopic vision; Figure 1a shows an example with a head-tracking device mounted on the HMD. The second is a projection-based system-a computer-assisted virtual environment (Cave) whose walls are back-projection stereo display screens. As Figure 1b shows, the participant wears shutter glasses so that each eye only receives the images that correspond to the view for that eye. The type of research discussed in this article also includes a third critical system-a full body motion capture suit that tracks the participant's body movements and drives in real time the virtual body that substitutes for the real body. Substantial evidence gathered over many years suggests that when people enter a virtual environment delivered by an IVR system, they tend to act as if they are in a real place engaging in real events. This type  25 of response is generally referred to as presence. 3 Originally, presence was simply thought of as the sensation of "being there" in the virtual environment, but more recent interpretations focus on the extent to which people respond realistically, with a fundamental distinction between the illusion of being in a place (place illusion, PI), and experiencing events as if they were real (plausibility illusion, Psi). 4 PI relies heavily on head (ideally eye) tracking because it enables head-based sensorimotor contingencies-in other words, perception of the environment through use of the body in a natural way: bending down, looking around, looking past, listening by turning the head toward the source, and so on. Psi is the extent to which the participant perceives events in the virtual environment as actually happening. It seems to be a function of the extent to which events in the environment refer specifically to the participant, the extent to which events respond to that person's actions (for example, the participant smiles at a virtual human that smiles back), and the overall credibility of the environment compared to expectations. An example of PI and Psi coming together in terms of participants' behavioral and physiological responses was the virtual reprise of the famous 1960s Stanley Milgram obedience experiments where volunteers were required to give electric shocks to a stranger at the behest of an authority figure. The IVR version generated PI because the scenario was displayed in a Cave with head tracking and Psi because the virtual human character responded to and argued with the participant. The responses of participants were similar to those reported in the original experiments. 5 In the late 1980s, Jaron Lanier experimented with the idea of using virtual reality to transform the self by endowing participants with virtual bodies. Writing about these experiences, he observed that "people could quickly learn to inhabit strange and different bodies and still interact with the virtual world. I became curious about how weird the body could get before the mind would become disoriented. I played around with elongated limb segments and strange limb placements. The most curious experiment involved a virtual lobster." 6 What he called "homuncular flexibility" was that people can learn to inhabit and naturally control virtual bodies that are quite different from their own. These observations about the brain's flexibility in attributing surrogate bodies or body parts to the self were not published and remained unnoticed and unexplored in the scientific literature. However, research over the past two decades in cognitive neuroscience has confirmed that the brain is remarkably plastic with respect to body representation. The rubber hand illusion (RHI) 7 shows that by tapping and stroking a rubber hand placed in an anatomically plausible position on a table in front of a person while synchronously tapping and stroking that person's corresponding but hidden real hand quickly leads to a strong illusion on the participant's part that the rubber hand is actually his or hers. This illusion is both subjective and can be measured objectively through proprioceptive drift: when asked to blindly point to his or her hand, the participant will point closer to the rubber version than the real hand after the stimulation compared to before. Similarly, if the rubber hand is seen to be threatened, the participant exhibits strong physiological responses to that perceived threat. 8, 9 The illusion of ownership and objective indicators occur to a significantly lesser extent when the visual and tactile stimuli are not both temporally and spatially synchronous. This illusion has been shown to work well in IVR too, where, instead of a rubber hand, the participant sees a virtual arm in stereo 3D coming out of his or her real shoulder. 10 The operator achieves synchronous visual-tactile stimulation by using a tracked wand to tap and stroke the participant's real hidden hand, while a virtual ball controlled by the wand gives the same touch on the virtual hand. Valeria Petkova and H. Henrik Ehrsson demonstrated that the same principles of multisensory stimulation can result in the illusion of ownership over a whole manikin body seen from the first-person perspective (1PP), 11 which also works well in IVR. 1 26 COMPUTER Imagine looking at your virtual body both directly by looking down it and in a virtual mirror. This is already a very powerful cue to the brain to feel that this virtual body is your own because throughout your life, whenever you look down toward your feet-or in a mirror-of course, you see your own body. This is why 1PP (the eyes of the virtual body coincident with the person's real eyes, and the virtual and real body being spatially coincident) is such a powerful factor in the body ownership illusion. 1, 12 In addition, realtime motion capture of the participant's movements can be mapped to the virtual body's movements in real time. We use the term embodiment for the setup in which a virtual body is spatially coincident with your real body and you see through the eyes of that virtual body, with various types of synchronous multisensory correlation. 13 But once you have this perceptual illusion that an alternate virtual body is yours, what are the implications for behavior, attitudes, and cognition, both in virtual reality and beyond? Researchers have investigated this issue under the paradigm known as the Proteus effect, 14 arguing that a person's digital self-representation in online and virtual environments could influence his or her attitudes and behavior. Essentially, the Proteus effect states that the personality associated with the digital representation influences participants' actual real-time behavior. The idea behind Proteus is mutability, but we want a term that also suggests that form carries information and meaning. Therefore, we use body semantics to express the finding that when participants have the illusion of ownership and agency over a virtual or robotic body via multisensory correlations, this has behavioral, attitudinal, and probably also cognitive correlates for the embodied person. Three recent examples illustrate this. Does the type of virtual body influence the participant's motor actions? We considered this question in the context of drumming performance. 15 Thirty-six people participated in an experiment in which they had to play a Djembe hand drum in an IVR environment. The participants were divided into two different groups: the casual dark-skinned (CD) group saw a virtual body dressed casually with darker skin, and the formal light-skinned (FL) group viewed a lighter-skinned body dressed formally. All were accompanied by the animated virtual drummer depicted in Figure 2a (www. youtube.com/watch?v=ydzSgLim5Y4). Throughout the experiment, we recorded for each participant 36 data points from motion capture, representing upper-body movement. We recorded this first in a baseline condition, with the participants represented solely by nontextured white hands (that is, without a body) and then during the actual embodiment period. To estimate the amount of movement, we used principle components analysis to compute the number of dimensions needed to capture 95 percent of the total variance in the complete data. We found no significant differences in the subjective level of body ownership between the two groups, as measured through a questionnaire whose scores indicated a strong ownership illusion. The results showed, however, that those in the CD group required on average more than one dimension extra during the embodiment period compared to the baseline period to attain 95 percent of the variance of the motion-capture data (p = 0.0002), whereas there was no difference from the baseline for those in the FL condition (p = 0.90). Overall, those in the CD group exhibited significantly greater upper-body movement while drumming than those in the FL group, which positively correlated with the extent of subjective body ownership. In other words, the type of body-including its clothes, hairstyle, and so on-influenced motor performance. We used the same virtual reality setup to study racial bias, with the aim of examining whether virtual embodiment could shift deep-seated implicit attitudes. 16 In this experiment, 60 females experienced one of four conditions: the embodied dark (ED) condition, depicted in Figure 2b , had a dark-skinned female avatar; the embodied light (EL) condition had a light-skinned avatar; the embodied alien (EA) condition had a purple-skinned body; and the nonembodied dark (ND) condition had no direct body representation but a reflection of a dark-skinned body in a virtual mirror with movements independent of the participants' own body movements. Some days before the virtual reality experience, our participants attended the laboratory and completed a racial bias implicit association test (IAT), 17 which measures bias by response time when categorizing relationships between concepts. For example, a bias against dark-skinned people is indicated when reaction times are faster in associating light-skinned faces with positive words and dark-skinned faces with negative words than vice versa. In the IVR experience, participants were embodied for approximately 12 minutes in one of the four conditions. For 7 of those minutes, 12 virtual characters (half dark-skinned, half light-skinned) simply looked at the participant while walking past (www.youtube.com/watch?v=HliN3iOX090). The participants subsequently completed the IAT test again, and answered a questionnaire. Although an explicit racial bias test showed a low degree of racial bias on average, the initial IAT test found implicit racial bias. However, the effect after the experiment was different among the four groups. Reduction in the mean level of racial bias as a result of the exposure was greater in the ED group compared with the EL group (p = 0.03). The difference between the ED group and the others was 27 striking when we took into account a covariate signifying presence. (The factor that represented presence was the degree of the participants' reported nervousness as the avatars walked past, breaking into their personal space.) We carried out an analysis of covariance of the change in IAT with respect to skin color (ED, EL, EA, ND) and nervousness as the covariate. We found a negative slope in the relationship between the change in IAT and nervousness, indicating that racial bias reduced with increasing nervousness (p = 0.002). The ED condition resulted in a significantly lower intercept than EL (p = 0.009), but there were no other significant differences. The level of subjective body ownership as measured by questionnaire responses did not differ among the ED, EL, and EA groups but was significantly lower in the ND group. The overall analysis of variance showed a difference between the mean levels of ownership with significance level of p = 0.004. All pairwise comparisons at an overall  COMPUTER significance level of 5 percent showed that the mean level of ND was less than each of the others, but that there were no differences between the embodiment conditions. Moreover, the mean levels of body ownership in the embodiment conditions were high (approximately 4 on a 1-to-5 Likert scale, with 1 meaning the least and 5 the greatest). In other words, the difference in body typesspecifically, skin color-was responsible for reducing implicit racial bias in the ED group. We conducted another pair of experiments to determine whether embodiment in a young child's body would influence perception of object sizes and attitudes about the self. 2 Initially, we embodied 30 adults in a body representing a child of approximately four years or in an adult body shrunk down to the same size as the child. As Figure 2c and 2d show, the only difference between the two conditions was the virtual body's shape. Whole body motion capture ensured that virtual body movements matched the participants' own (www.youtube.com/watch?v=8Oy83OVgbSM). The adult and child conditions showed no differences with respect to the subjective illusion of body ownership, which was high. We asked participants before and after the experience to indicate the sizes of several objects by the distance between their hands. As expected, both groups overestimated object sizes, but compared to the baseline readings, the child condition estimates were almost double those of the adult condition. For example, the differences in mean estimates for the three object sizes tested all had significance levels of p < 0.006. Moreover, our IAT results showed the child condition resulting in significantly greater implicit association of the self with child-like categories than the adult condition (p = 0.0001). To be sure that the results reflected body ownership, we ran a second experiment with 16 additional participants. The conditions were identical, except that the virtual body moved independently of the person's real body movements-the body was still viewed from 1PP and in the mirror as before. Both groups overestimated object sizes to about the same extent as those in the adult condition in the first experiment, but there were no differences between the adult and child conditions and no difference in IAT scores. In other words, the introduction of body movements that did not match real body movements extinguished body ownership and agency, and all of the differences between the child and adult conditions. The field of body representation raises several very interesting issues for virtual reality. The first is that IVR is excellent as a practical tool for cognitive neuroscientists to study how the brain represents the body. It offers flexibility for manipulating body representations that far surpasses typical tools of the trade such as rubber arms, mirrors, and LEDs. Everything is fully programmable, and the cost of setting up a virtual reality laboratory continues to decline dramatically. Second, IVR is helping uncover the brain's striking plasticity in terms of body representation. Lanier had this insight decades ago, but it was never followed up scientifically. Through a completely different paradigm, cognitive neuroscience has come to the same conclusions. Third, and perhaps most interesting of all, is the idea of body semantics-that the brain is apparently able to drive attitudes and behavior of individuals according to their level of body ownership with respect to the type virtual body in which they are embodied. Although the three studies that we presented here are quite different, they are closely linked by the idea that the form or type of body over which participants have the illusion of ownership can influence, for example, their motor behavior (the first experiment), implicit racial bias (the second experiment), or perception of object sizes and self-categorization as child-or adult-like (the third experiment). These are all examples of what we have termed body semantics-the body form influencing various aspects of perception, attitudes, and behavior. It is worth emphasizing both the similarities and differences between the Proteus effect and body semantics. The similarity is that both claim (and have demonstrated) that the type of body can influence attitudes and behaviors. However, this conclusion is arrived at through quite different paradigms. The Proteus effect is from a social psychological perspective and based on the idea that people behave according to how others would expect someone with that type of body to behave. It is based on "self-perception theory," where participants "conform to the behavior that they believe others would expect them to have" and that "self-perception theory is not predicated on the actual presence of other people but simply that a person evaluates him or herself from a third-person perspective (namely, an imagined third party)." 14 In contrast, the idea of body semantics is that when the brain generates an illusion of body ownership or agency over a virtual body, it also generates attitudes and behaviors concomitant with that type of body, independently of any other factors such as social expectation. The interesting yet unresolved question is how the brain derives those attitudes and behaviors. In the case of being a virtual child, it could simply be that we have all been children, so the brain relies on autobiographical memory to determine perception and other mental processing. However, none of the participants in our drumming experiment had ever vaguely resembled Jimi Hendrix. It is possible that the brain here is relying on statistical information about how people with such bodies that they have seen in the past behaved in that setting. Or, perhaps independently of 29 statistical information gathered from living in a particular society and culture, brains have intrinsic mechanisms that encapsulate extremely tight associations between the type of body and the behaviors of which it is capable.
The latter is the extent to which the events within the virtual environment are perceived as really happening (ibid) and is said to be influenced by three factors: "(a) the extent to which events in the environment refer specifically to the participant (b) the extent to which there are events that respond to the actions of the person (e.g., the participant smiles at a virtual human that smiles back) and (c) the overall credibility of the environment in comparison to expectations" REF .
Transcending the Self in Immersive Virtual Reality
Computer
Computer
Computer Science