The Use of Immersive Virtual Reality in the Learning Sciences: Digital Transformations of Teachers, Students, and Social Context

This paper illustrates the utility of using virtual environments to transform social interaction via behavior and context, with the goal of improving learning in digital environments. We first describe the technology and theories behind virtual environments, and then report data from four empirical studies. In Experiment 1 we demonstrated that teachers with augmented social perception (i.e., receiving visual warnings alerting them to students not receiving enough teacher eye gaze) were able to spread their attention more equally among students than teachers without augmented perception. In Experiments 2 and 3, we demonstrated that by breaking the rules of spatial proximity that exist in physical space, students can learn more by being in the center of the teacher’s field of view (compared to the periphery) and by being closer to the teacher (compared to farther away). In Experiment 4, we demonstrated that inserting virtual co-learners who were either model students or distracting students changed the learning abilities of experimental subjects who conformed to the virtual co-learners. Results suggest that virtual environments will have a unique ability to alter the social dynamics of learning environments via transformed social interaction. Immersive Virtual Environments in Learning 3 Introduction Many researchers have investigated the viability of virtual environments (VEs), digital simulations that involve representations of teachers, students and/or content, for learning applications. In this article, we describe how VEs enable transformed social interaction (TSI), the ability of teachers and students to use digital technology to strategically alter their online representations and contexts in order to improve learning. We present evidence from a series of empirical studies that demonstrate how breaking the social physics of traditional learning environments can increase learning in VEs. Of course immersive virtual reality currently is not yet an easily acquired technology in classroom settings. Nevertheless, VEs are becoming more common place, and it is important to understand how this digital technology will aid the basic learning process. In this Introduction, we first provide a discussion of the taxonomies of VEs in general, and discuss previous implementations of learning systems in VEs. We next provide an assimilation of the literature on learning in VEs, focusing on the unique affordances provided by VEs not possible in face-to-face settings, including explicating our theory of TSI. Finally, we provide an overview of the current experiments. Definitions and Taxonomies of Virtual Environments VEs are distinct from other types of multimedia learning environments (e.g., Mayer, 2001). In this paper, we define VEs as “synthetic sensory information that leads to perceptions of environments and their contents as if they were not synthetic” (Blascovich, et al., 2002, pg. 105). Typically, digital computers are used to generate these images and to enable real-time interaction between users and VEs. In principle, people can interact with a VE by using any perceptual channel, including visual (e.g., by wearing Immersive Virtual Environments in Learning 4 a head-mounted display with digital displays that project VEs), auditory (e.g., by wearing earphones that help localize sound in VEs), haptic (e.g., by wearing gloves that use mechanical feedback or air blasts systems that simulate contact with objects VEs), or olfactory (e.g., by wearing a nosepiece or collars that release different smells when a person approaches different objects in VEs). An immersive virtual environment (IVE) is one that perceptually surrounds the user, increasing his or her sense of presence or actually being within it. Consider a child’s video game; playing that game using a joystick and a television set is a VE. On the other hand, if the child were to have special equipment that allowed her to take on the actual point of view of the main character of the video game, that is, to control that character’s movements with her own movements such that the child is actually inside the video game, then she is in an IVE. In other words, in an IVE, the sensory information of the VE is more psychologically prominent and engaging than the sensory information of the outside physical world. For this to occur, IVEs typically include two characteristic systems. First, the users are unobtrusively tracked physically as they interact with the IVE. User actions such as head orientation and body position (e.g., the direction of their gaze) are automatically and continually recorded and the IVE, in turn, is updated to reflect the changes resulting from these actions. In this way, as a person in the IVE moves, the tracking technology senses this movement and renders the virtual scene to match the user’s position and orientation. Second, sensory information from the physical world is kept to a minimum. For example, in an IVE that relies on visual images, the user wears a head-mounted display (HMD) or sits in a dedicated projection room. By doing so, the Immersive Virtual Environments in Learning 5 user cannot see the objects from the physical world, and consequently it is easier for them to become enveloped by the synthetic information. There are two important features of IVEs that will continually surface in later discussions. The first is that IVEs necessarily track a user’s movements, including body position, head direction, as well as facial expressions and gestures, thereby providing a wealth of information about where in the IVE the user is focusing his or her attention, what he observes from that specific vantage point, and his reactions to the environment. The second is that the designer of an IVE has a tremendous control over the user’s experience, and can alter the appearance and design of the virtual world to fit experimental goals, providing a wealth of real-time adjustments to specific user actions. Of course there are limitations to IVEs given current technology. The past few years have demonstrated a sharp acceleration of the realism of VEs and IVEs. However, the technology still has quite a long way to go before the photographic realism and behavioral realism (i.e., gestures, intonations, facial expressions) of avatars digital representations of one another in IVEs approaches the realism of actual people. Moreover, while technology for visual and auditory IVEs steadily develops, systems for the other senses (i.e., haptic) are not progressing as quickly. Consequently, it may be some years before the technology rivals a “real world” experience. And finally, some users of IVEs experience simulator sickness, a feeling of discomfort resulting from the optics of particular technological configurations. However, a recent longitudinal study has demonstrated that simulator sickness is extremely rare today, given the speed of current tracking and graphics systems, and also that the effects for a given user tends to diminish over time (Bailenson & Yee, 2006). Immersive Virtual Environments in Learning 6 Collaborative Virtual Environments (CVEs) involve more than a single user. CVE users interact via avatars. For example, while in a CVE, as Person A communicates verbally and nonverbally in one location, the CVE technology can nearly instantaneously track his or her movements, gestures, expressions and sounds. Person B, in another location, sees and hears Person A’s avatar exhibiting these behaviors in his own version of the CVE when it is networked to Person A’s CVE. Person B’s CVE system then sends all of the tracking information relevant to his own communications over the network to Person A’s system, which then renders all of those movements via Person B’s avatar, which Person A can see and hear. This bidirectional process—tracking the users’ actions, sending those actions over the network, and rendering those actions simultaneously for each user—occurs at an extremely high frequency (e.g., 60 Hz). Traditionally, researchers have distinguished embodied agents, which are models driven by computer algorithms, from avatars, which are models driven by humans in realtime. Most research examining learning in virtual environments has utilized embodied agents (as opposed to avatars see Bailenson & Blascovich, 2004, for a discussion). One reason for this disparity is that readily available commercial technology allowing individuals to create digital avatars that can look like and behave in real-time like the person it represents has emerged only recently. Previously, producing real-time avatars that captured the user’s voice, visual features and subtle movements was quite difficult. Consequently, understanding the implications of the visual and behavioral veridicality of an avatar on the quality of interaction is an important question that has received very little