Effect of a Socratic Animated Agent on Student Performance in a Computer-Simulated Disassembly Process

The purpose of this study was to examine the effect of a Socratic animated agent on user performance in a computer-based CD player disassembly simulation. Vocational Education students were randomly assigned to one of two versions of the simulation. One version offered text-based feedback in the form of statements; the other one featured an animated agent providing hints and clues. Results showed that performance (measured by number of wrong tool choices) did not significantly differ between groups. Similarly, both groups stopped making wrong tool choices at approximately the same point in the disassembly process. The short exposure to the treatment may account for these results. Suggested modifications that could be incorporated in future studies are proposed in light of this factor. ********** Computer-based instruction (CBI) has significantly evolved in recent years, moving from an initial, basic drill-and-practice approach, to complex applications that incorporate simulations and animations. One such application is animated agents, which have been claimed to allow for a more natural, human-like interactivity between user and computer (Baylor, 2002; Johnson, Rickel, & Lester, 2000). Animated agents are characters, or conversational interfaces, that mediate between the user and the computer program, to offer different forms of assistance, guidance, and feedback in the use of that program. These agents range from animated shapes (e.g., the Clippit desktop assistant in Microsoft Windows 95 and beyond) to human-like entities, including cartoon-like shapes and characters, talking animals, and a variety of others. A number of studies have been carried out on animated agents as used in CBI, to learn about their effect on users. Dehn and Van Mulken (2000), in their review of the most salient experimental studies conducted on the use of animated agents, compared existing literature by classifying studies according to the kind of agents used (e.g., physical appearance, voice, etc.), the variables examined (both on the cognitive and the affective domains), and the results obtained. This comparison of current literature shows that agents can be engaging (Koda & Maes, 1996), but also create an uncomfortable feeling (Sproull, Subramani, Kiesler, Walker, & Waters, 1996); they can be helpful (Van Mulken, Andre, & Muller, 1998), but to some extent distracting (Koda & Maes); and users can interact with them (Takeuchi & Naito, 1995), or they can ignore agents completely (Lester, Stone, Converse, Kahler, & Barlow, 1997). Dehn and Van Mulken concluded that, given the contradictory results shown by current literature, it appears that the effectiveness of animated agents is directly affected by the combined elements of physical features of the agent, target audience, domain in which agent is used, and type of task to perform. Among the studies described by current literature, only a few of them focus on how the pedagogic style of the agent may affect user performance and attitude. In one of those studies, Baylor (2002) compared two agents that advised students through the development of an instructional plan. One of the agents represented a constructivist approach (emphasized learner-centeredness, process over product, etc.), and the other displayed an instructivist style (followed a traditional, systematic method). Students who worked with the constructivist agent reported a significant change in their perspectives about instructional planning, and reflected an underlying constructivist pedagogy in their plans. Students who worked with the instructivist agent reported a lower disposition towards instructional planning. Other studies have examined different agent personal characteristics (which affect their perceived pedagogical style). Hietala and Niemirepo (1998), for example, compared four agents with different genders and different skill levels. Users tended to prefer one or the other depending on their own personality type (introvert or extrovert), as well as the difficulty of the task. …