Posture as a Predictor of Learner's Affective Engagement

Posture as a Predictor of Learner’s Affective Engagement Sidney D’Mello 1 (sdmello@memphis.edu) Patrick Chipman 2 (pchipman@memphis.edu) Art Graesser 2 (a-graesser@memphis.edu) Department of Computer Science, 2 Department of Psychology University of Memphis, Memphis TN, 38152, USA. Abstract An exciting new alternative involves the use of emotion- sensitive intelligent tutoring systems (ITSs). These tutoring systems attempt to incorporate the learner’s emotions (or affective states) into their pedagogical strategies in order to enhance engagement, motivation, and learning (D’Mello, et al., 2005; Forbes-Riley & Litman, 2004; Kort, Reilly, & Picard, 2001). A fundamental challenge in the development of an emotion-sensitive ITS involves reliably measuring the affective states of the learner. This requires the development of a user model that captures the manner in which learners intentionally or implicitly exhibit affect in a naturalistic learning environment. With the user model in hand, the next step is to develop a computational system to automatically diagnose and incorporate the emotions of learner into the pedagogical strategies of the ITS. This paper addresses these challenges by an in-depth analysis of body posture, a dimension that has been rarely investigated by cognitive scientists. We identified the manner in which learners express particular emotions by modulating their gross body language. We used automated posture tracking hardware and software instead of human coders. This allowed us to develop automated algorithms to identify learner emotions on the basis of the detected patterns between posture and affect. The use of posture to infer affect is interesting because it is rarely the case that posture is intentionally monitored by humans. The significance of nonverbal behaviors in expressing affect is widely acknowledged, but the vast majority of the scientific literature is restricted to the monitoring of facial features (Ekman & Friesen, 1978), speech contours (Ang et al., 2002), and physiological signals such as electromyography, heart rate, and skin conductance. Our rationale for expecting posture features to be diagnostic of affect in learning environments is motivated by embodied theories of cognition (Clark, 1997; Glenberg, Havas, Becker & Rinck, 2005; de Vega, 2002). Theories of embodied cognition postulate that cognitive processes are constrained substantially by the environment and by the coupling of perception and action. If the embodied theories are correct, then the cognitive and emotional states of a learner are manifested in their body language. An added advantage of monitoring posture patterns is that these motions are ordinarily unconscious, unintentional, and thereby not susceptible to social editing, at least compared with facial expressions and gestures. Ekman and Friesen (1969), in their studies of deception, have coined This research demonstrates the utility of automatically monitoring a student’s posture to track the affective states of boredom (low engagement) and flow (high engagement), which have been shown to influence learning. After a tutoring session with AutoTutor, the affective states of the student were rated by the learner, a peer, and two trained judges. Our results indicated that the affective state of flow was manifested through heightened pressure exerted on the seat of a pressure sensitive chair. Boredom, in turn, was associated with an increase in the pressure exerted on the back coupled with a rapid change in pressure on the seat, perhaps indicative of a state of restlessness. We also investigated the diagnosticity of each of the posture features and the reliability of a computer automatically discriminating episodes of boredom versus flow, which is a major discrimination in any affect-sensitive tutoring system. Keywords: Posture patterns; affective states; emotions; learning; AutoTutor; classifying affect; Intelligent Tutoring Systems Introduction The task of maintaining a student’s engagement in educational activities is extremely challenging. Establishing and maintaining the engagement of learners is especially critical in situations with high degrees of learner control, such as in distance education, computer-based tutoring, and informal learning environments. For instance, with web- based instruction, individuals are one-mouse-click-away from ending the session. Several traditional approaches have directly addressed this problem, such as collaborative learning (Palincsar & Brown, 1984), apprenticeship learning (Rogoff, 1990), educational games and simulations (Ferrari, Taylor, & VanLehn, 1999), and inquiry learning (Chinn & Malhotra, 2002). All of these approaches both promote active learning and offer scaffolding to sustain motivation and engagement. They also structure the learning environment so that it matches a student’s zone of proximal development (Brown, Ellery, & Campione, 1998) and learning rate (Metcalfe & Kornell, 2005). Determining the appropriate level of difficulty is a non-trivial task, however. It is not obvious what metrics accurately scale the relative difficulty of complex topics such as those in undergraduate science courses. Each individual has his/her own specific zone of proximal development which changes over time in response to instruction. The goal of providing training that hits this ‘moving target’ is exceptionally difficult.