Emotional intelligence for computer tutors

We are developing intelligent tutors that elicit, sense, communicate, measure and respond to students’ emotion. The vision is to use affect in conjunction with cognition to customize instruction and exploit the potential impact of responding to a student’s affective state. This paper describes software and hardware solutions to redress the cognitive vs. affective imbalance to teaching. We investigate problems of sensing and responding to affective experiences, such as frustration, motivation, and self-confidence. We describe theories and technologies to appropriately integrate affect into intelligent tutors and are investigating how to respond to emotions. 1. Connecting emotion to learning The obvious next frontier in computational instruction is to systematically examine the relationship(s) between affective components in feedback and learning outcome (performance) (Shute 2006). If computer tutors are to interact naturally with humans, they need to recognize emotion and express social competencies. Emotion is completely intertwined with cognition in guiding rational behavior, including memory and decision-making and the human brain is described as a system in which emotion and cognitive functions are inextricably integrated (Cytowic 1989). Emotion has been shown to be more influential than cognitive abilities for personal, career and scholastic success (Goleman, 1996) and understanding human emotion is one of the greatest challenges of our time (Norman, 1981). For instance, impulsivity was twice as powerful a predictor as verbal IQ in future delinquent behavior (Block, 1995). Recent findings suggest that too little emotion is not desirable. When basic mechanisms of emotion are missing in the brain, intelligent functioning is hindered. Teachers recognize the central role of emotion, devoting as much time to the achievement of students’ motivational goals as to their cognitive and informational goals (Lepper and Hodell, 1989). Students who feel anxious or depressed do not assimilate information properly (Goleman, 1996). These emotions paralyze the so-called ‘active’ or ‘working memory,’ which sustains the ability to keep information about a task (Baddeley, 1986); in fact, some studies found that the level of worry of failure is directly proportional to achievement level (Ortony et al., 1988). Students with high intrinsic motivation often outperform students with low intrinsic motivation. Caring relationships have also been shown to be related to academic performance. The encoding of affect within human-human interactions is very powerful. In their research on “thin slices,” Ambady and Rosenthal demonstrate that when participants in their studies are shown a short segment of video, as little as six seconds of a teacher’s first interactions with their student, they can predict teacher effectiveness and student end of term grades (Ambady and Rosenthal, 1993). Curiously, however, theory and classroom practice have tended to privilege the ‘cognitive’ over the ‘affective,’ where affect is often ignored or marginalized in theories of learning that often view thinking and learning as information processing (Picard et al., 2004). The extension of cognitive theory to explain and exploit the role of affect in learning is at best in its infancy. This research aims to redress the imbalance between cognition and affect by developing theories and technologies to appropriately integrate both in human-computer tutors. We exploit the potential on learning outcome of responding to a student’s affective state and investigate the problems of sensing and responding to affective experiences, such as frustration, motivation, and self-confidence. Research questions include: • How do emotions predict learning? • How is emotion evidenced in student behavior? Are these dependencies consistent across teaching environments, e.g., open-ended critical thinking vs. problem solving systems? • How accurate are different models at predicting emotions from student behaviors? How do latent Bayesian or hidden Markov models compare to other models (such as plain regression)?