Considering Alternate Futures to Classify Off-Task Behavior as Emotion Self-Regulation: A Supervised Learning Approach
Author: Sabourin, Jennifer L.
Date: 2013-00-00
Type: Journal Articles
Subjects: Student Behavior,Classification,Learner Engagement,Data Analysis,Interaction,Middle School Students,Microbiology,Science Instruction,Intelligent Tutoring Systems,Psychological Patterns,Self Control,Artificial Intelligence,Bayesian Statistics,Models,Futures (of Society),Comparative Analysis,Accuracy,Reliability,Grade 8,Prediction,Educational Games,Computer Games

Over the past decade, there has been growing interest in real-time assessment of student engagement and motivation during interactions with educational software. Detecting symptoms of disengagement, such as off-task behavior, has shown considerable promise for understanding students' motivational characteristics during learning. In this paper, we investigate the affective role of off-task behavior by analyzing data from student interactions with CRYSTAL ISLAND, a narrative-centered learning environment for middle school microbiology. We observe that off-task behavior is associated with reduced student learning, but preliminary analyses of students' affective transitions suggest that off-task behavior may also serve a productive role for some students coping with negative affective states such as frustration. Empirical findings imply that some students may use off-task behavior as a strategy for self-regulating negative emotional states during learning. Based on these observations, we introduce a supervised machine learning procedure for detecting whether students' off-task behaviors are cases of emotion self-regulation. The method proceeds in three stages. During the first stage, a dynamic Bayesian network (DBN) is trained to model the valence of students' emotion self-reports using collected data from interactions with the learning environment. In the second stage, a novel simulation process uses the DBN to generate &quot;alternate futures&quot; by modeling students' affective trajectories as if they had engaged in fewer off-task behaviors than they did during their actual learning interactions. The alternate futures are compared to students' actual traces to produce labels denoting whether students' off-task behaviors are cases of emotion self-regulation. In the final stage, the generated emotion self-regulation labels are predicted using off-the-shelf classifiers and features that can be computed in run-time settings. Results suggest that this approach shows promise for identifying cases of off-task behavior that are emotion self-regulation. Analyses of the first two phases suggest that trained DBN models are capable of accurately modeling relationships between students' off-task behaviors and self-reported emotional valence in CRYSTAL ISLAND. Additionally, the proposed simulation process produces emotion self-regulation labels with high levels of reliability. Preliminary analyses indicate that support vector machines, bagged trees, and random forests show promise for predicting the generated emotion self-regulation labels, but room for improvement remains. The findings underscore the methodological potential of considering alternate futures when modeling students' emotion self-regulation processes in narrative-centered learning environments.