Do Learning by Teaching Environments with Metacognitive Support Help Students Develop Better Learning Behaviors

Do Learning by Teaching Environments with Metacognitive Support Help Students Develop Better Learning Behaviors? John Wagster 1 , Jason Tan 1 , Yanna Wu 1 , Gautam Biswas 1 , and Dan Schwartz 2 Department of EECS/ISIS Vanderbilt University Nashville, TN 37235. School of Education Stanford University Stanford, CA 94305. where students teach a computer agent using well-structured visual representations. Using their agent’s performance (which is a function of how well it was taught) as a motiva- tion, students work to remediate the agent’s knowledge, and, in this process, they learn better on their own. We discuss one of our Teachable Agent Systems, Betty’s Brain, below. An important property of our TA environments is that stu- dents ideally monitor how their agents answer questions and solve problems, and they can correct them when they notice discrepancies between their own knowledge and the agent’s. For this reason our learning-by-teaching environments are well-suited to helping students become more knowledgeable of and responsible for their own cognition and reasoning. As a result, the students are likely to develop problem solving and monitoring skills that go beyond the learning of specific domain content; rather they provide the much larger frame- work that guide students on how to learn and how to pre- pare for future learning (Schwartz and Martin, 2004). Previous studies conducted in 5 th grade science class- rooms showed evidence that learning-by-teaching with me- tacognitive support helped students develop better learning and self-monitoring strategies, and this prepared them for future learning on related topics, even when this learning happened outside of the support provided by the TA envi- ronment (Biswas, et al., 2005). We also conjectured that the metacognitive support produced “learned behaviors” that were indicative of good learning practices. We combined in- tuition and empirical observations to select behaviors that we believed were indicative of independent learning with understanding. Preliminary analysis demonstrated that stu- dents in the learning by teaching condition with metacogni- tive feedback were more likely to demonstrate these beha- viors than students who did not receive this kind of feed- back. Students with these behaviors also showed better learning performance (see Tan, Biswas, and Schwartz (2006) and Tan, et al. (2007)). In this paper we perform a more systematic statistical analysis to link student learning and observed student beha- viors. The student learning measure is defined by their per- formance on the transfer task. As a second step, differences in the use of these behaviors between the three conditions are studied. The rest of the paper provides an overview of Betty’s Brain and the metacognitive support in the system, a description of our experimental study, and a summary of our findings and future work. Abstract We have developed Teachable Agent environments that use learning by teaching with metacognitive support to help mid- dle school students learn about complex science topics. To demonstrate the effectiveness of this approach, we have run studies that compare three systems where (i) students are taught by an agent, (ii) students teach a computer agent, and (iii) students teach a computer agent and receive metacogni- tive support while teaching. Students’ activities on the sys- tem, captured in log files, were coded using six primary learn- ing activities. In this paper, we analyze behavior fragments systematically derived from the activity sequences, and iden- tify behaviors that correlate well with high and low student performance. Our results show that students who teach and receive metacognitive support exhibit more of the high per- forming behaviors than the other two groups. Keywords: learning-by-teaching; metacognitive support, learning behaviors. Introduction We have been using learning by teaching models to create learning environments for middle school students that pro- mote the development of higher-order cognitive skills for problem solving in science and math domains (Biswas, et al., 2001; 2005; Schwartz, et al., to appear). To teach, one must gain a good understanding of the domain knowledge and then structure the knowledge in a form that they can present to others (Bargh, 1980). Preparing to teach is a self- directed and open-ended activity where one explores, inte- grates, and structures knowledge first for oneself, and then for others. In addition to preparatory activities, teachers an- swer questions, and provide explanations and demonstra- tions during teaching and receive feedback from their stu- dents. These activities also seem to have significant cogni- tive consequences. For example, we might expect that the teachers’ knowledge structures would become better orga- nized and differentiated through the process of communicat- ing key ideas and relationships to students and reflecting on students’ questions and feedback (Chi, 2001). We look upon teaching as a metacognitive, reflective, and iterative process with three main phases: decision-making, performing ac- tions, and monitoring (McAlpine, 1999). We have designed teachable agents (TA’s) that provide important structures to help shape teacher thinking (Biswas, et al, 2005; Blair, et al., 2004). TA’s are software programs