A Computational Model of How Learner Errors Arise from Weak Prior Knowledge

A Computational Model of How Learner Errors Arise from Weak Prior Knowledge oboru Matsuda (noboru.matsuda@cs.cmu.edu) Andrew Lee (andrewlee@cmu.edu) William W. Cohen (wcohen@cs.cmu.edu) Kenneth R. Koedinger (koedinger@cmu.edu) School of Computer Science, Carnegie Mellon University 5000 Forbes Ave., Pittsburgh, PA 15213 USA Abstract How do differences in prior conceptual knowledge affect the nature and rate of learning? To answer this question, we built a computational model of learning, called SimStudent, and conducted a controlled simulation study to investigate how learning a complex skill changes when the system is given “weak” domain-general vs. “strong” domain-specific prior knowledge. We measured SimStudent’s learning outcomes as the rate of learning, the accuracy of learned skills (test scores), and the fit to the pattern of errors made by real students. We found that when the “weak” prior knowledge is given, not only the accuracy of learned skills decreases, but also the learning rate significantly slows down. The accuracy of predicting student errors increased significantly – namely, SimStudent with the weak prior knowledge made the same errors that real students commonly make. These modeling results help explain empirical results connecting prior knowledge and student learning (Booth & Koedinger, 2008). Keywords: Computational model of learning; machine learning; SimStudent; weak prior knowledge; patterns of student errors; mathematics education. Introduction In this paper, we present an innovative application of a synthetic student for modeling the error-prone process of student learning in a complex problem-solving domain. Previous studies have shown that student misconceptions or flaws in their prior knowledge not only directly cause errors in solving problems (VanLehn & Jones, 1993), but may also affect learning. For instance, Booth and Koedinger (2008) demonstrated that particular limitations in prior knowledge (e.g., treating terms in an equation as though terms and numbers were equivalent concepts) were correlated with particular strategic errors later in instruction (e.g., subtracting 4 from both sides of x- 4=13). The presumed causal connection is that the nature of student prior knowledge changes the learning process and thus leads to differences in the problem-solving knowledge that is acquired. But what is this learning process and how is it affected by differences in prior knowledge? A classic result from Chi, Feltovich, and Glaser’s study (1981) that experts categorize problems with deep solution- relevant features while novices categorize problems with shallow, perceptually apparent, features is also relevant to our endeavor. Also, Novick and Holyak (1991) found that domain expertise is a significant predictor of analogical transfer, but general analogical reasoning skill is not. We are ultimately interested in understanding how a novice goes from only being aware of shallow features to learning to encode problems in terms of deep features. Our strategy toward tackling this important question is to create a computational model of the learning process in complex math and science domains and to use fine-grain data from student learning over time to constrain model development. Our first steps involve demonstrating how a computational model of learning can learn when given shallow (or “weak”) knowledge, how such learning is slower than when deep (or “strong”) knowledge is available, and how learning based on shallow/weak knowledge better predicts patterns of real student errors. In this study, we focus on the process of learning problem-solving skills from examples, where students generalize examples to inductively learn skills to solve problems. We are particularly interested in errors that are made by applying incorrect skills, and our computational model explains the processes of learning such incorrect skills as incorrect induction from examples. A number of models of student errors have been proposed (Brown & Burton, 1978; Langley & Ohlsson, 1984; Sleeman, Kelly, Martinak, Ward, & Moore, 1989; Weber, 1996; Young & O'Shea, 1981). Our effort builds on the past works by exploring how differences in prior knowledge affect the nature of the incorrect skills acquired and the errors derived. We hypothesize that incorrect generalizations are more likely when students have weaker, more general prior knowledge for encoding incoming information. This knowledge is typically perceptually grounded and is in contrast to deeper or more abstract encoding knowledge. An example of such perceptually grounded prior knowledge is to recognize 3 in x/3 simply as a number instead of as a denominator. Such an interpretation might lead students to learn an inappropriate generalization such as “multiply both sides by a number in the left hand side of the equation” after observing x/3=5 gets x=15. If this generalization gets applied to an equation like 4x=2, the error of multiplying both sides by 4 is produced. We call this type of perceptually grounded prior knowledge “weak” prior knowledge in a similar sense as Newell and Simon’s weak reasoning methods (1972). Weak knowledge can apply across domains and can yield successful results prior to domain-specific instruction. However, in contrast to “strong” domain-specific knowledge, weak knowledge is more likely to lead to incorrect conclusions. The goal of the present paper is to investigate an impact of the prior knowledge on learning problem-solving skills