ethanporcaro/eric-fulltext-meta huggingface dataset:
Identifying Collaborative Learning States Using Unsupervised Machine Learning on Eye-Tracking, Physiological and Motion Sensor Data
Author: Huang, Karina
Date: 2019-07-00
Type: Speeches/Meeting Papers
Subjects: Cooperative Learning,Artificial Intelligence,Eye Movements,Learning Analytics,Motion,Physiology,Attention,Human Posture

With the advent of new data collection techniques, there has been a growing interest in studying co-located groups of students using Multimodal Learning Analytics to automatically identify collaborative learning states. In this paper, we analyze a multimodal dataset (N=84) made of eye-tracking, physiological and motion sensing data. We leverage unsupervised machine learning algorithms to find (un)productive collaborative states. We found a three-states solution where different states (and transitions between them) were significantly correlated with task performance, collaboration quality and learning gains. We interpret these findings in light of collaborative learning theories and discuss their implications for studying groups of students using MMLA. [For the full proceedings, see ED599096.]
Leveraging Deep Reinforcement Learning for Pedagogical Policy Induction in an Intelligent Tutoring System
Author: Ausin, Markel Sanz
Date: 2019-07-00
Type: Speeches/Meeting Papers
Subjects: Reinforcement,Intelligent Tutoring Systems,Teaching Methods,Instructional Effectiveness,Educational Policy,Computer Simulation,Online Courses,Barriers,Assignments,Rewards,Scores,Pretests Posttests,College Students,Mathematics Instruction,Mathematical Logic,Validity

Deep Reinforcement Learning (DRL) has been shown to be a very powerful technique in recent years on a wide range of applications. Much of the prior DRL work took the &quot;online&quot; learning approach. However, given the challenges of building accurate simulations for modeling student learning, we investigated applying DRL to induce a pedagogical policy through an &quot;offiine&quot; approach. In this work, we explored the effectiveness of offiine DRL for pedagogical policy induction in an Intelligent Tutoring System. Generally speaking, when applying offiine DRL, we face two major challenges: one is limited training data and the other is the credit assignment problem caused by delayed rewards. In this work, we used Gaussian Processes to solve the credit assignment problem by estimating the inferred immediate rewards from the final delayed rewards. We then applied the DQN and Double-DQN algorithms to induce adaptive pedagogical strategies tailored to individual students. Our empirical results show that without solving the credit assignment problem, the DQN policy, although better than Double-DQN, was no better than a random policy. However, when combining DQN with the inferred rewards, our best DQN policy can outperform the random yet reasonable policy, especially for students with high pre-test scores. [For the full proceedings, see ED599096.]
Measuring Item Teaching Value in an Online Learning Environment
Author: Harmon, Jon
Date: 2019-07-00
Type: Speeches/Meeting Papers
Subjects: Electronic Learning,Factor Analysis,Goodness of Fit,Item Response Theory,College Students,Intelligent Tutoring Systems,Correlation,Homework,Program Effectiveness,Skill Development,Predictive Validity,Chemistry

The Additive Factor Model (AFM) is a cognitive diagnostic model that can be used to predict student performance on items in a context that allows for student learning. Within AFM, &quot;skills&quot; have a learning rate, and student acquisition of a skill depends only on the number of opportunities a student has had to exercise that skill and the learning rate of that skill. Here we demonstrate an approach to measure the &quot;teaching value&quot; of individual &quot;items&quot; with respect to one another. The teaching values estimated through this approach may be useful for structuring intelligent tutoring systems and for content improvement. [For the full proceedings, see ED599096.]
Machine-Learned School Dropout Early Warning at Scale
Author: Christie, S. Thomas
Date: 2019-07-00
Type: Speeches/Meeting Papers
Subjects: Dropout Prevention,At Risk Students,Artificial Intelligence,Decision Support Systems,Prediction,Student Records,Enrollment,Information Systems,Models,Middle School Students,High School Students,Statistical Analysis

Schools across the United States suffer from low on-time graduation rates. Targeted interventions help at-risk students meet graduation requirements in a timely manner, but identifying these students takes time and practice, as warning signs are often context-specific and reflected in a combination of attendance, social, and academic signals scattered across data sources. Extremely high caseloads for counselors compound the problem. At Infinite Campus, a large student information system provider, we modeled statistical relationships between student educational records and enrollment outcomes, using de-identified records and in-system analysis to guarantee student data privacy. The resulting risk scores are highly predictive, context-sensitive, nationally available, integrated into the existing student information system, and updated daily. [For the full proceedings, see ED599096.]
Exploring the Link between Motivations and Gaming
Author: Dang, Steven
Date: 2019-07-00
Type: Speeches/Meeting Papers
Subjects: Correlation,Self Management,Student Motivation,Student Behavior,Personality Traits,Intelligent Tutoring Systems,Middle School Students,Suburban Schools,Mathematics Instruction,Student Educational Objectives,Self Efficacy

A student's ability to regulate their thoughts, emotions and behaviors in the face of temptation is linked to their task specific motivational goals and dispositions. Behavioral tasks are designed to strain a targeted resource to differentiate individuals through measures of their performance. In this paper, we explore how student behavior on differentially gamed learning material relates to estimates of student motivational goals and dispositions. We leverage observations of students in two different courses using an intelligent tutoring system over an entire academic year. We use a previously validated heuristic model of gaming detection to label instances of gaming. Each student's tendency to game is estimated separately using highly gamed and non-highly gamed sections of the course. Each estimate of student gaming is compared to precourse self-reported measures of student motivations. Results indicate that in naturalistic settings, gaming on more challenging materials is less influenced by student motivations and potentially a result of adaptive learning behaviors. Similarly, student gaming estimates using only non-highly gamed material are significantly related to all targeted motivation measures. Implications and future directions are discussed. [For the full proceedings, see ED599096.]
Success Prediction in MOOCs: A Case Study
Author: Pigeau, Antoine
Date: 2019-07-00
Type: Speeches/Meeting Papers
Subjects: Success,Large Group Instruction,Online Courses,Prediction,Foreign Countries,Educational Technology,Technology Uses in Education,Classification,Learning Analytics,Artificial Intelligence

Success prediction in Massive Open Online Courses (MOOCs) is now tackled in numerous works, but still needs new case studies to compare the solutions proposed. We study here a specific dataset from a French MOOC provided by the OpenClassrooms company, featuring 12 courses. We exploit various features present in the literature and test several classification models. [For the full proceedings, see ED599096.]
One Minute Is Enough: Early Prediction of Student Success and Event-Level Difficulty During a Novice Programming Task
Author: Mao, Ye
Date: 2019-07-00
Type: Reports - Research
Subjects: Difficulty Level,Learning Activities,Prediction,Programming,Computer Science Education,Short Term Memory,Intelligent Tutoring Systems,Models,Academic Achievement,Task Analysis,Dropouts,Prevention,Accuracy,Novices,Classification,College Students,Research Universities,Bayesian Statistics,Data Analysis,Teaching Methods

Early prediction of student difficulty during long-duration learning activities allows a tutoring system to intervene by providing needed support, such as a hint, or by alerting an instructor. To be effective, these predictions must come early and be highly accurate, but such predictions are difficult for open-ended programming problems. In this work, Recent Temporal Patterns (RTPs) are used in conjunction with Support Vector Machine and Logistic Regression to build robust yet interpretable models for early predictions. We performed two tasks: to predict student success and difficulty during one, open-ended novice programming task of drawing a square-shaped spiral. We compared RTP against several machine learning models ranging from the classic to the more recent deep learning models such as Long Short Term Memory to predict whether students would be able to complete the programming task. Our results show that RTP-based models outperformed all others, and could successfully classify students after just one minute of a 20- minute exercise (students can spend more than 1 hour on it). To determine when a system might intervene to prevent incompleteness or eventual dropout, we applied RTP at regular intervals to predict whether a student would make progress within the next five minutes, reflecting that they may be having difficulty. RTP successfully classified these students needing interventions over 85% of the time, with increased accuracy using data-driven program features. These results contribute significantly to the potential to build a fully data-driven tutoring system for novice programming. [For the full proceedings, see ED599096.]
Using Knowledge Component Modeling to Increase Domain Understanding in a Digital Learning Game
Author: Nguyen, Huy
Date: 2019-07-00
Type: Speeches/Meeting Papers
Subjects: Middle School Students,Knowledge Representation,Models,Computer Games,Educational Games,Number Concepts,Goodness of Fit,Grade 6,Regression (Statistics),Learning Processes,Intelligent Tutoring Systems

Knowledge components (KCs) define the underlying skill model of intelligent educational software, and they are critical to understanding and improving the efficacy of learning technology. In this research, we show how learning curve analysis is used to fit a KC model--one that was created after use of the learning technology--which can then be improved by human-centered data science methods. We analyzed data from 417 middle-school students who used a digital learning game to learn decimal numbers and decimal operations. Our initial results showed that problem types (e.g., ordering decimals, adding decimals) capture students' performance better than underlying decimal misconceptions (e.g., longer decimals are larger). Through a process of KC model refinement and domain knowledge interpretation, we were able to identify the difficulties that students faced in learning decimals. Based on this result, we present an instructional redesign proposal for our digital learning game and outline a framework for post-hoc KC modeling in a tutoring system. More generally, the method we used in this work can help guide changes to the type, content and order of problems in educational software. [For the full proceedings, see ED599096.]
Early Detection of Wheel Spinning: Comparison across Tutors, Models, Features, and Operationalizations
Author: Zhang, Chuankai
Date: 2019-07-00
Type: Speeches/Meeting Papers
Subjects: Identification,Intelligent Tutoring Systems,Academic Failure,Criteria,Algebra,Geometry,Mathematics Instruction,Models,Comparative Analysis,Knowledge Level

&quot;Wheel spinning&quot; is the phenomenon in which a student fails to master a Knowledge Component (KC), despite significant practice. Ideally, an intelligent tutoring system would detect this phenomenon early, so that the system or a teacher could try alternative instructional strategies. Prior work has put forward several criteria for wheel spinning and has demonstrated that wheel spinning can be detected reasonably early. Yet the literature lacks systematic comparisons among the multiple wheel spinning criteria, features, and models that have been proposed, across multiple evaluation criteria (e.g., earliness, precision, and generalizability) and datasets. In our experiments, we constructed six wheel spinning detectors and compared their performance under two different wheel spinning criteria with three datasets. The results show that two prominent criteria for wheel spinning diverge substantially, and that a Random Forest model has the most consistent performance in early detection of wheel spinning across datasets and wheel spinning criteria. In addition, we found that a simple model overlooked by previous research (Logistic Regression trained on a single feature) is able to detect wheel spinning at an early stage with decent performance. This work brings us closer to unifying strands of prior work on wheel spinning (e.g., understanding how different criteria compare) and to early detection of wheel spinning in educational practice. [For the corresponding grantee submission, see ED594575. For the full proceedings, see ED599096.]
Predicting Early and Often: Predictive Student Modeling for Block-Based Programming Environments
Author: Emerson, Andrew
Date: 2019-07-00
Type: Speeches/Meeting Papers
Subjects: Programming,Computer Science Education,Feedback (Response),Prediction,Models,Prior Learning,Cues,Program Effectiveness,Success,College Students,Coding,Educational Technology,Technology Uses in Education

Recent years have seen a growing interest in block-based programming environments for computer science education. While these environments hold significant potential for novice programmers, they lack the adaptive support necessary to accommodate students exhibiting a wide range of initial capabilities and dispositions toward computing. A promising approach to addressing this problem is introducing adaptive feedback. This work investigates a key capability for adaptive support: training student models that predict student success in block-based programming activities for novice programmers. The predictive student models utilize four categories of features: prior performance, hint usage, activity progress, and interface interaction. In addition to evaluating the accuracy of these models for multiple block-based programming activities, we also investigate how quickly the models converge to accurate prediction, and we evaluate the additive value of each of the four categories of features. Results show that the predictive models are able to predict whether a student will successfully complete an exercise with high accuracy, as well as converge on this prediction early in the sequence of student interactions. [For the full proceedings, see ED599096.]
What Will You Do Next? A Sequence Analysis on the Student Transitions Between Online Platforms in Blended Courses
Author: Gitinabard, Niki
Date: 2019-07-00
Type: Speeches/Meeting Papers
Subjects: Blended Learning,Student Adjustment,Online Courses,Study Habits,High Achievement,Low Achievement,Behavior Patterns,Sequential Approach,Intelligent Tutoring Systems,Data Analysis,State Universities,Discussion Groups,Integrated Learning Systems,Introductory Courses,Undergraduate Students,Mathematics Instruction,Predictive Measurement

Students' interactions with online tools can provide us with insights into their study and work habits. Prior research has shown that these habits, even as simple as the number of actions or the time spent on online platforms can distinguish between the higher performing students and low-performers. These habits are also often used to predict students' performance in classes. One key feature of these actions that is often overlooked is how and when the students transition between different online platforms. In this work, we study sequences of student transitions between online tools in blended courses and identify which habits make the most difference between the higher and lower performing groups. While our results showed that most of the time students focus on a single tool, we were able to find patterns in their transitions to differentiate high and low performing groups. These findings can help instructors to provide procedural guidance to the students, as well as to identify harmful habits and make timely interventions. [For the full proceedings, see ED599096.]
Modeling and Experimental Design for MOOC Dropout Prediction: A Replication Perspective
Author: Gardner, Josh
Date: 2019-07-00
Type: Speeches/Meeting Papers
Subjects: Online Courses,Large Group Instruction,Prediction,Models,Long Term Memory,Short Term Memory,Artificial Intelligence,Dropouts,Generalization,Learning Analytics

Replication of machine learning experiments can be a useful tool to evaluate how both &quot;modeling&quot; and &quot;experimental design&quot; contribute to experimental results; however, existing replication efforts focus almost entirely on modeling alone. In this work, we conduct a three-part replication case study of a state-of-the-art LSTM dropout prediction model. In our first experiment, we replicate the original authors' methodology as precisely as possible in collaboration with the original authors. In a second experiment, we demonstrate that this initial experiment likely overestimates the generalization performance of the proposed model due to the design of its validation. In a third experiment, we attempt to achieve the previously-reported performance in a more difficult, but more relevant, hold-out set design by exploring a large space of model regularization configurations. We demonstrate that we can reduce overfitting and improve generalization performance of the model, but cannot achieve the previously-reported level of performance. This work demonstrates the importance of replication of predictive modeling experiments in education, and demonstrates how experimental design and modeling decisions can impact the extent to which mode [For the full proceedings, see ED599096.]
Predicting the Quality of Collaborative Problem Solving through Linguistic Analysis of Discourse
Author: Reilly, Joseph M.
Date: 2019-07-00
Type: Speeches/Meeting Papers
Subjects: Problem Solving,Discourse Analysis,Cooperative Learning,Computer Assisted Instruction,Nonverbal Communication,Natural Language Processing,Computational Linguistics,Achievement Gains,Programming Languages,Data Analysis,Academic Achievement,Connected Discourse,Eye Movements,College Students,Computer Science Education,Correlation,Intervention,Teaching Methods,Task Analysis,Prediction

Collaborative problem solving in computer-supported environments is of critical importance to the modern workforce. Coworkers or collaborators must be able to co-create and navigate a shared problem space using discourse and non-verbal cues. Analyzing this discourse can give insights into how consensus is reached and can estimate the depth of their understanding of the problem. This study uses Coh-Metrix, a natural language processing tool that measures cohesion, to analyze participant discourse from a recent multi-modal learning analytics study where novice programmers collaborated to use a block-based programming language to instruct a robot on how to solve a series of mazes. We significantly correlated thirty-five Coh-Metrix indices from the transcripts of dyads' discourse with collaboration, learning gains, and multimodal sensor values. We then fit a variety of machine learning classifiers to predict collaboration using the indices generated by Coh-Metrix as features. This study paves the way for real-time detection of (un)productive interactions from multimodal data and could lead to real-time interventions to support collaborative learning. [For the full proceedings, see ED599096.]
Why Deep Knowledge Tracing Has Less Depth than Anticipated
Author: Ding, Xinyi
Date: 2019-07-00
Type: Speeches/Meeting Papers
Subjects: Intelligent Tutoring Systems,Artificial Intelligence,Models,Skills,Relationship,Time,Ability

Knowledge tracing allows Intelligent Tutoring Systems to infer which topics or skills a student has mastered, thus adjusting curriculum accordingly. Deep Knowledge Tracing (DKT) uses recurrent neural networks (RNNs) for knowledge tracing and has achieved significant improvements compared with models like Bayesian Knowledge Tracing (BKT) and Performance Factor Analysis (PFA). However, DKT is not as interpretable as other models because the decision-making process learned by recurrent neural networks is not wholly understood by the research community. In this paper, we critically examine the DKT model, visualizing and analyzing the behaviors of DKT in high dimensional space. We modify and explore the DKT model and discover that Deep Knowledge Tracing has some critical pitfalls: 1). instead of tracking each skill through time, DKT is more likely to learn an 'ability' model; 2) the recurrent nature of DKT reinforces irrelevant information that it uses during the tracking task; 3) an untrained recurrent network can achieve similar results to a trained DKT model, supporting a conclusion that recurrence relations are not properly learned and, instead, improvements are simply a benefit of projection into a high dimensional, sparse vector space. Based on these observations, we propose improvements and future directions for conducting knowledge tracing research using deep models. [For the full proceedings, see ED599096.]
On Longitudinal Item Response Theory Models: A Didactic
Author: Wang, Chun
Date: 2019-00-00
Type: Reports - Research
Subjects: Longitudinal Studies,Item Response Theory,Comparative Analysis,Models,Data Analysis,Measurement Techniques,Outcomes of Education,Computer Software,Multivariate Analysis

Recent work on measuring growth with categorical outcome variables has combined the item response theory (IRT) measurement model with the latent growth curve (LGC) model (e.g., McArdle, 1988) and extended the assessment of growth to multidimensional IRT models (e.g., Hsieh, von Eye, &amp; Maier, 2010; Huang, 2013) and higher-order IRT models (e.g., Huang, 2015). However, there is a lack of synthetic studies that clearly evaluate the strength and limitations of different multilevel IRT models for measuring growth. This study aims to introduce the various longitudinal IRT models, including the longitudinal unidimensional IRT model (L-UIRT), longitudinal multidimensional IRT model (L-MIRT), and longitudinal higher-order IRT model (L-HO-IRT), which cover a broad range of applications in education and social science. Following a comparison of the parameterizations, identification constraints, strengths, and weaknesses of the different models, a real data example is provided to illustrate the application of different longitudinal IRT models to model students' growth trajectories on multiple latent abilities. [The paper will be published in &quot;Journal of Educational and Behavioral Statistics.&quot;]