ethanporcaro/eric-fulltext-meta huggingface dataset:
Variational Item Response Theory: Fast, Accurate, and Expressive
Author: Wu, Mike
Date: 2020-07-00
Type: Speeches/Meeting Papers
Subjects: Item Response Theory,Accuracy,Data Analysis,Public Policy,Bayesian Statistics,Computation,Responses,Open Source Technology,Scoring,Inferences,Monte Carlo Methods,Grading,Computer Software,Second Language Learning,Second Language Instruction,Achievement Tests,International Assessment,Secondary School Students,Foreign Countries,Measures (Individuals),Language Skills,Language Tests,Vocabulary Development,English (Second Language)

Item Response Theory (IRT) is a ubiquitous model for understanding humans based on their responses to questions, used in fields as diverse as education, medicine and psychology. Large modern datasets offer opportunities to capture more nuances in human behavior, potentially improving test scoring and better informing public policy. Yet larger datasets pose a difficult speed/accuracy challenge to contemporary algorithms for fitting IRT models. We introduce a variational Bayesian inference algorithm for IRT, and show that it is fast and scaleable without sacrificing accuracy. Using this inference approach we then extend classic IRT with expressive Bayesian models of responses. Applying this method to five large-scale item response datasets from cognitive science and education yields higher log likelihoods and improvements in imputing missing data. The algorithm implementation is open-source, and easily usable. [For the full proceedings, see ED607784.]
Prescribing Deep Attentive Score Prediction Attracts Improved Student Engagement
Author: Lee, Youngnam
Date: 2020-07-00
Type: Speeches/Meeting Papers
Subjects: Intelligent Tutoring Systems,Prediction,Scores,Learner Engagement,Accuracy,Artificial Intelligence,English (Second Language),Second Language Instruction,Foreign Countries,Language Tests,Second Language Learning

Intelligent Tutoring Systems (ITSs) have been developed to provide students with personalized learning experiences by adaptively generating learning paths optimized for each individual. Within the vast scope of ITS, score prediction stands out as an area of study that enables students to construct individually realistic goals based on their current position. Via the expected score provided by the ITS, a student can instantaneously compare one's expected score to one's actual score, which directly corresponds to the reliability that the ITS can instill. In other words, refining the precision of predicted scores strictly correlates to the level of confidence that a student may have with an ITS, which will evidently ensue improved student engagement. However, previous studies have solely concentrated on improving the performance of a prediction model, largely lacking focus on the benefits generated by its practical application. In this paper, we demonstrate that the accuracy of the score prediction model deployed in a real-world setting significantly impacts user engagement by providing empirical evidence. To that end, we apply a state-of-the-art deep attentive neural network-based score prediction model to &quot;Santa,&quot; a multi-platform English ITS with approximately 780K users in South Korea that exclusively focuses on the TOEIC (Test of English for International Communications) standardized examinations. We run a controlled A/B test on the ITS with two models, respectively based on collaborative filtering and deep attentive neural networks, to verify whether the more accurate model engenders any student engagement. The results conclude that the attentive model not only induces high student morale (e.g. higher diagnostic test completion ratio, number of questions answered, etc.) but also encourages active engagement (e.g. higher purchase rate, improved total profit, etc.) on &quot;Santa.&quot; [For the full proceedings, see ED607784.]
Student Teamwork on Programming Projects What Can GitHub Logs Show Us?
Author: Gitinabard, Niki
Date: 2020-07-00
Type: Speeches/Meeting Papers
Subjects: Teamwork,Group Activities,Student Projects,Programming,College Students,Computer Science Education,Classification,Data Analysis,Automation

Teamwork, often mediated by version control systems such as Git and Apache Subversion (SVN), is central to professional programming. As a consequence, many colleges are incorporating both collaboration and online development environments into their curricula even in introductory courses. In this research, we collected GitHub logs from two programming projects in two offerings of a CS2 Java programming course for computer science majors. Students worked in pairs for both projects (one optional, the other mandatory) in each year. We used the students' GitHub history to classify the student teams into three groups, &quot;collaborative,&quot; &quot;cooperative,&quot; or &quot;solo-submit,&quot; based on the division of labor. We then calculated different metrics for students' teamwork including the total number and the average number of commits in different parts of the projects and used these metrics to predict the students' teamwork style. Our findings show that we can identify the students' teamwork style automatically from their submission logs. This work helps us to better understand novices' habits while using version control systems. These habits can identify the harmful working styles among them and might lead to the development of automatic scaffolds for teamwork and peer support in the future. [For the full proceedings, see ED607784.]
What Time Is It? Student Modeling Needs to Know
Author: Mao, Ye
Date: 2020-07-00
Type: Speeches/Meeting Papers
Subjects: Time,Models,Artificial Intelligence,Bayesian Statistics,Individualized Instruction,Pacing,Intelligent Tutoring Systems,Prediction,Learning Analytics,Computer Science Education,College Students,Probability,Mathematics Instruction,Success,Achievement Gains

Modeling student learning processes is highly complex since it is influenced by many factors such as motivation and learning habits. The high volume of features and tools provided by computer-based learning environments confounds the task of tracking student knowledge even further. Deep Learning models such as Long-Short Term Memory (LSTMs) and classic Markovian models such as Bayesian Knowledge Tracing (BKT) have been successfully applied for student modeling. However, much of this prior work is designed to handle sequences of events with &quot;discrete timesteps,&quot; rather than considering the continuous aspect of time. Given that time elapsed between successive elements in a student's trajectory can vary from seconds to days, we applied a Time-aware LSTM (T-LSTM) to model the dynamics of student knowledge state &quot;in continuous time.&quot; We investigate the effectiveness of T-LSTM on two domains with very different characteristics. One involves an open-ended programming environment where students can &quot;self-pace&quot; their progress and T-LSTM is compared against LSTM, Recent Temporal Pattern Mining, and the classic Logistic Regression (LR) on the early prediction of student success; the other involves a classic tutor-driven intelligent tutoring system where the tutor scaffolds the student learning step by step and T-LSTM is compared with LSTM, LR, and BKT on the early prediction of student learning gains. Our results show that TLSTM significantly outperforms the other methods on the self-paced, open-ended programming environment; while on the &quot;tutor-driven&quot; ITS, it ties with LSTM and outperforms both LR and BKT. In other words, while time-irregularity exists in both datasets, T-LSTM works significantly better than other student models when the pace is driven by students. On the other hand, when such irregularity results from the tutor, T-LSTM was not superior to other models but its performance was not hurt either. [For the full proceedings, see ED607784.]
Problem Detection in Peer Assessments between Subjects by Effective Transfer Learning and Active Learning
Author: Xiao, Yunkai
Date: 2020-07-00
Type: Speeches/Meeting Papers
Subjects: Peer Evaluation,Problems,Identification,Natural Language Processing,Artificial Intelligence,Classification,Computer Uses in Education

Peer assessment adds value when students provide &quot;helpful&quot; feedback to their peers. But, this begs the question of how we determine &quot;helpfulness.&quot; One important aspect is whether the review detects problems in the submitted work. To recognize problem detection, researchers have employed NLP and machine-learning text classification methods. Past studies have used datasets that were narrowly focused on a small number of classes in specific academic fields. This paper reports on how well models trained on one dataset or field perform on data from classes that are unlike the classes whose data they have been trained on. Specifically we took a model developed with data from a computer science class with several programming assignments, and tried to transfer it onto an education class focused more on writing research papers. We have attempted to perform such a task on a few models including logistic regression classifier, random forest classifier, multinomial naive bayes classifier and support vector machine. We made several attempts to raise the accuracy of classification, including lemmatizing to deduct variation in data input, and active learning strategies. [For the full proceedings, see ED607784.]
The Ebb and Flow of Student Engagement: Measuring Motivation through Temporal Pattern of Self-Regulation
Author: Dang, Steven C.
Date: 2020-07-00
Type: Speeches/Meeting Papers
Subjects: Learner Engagement,Intelligent Tutoring Systems,Student Behavior,Student Motivation,Time,Data Analysis,Self Management

Effective teachers recognize the importance of transitioning students into learning activities for the day and accounting for the natural drift of student attention while creating lesson plans. In this work, we analyze temporal patterns of gaming behaviors during work on an intelligent tutoring system with a broader goal of detecting temporal trends in students' motivation. Findings demonstrate that observing gaming the system behaviors in the near beginning or end of a working session correspond with predictions made by self-regulation theories of ego-depletion and task-switching. Furthermore, analyses provide initial evidence these gaming behaviors are indicative of partial cognitive engagement and session-level influences on student motivation. These findings provide evidence for how temporal fluctuations in students motivations might be inferred through self-regulated behaviors like gaming the system, and how such information could inform better more intelligent tutoring systems that are responsive to cognitive and motivational dynamics during student work. [For the full proceedings, see ED607784.]
Using Online Textbook and In-Class Poll Data to Predict In-Class Performance
Author: Hunt-Isaak, Noah
Date: 2020-07-00
Type: Speeches/Meeting Papers
Subjects: Textbooks,Surveys,Grade Prediction,Undergraduate Students,Introductory Courses,Computer Science Education,Data Analysis,Data Use,Electronic Publishing

National failure rates seen in undergraduate introductory CS courses are quite high. In this paper, we develop a predictive model for student in-class performance in an introductory CS course. The model can serve as an early warning system, flagging struggling students who might benefit from additional support. We use a variety of features from the first few weeks of the course such as scores on assignments, interaction with the online textbook, and participation with the in-class polling system in order to train our models. We compare the performance of a number of machine learning algorithms on predicting final exam scores as well as final course grade. We find that the Support Vector Machine and AdaBoost are the most effective, and that we can achieve increasingly accurate predictions as we use data from further into the course. The regression coefficients give us insights into which features are most correlated with student success, suggesting that certain types of assignments are more indicative of learning than others. [For the full proceedings, see ED607784.]
The Results of Implementing Zone of Proximal Development on Learning Outcomes
Author: Baker, Ryan
Date: 2020-07-00
Type: Speeches/Meeting Papers
Subjects: Individualized Instruction,Electronic Learning,Learning Theories,Cognitive Development,Outcomes of Education,Foreign Countries,Middle School Students,Elementary School Students,Homework,Mathematics Instruction,English (Second Language),Second Language Instruction,Mastery Learning,Instructional Effectiveness

With the development of personalized learning in technological platforms, more data and information are given to instructors on what contents are appropriate for a learner's next step, with an aim of helping them support their students in navigating an optimized learning path that can promote an enhanced learning outcome. In this study, we collected data from an online learning platform, Learnta® TAD , which allows teachers to distribute tasks based on system recommendations. The recommendations are directed by the system's knowledge graph algorithm, determining whether the student is ready to learn the task (i.e. the task is within the student's Zone of Proximal Development), whether the student is not yet ready to learn the task, or whether the student has already mastered the task. We used the acquired data to investigate whether giving content in each of these groups results in different learning outcomes. Statistical methods such as subgroup analysis, Fisher's exact test, and logistic regression are conducted to address the proposed topic. Replicating a prior, smaller-scale study, our findings suggest that the student gains more mastery when assigned Ready-to-Learn tasks than when assigned Unready-to-Learn tasks, across Math and English, more and less successful students, and in-class and homework. Moreover, students who are given already mastered tasks perform better than those who are given Ready-to-Learn and Unready-to-Learn tasks across all groups. [For the full proceedings, see ED607784.]
Zero-Shot Learning of Hint Policy via Reinforcement Learning and Program Synthesis
Author: Efremov, Aleksandr
Date: 2020-07-00
Type: Speeches/Meeting Papers
Subjects: Intelligent Tutoring Systems,Feedback (Response),Computer Science Education,Artificial Intelligence,Programming,Educational Technology,Learning Analytics

Intelligent tutoring systems for programming education can support students by providing personalized feedback when a student is stuck in a coding task. We study the problem of designing a hint policy to provide a next-step hint to students from their current partial solution, e.g., which line of code should be edited next. The state of the art techniques for designing a hint policy use supervised learning approach, however, require access to historical student data containing trajectories of partial solutions written when solving the task successfully. These techniques are limited in applicability when needed to provide feedback for a new task without any available data, or to a new student whose trajectory of partial solutions is very different from that seen in historical data. To this end, we tackle the &quot;zero-shot&quot; challenge of learning a hint policy to be able to assist the very first student who is solving a task, without relying on any data. We propose a novel &quot;reinforcement learning&quot; (RL) framework to solve the challenge by leveraging recent advancements in RL-based neural &quot;program synthesis.&quot; Our framework is modular and amenable to several extensions, such as designing appropriate reward functions for adding a desired feature in the type of provided hints and allowing to incorporate student data from the same or related tasks to further boost the performance of the hint policy. We demonstrate the effectiveness of our RL-based hint policy on a publicly available dataset from Code.org, the world's largest programming education platform. [For the full proceedings, see ED607784.]
Replicating the CGI Experiment in Diverse Environments: Effects on Grade 1 and 2 Student Mathematics Achievement in the First Program Year. Research Report No. 2020-02
Author: Schoen, Robert C.
Date: 2020-00-00
Type: Reports - Research
Subjects: Elementary School Students,Grade 1,Grade 2,Mathematics Achievement,Effect Size,Mathematics Tests,Problem Solving,Mathematical Applications,Algebra,Mathematics Instruction,Mathematics Teachers,Faculty Development,Achievement Tests,Standardized Tests,Program Evaluation,Pedagogical Content Knowledge,Teacher Workshops,Student Characteristics

Using a cluster-randomized controlled trial research design, this study investigates the effect of the first year of a three-year teacher professional development program on grades 1 and 2 student achievement in mathematics. Although the findings reported in the present report do not meet the standard cutoff for statistical significance (e.g., 95% confidence), effect-size estimates for student achievement in the first year of implementation were positive for the tests that focused on problem solving, applications of mathematics, and algebraic thinking, and they were negative for the computation-focused tests. Those effects varied by grade level. Implications and suggestions for next steps are discussed. [This report was prepared by the Florida Center for Research in Science, Technology, Engineering, and Mathematics (FCR-STEM), Learning Systems Institute, Florida State University.]
Towards Suggesting Actionable Interventions for Wheel-Spinning Students
Author: Mu, Tong
Date: 2020-07-00
Type: Reports - Research
Subjects: Computer Uses in Education,Artificial Intelligence,Academic Failure,Automation,Intervention,Prediction,Accuracy,Educational Technology,Foreign Countries,English (Second Language),Reading Programs,Students

In some computerized educational systems, there is evidence of students &quot;wheel-spinning,&quot; where a student tries and repeatedly fails at an educational task for learning a skill. This may be particularly concerning in low resource settings. Prior research has focused on predicting and modeling wheel-spinning, but there has been little work on how to best help students stuck in wheel-spinning. We use past student system interaction data and a minimal amount of expert input to automatically inform individualized interventions, without needing experts to label a large dataset of interventions. Our method trains a model to predict wheel-spinning and utilizes a popular tool in interpretable machine learning, Shapley values, to provide individualized credit attribution over the features of the model, including actionable features like possible gaps in prerequisites. In simulation on two different statistical student models, our approach can identify a correct intervention with over 80% accuracy before the simulated student begins the activity they will wheel spin on. In our real dataset we show initial qualitative results that our proposed interventions match what an expert would prescribe. [For the full proceedings, see ED607784.]
Unsupervised Approach for Modeling Content Structures of MOOCs
Author: Alsaad, Fareedah
Date: 2020-07-00
Type: Speeches/Meeting Papers
Subjects: Online Courses,Graphs,Course Content,Cluster Grouping,Mathematics

With the increased number of MOOC offerings, it is unclear how these courses are related. Previous work has focused on capturing the prerequisite relationships between courses, lectures, and concepts. However, it is also essential to model the content structure of MOOC courses. Constructing a precedence graph that models the similarities and variations of learning paths followed by similar MOOCs would help both students and instructors. Students can personalize their learning by choosing the desired learning path and lectures across several courses guided by the precedence graph. Similarly, by examining the precedence graph, instructors can 1) identify knowledge gaps in their MOOC offerings, and 2) find alternative course plans. In this paper, we propose an unsupervised approach to build the precedence graph of similar MOOCs, where nodes are clusters of lectures with similar content, and edges depict alternative precedence relationships. Our approach to cluster similar lectures based on PCK-Means clustering algorithm that incorporates pairwise constraints: Must-Link and Cannot-Link with the standard K-Means algorithm. To build the precedence graph, we link the clusters according to the precedence relations mined from current MOOCs. Experiments over real-world MOOC data show that PCK-Means with our proposed pairwise constraints outperform the K-Means algorithm in both Adjusted Mutual Information (AMI) and Fowlkes-Mallows scores (FMI). [For the full proceedings, see ED607784.]
Recommending Remedial Readings Using Student Knowledge State
Author: Thaker, Khushboo
Date: 2020-07-00
Type: Speeches/Meeting Papers
Subjects: Automation,Textbooks,Electronic Learning,Artificial Intelligence,Feedback (Response),Knowledge Level,Graduate Students

Assessment plays a vital role in learning, as it provides both instructors and students with feedback on the overall effectiveness of their teaching or learning. However, when a student fails to correctly answer certain questions in an assessment (such as a quiz), the student needs specific recommendations that are tailored to their learning needs and to the knowledge deficiency exposed by the assessment outcomes. In this paper, we explore the methods for automatically identifying the recommended textbook materials that are most relevant and suitable to the student. In particular, we conducted experiments on how to incorporate students' current knowledge state on domain concepts associated with the activity to recommend personalized remedial sections to each student. The results show that incorporating student knowledge states can significantly improve the quality of recommendations as compared to traditional content-based recommendations. [For the full proceedings, see ED607784.]
The Effect of Teachers Reassigning Students to New Cognitive Tutor Sections
Author: Sales, Adam C.
Date: 2020-07-00
Type: Speeches/Meeting Papers
Subjects: Student Placement,Intelligent Tutoring Systems,Algebra,Mathematics Instruction,Scores,Pretests Posttests,Mastery Learning,High School Students,Statistical Analysis,Instructional Effectiveness

The design of the Cognitive Tutor Algebra I (CTA1) intelligent tutoring system assumes that students work through sections of material following a pre-specified order, and only move on from one section to the next after mastering the first section's skills. However, the software gives teachers the flexibility to override that structure, by reassigning students to different sections of the curriculum. Which students get reassigned? Does reassignment hurt student learning? Does it help? This paper used data from the treatment arm of a large effectiveness study of the CTA1 curriculum to estimate the effects of reassignment on students' scores on an Algebra I posttest. Since reassignment is not randomized, we used a multilevel propensity score matching design, along with assessments of sensitivity to bias from unmeasured confounding, to estimate the effects of reassignment. We found that reassignment reduces posttest scores by roughly 0.2 standard deviations--about the same as the overall CTA1 treatment effect--that unmeasured confounding is unlikely to completely explain this observed effect, and that the effect of reassignment may vary widely between classrooms. [For the full proceedings, see ED607784.]