ethanporcaro/eric-fulltext-meta huggingface dataset:
Measuring Similarity of Educational Items Using Data on Learners&apos; Performance
Author: Rihák, Jirí
Date: 2017-06-00
Type: Speeches/Meeting Papers
Subjects: Item Analysis,Data Analysis,Visualization,Simulation,Correlation,Reaction Time,Measures (Individuals),Slavic Languages,Intelligent Tutoring Systems,Mathematics Skills,Mathematics Instruction,Spelling,Grammar,Language Tests,Comparative Analysis,Evaluation Methods

Educational systems typically contain a large pool of items (questions, problems). Using data mining techniques we can group these items into knowledge components, detect duplicated items and outliers, and identify missing items. To these ends, it is useful to analyze item similarities, which can be used as input to clustering or visualization techniques. We describe and evaluate different measures of item similarity that are based only on learners' performance data, which makes them widely applicable. We provide evaluation using both simulated data and real data from several educational systems. The results show that Pearson correlation is a suitable similarity measure and that response times are useful for improving stability of similarity measures when the scope of available data is small. [For the full proceedings, see ED596512.]
Modeling Classifiers for Virtual Internships without Participant Data
Author: Gautam, Dipesh
Date: 2017-06-00
Type: Speeches/Meeting Papers
Subjects: Computer Simulation,Internship Programs,Semantics,College Students,Corporations,Recall (Psychology),Models,Land Use,Programming,Teaching Methods,Journal Writing,Form Classes (Languages),Scoring,Public Health,Urban Planning,Information Retrieval,Computer Software

Virtual internships are online simulations of professional practice where students play the role of interns at a fictional company. During virtual internships, participants complete activities and then submit write-ups in the form of short answers, digital notebook entries. Prior work used classifiers trained on participant data to automatically assess notebook entries from these learning environments. However, when teachers create new internships using available authoring tools, no such data exists. We evaluate a method for generating classifiers using specifications provided by teachers during their authoring process instead of participant data. Our models rely on Latent Semantic Analysis based and Neural Network based semantic similarity approaches in which notebook entries are compared to ideal, expert generated responses. We also investigated a Regular Expression based model. The experiments on the proposed models on unseen data showed high precision and recall values for some classifiers using a similarity based approach. Regular Expression based classifiers performed better where the other two approaches did not, suggesting that these approaches may complement one another in future work. [For the full proceedings, see ED596512.]
Sequence Modelling for Analysing Student Interaction with Educational Systems
Author: Hansen, Christian
Date: 2017-06-00
Type: Speeches/Meeting Papers
Subjects: Foreign Countries,Markov Processes,Mathematical Models,Student Behavior,Probability,Mathematics,Interaction

The analysis of log data generated by online educational systems is an important task for improving the systems, and furthering our knowledge of how students learn. This paper uses previously unseen log data from Edulab, the largest provider of digital learning for mathematics in Denmark, to analyse the sessions of its users, where 1.08 million student sessions are extracted from a subset of their data. We propose to model students as a distribution of different underlying student behaviours, where the sequence of actions from each session belongs to an underlying student behaviour. We model student behaviour as Markov chains, such that a student is modelled as a distribution of Markov chains, which are estimated using a modified k-means clustering algorithm. The resulting Markov chains are readily interpretable, and in a qualitative analysis around 125,000 student sessions are identified as exhibiting unproductive student behaviour. Based on our results this student representation is promising, especially for educational systems offering many different learning usages, and offers an alternative to common approaches like modelling student behaviour as a single Markov chain often done in the literature. [For the full proceedings, see ED596512.]
Predicting Student Retention from Behavior in an Online Orientation Course
Author: Kai, Shimin
Date: 2017-06-00
Type: Speeches/Meeting Papers
Subjects: Online Courses,Student Behavior,Prediction,Models,College Students,Higher Education,At Risk Students,Identification,Orientation,Academic Persistence,School Holding Power,Computer Software,Program Descriptions,Educational Improvement,Program Development,Management Systems,Intervention

As higher education institutions develop fully online course programs to provide better access for the non-traditional learner, there is increasing interest in identifying students who may be at risk of attrition and poor performance in these online course programs. In our study, we investigate the effectiveness of an online orientation course in improving student retention in an online college program. Using student activity data from the orientation course, Engage, we make use of machine learning methods to develop prediction models of whether students will be retained and continue to register for program-specific courses in the eVersity program. We then discuss the implications of our findings on improvements that may be made to the existing orientation course to improve student retention in the program. [For the full proceedings, see ED596512.]
Inferring Frequently Asked Questions from Student Question Answering Forums
Author: Sindhgatta, Renuka
Date: 2017-06-00
Type: Reports - Research
Subjects: Questioning Techniques,Interviews,Electronic Learning,Online Courses,Semantics,Cluster Grouping,Teaching Methods,Middle School Students,High School Students

Question answering forums in online learning environments provide a valuable opportunity to gain insights as to what students are asking. Understanding frequently asked questions and topics on which questions are asked can help instructors in focusing on specific areas in the course content and correct students' confusions or misconceptions. An underlying task in inferring frequently asked questions is to identify similar questions based on their content. In this work, we use hierarchical agglomerative clustering that exploits similarities between words and their distributed representations, reflecting both lexical and semantic similarity of questions. We empirically evaluate our results on real world labeled dataset to demonstrate the effectiveness of the method. In addition, we report the results of inferring frequently asked questions from discussion forums of online learning environment providing lectures to middle school and high school students. [For the full proceedings, see ED596512.]
Linking Language to Math Success in an On-Line Course
Author: Crossley, Scott
Date: 2017-06-00
Type: Reports - Research
Subjects: Success,Mathematics Instruction,Language Usage,Blended Learning,Conventional Instruction,Undergraduate Students,College Mathematics,Models,Scores,Peer Teaching,Data Collection,Data Analysis,Language Proficiency,Syntax,Computer Mediated Communication,Educational Technology,Technology Uses in Education,Online Courses

This study takes a novel approach toward understanding success in a math course by examining the linguistic features and affect of students' language production within a blended (with both on-line and traditional face to face instruction) undergraduate course (n=158) on discrete mathematics. Three linear effects models were compared: (a) a baseline linear model including nonlinguistic fixed effects, (b) a model including only linguistic factors, (c) a model including both linguistic and non-linguistic effects. The best model (c) explained 16% of the variance of final course scores, revealing significant effects for one non-linguistic feature (days on the system) and two linguistic features (&quot;Number of dependents per prepositional object nominal and Sentence linking connectives&quot;). One non-linguistic factor (&quot;Is a peer tutor&quot;) and two linguistic variables (&quot;Words related to self and Words related to tool use&quot;) demonstrated marginal significance. The findings indicate that language proficiency is strongly linked to math performance such that more complex syntactic structures and fewer explicit cohesion devices equate to higher course performance. The linguistic model also indicated that less self-centered students and students using words related to tool use were more successful. In addition, the results indicate that students that are more active in on-line discussion forums are more likely to be successful. [For the full proceedings, see ED596512.]
Making the Grade: How Learner Engagement Changes after Passing a Course
Author: Lang, David
Date: 2017-06-00
Type: Reports - Research
Subjects: Learner Engagement,Outcomes of Education,Academic Achievement,Grades (Scholastic),Curriculum Design,Learning Motivation,Grading,Online Courses,Video Technology,Attitude Change,Interaction,Cutting Scores

Understanding how individuals interact with a course after receiving a passing grade could have important implications for course design. If individuals become disengaged after passing a class, then this may raise questions about optimal ordering of content, course difficulty, and grade transparency. Using a person-fixed effects model, we analyze how individuals who obtained passing grades subsequently behaved within a course. These learners were less likely to complete videos and more likely to watch videos faster after receiving notice of a passing grade in the class. These learners were also less likely to reattempt items they initially got wrong. [For the full proceedings, see ED596512. For the corresponding grantee submission, see ED578433.]
Predicting Prospective Peer Helpers to Provide Just-In-Time Help to Users in Question and Answer Forums
Author: Ishola, Oluwabukola Mayowa
Date: 2017-06-00
Type: Speeches/Meeting Papers
Subjects: Prediction,Peer Relationship,Lifelong Learning,Interviews,Helping Relationship,Help Seeking,Electronic Learning,Reaction Time,Discussion Groups

Question and answer forums are becoming more popular as increasing numbers of lifelong learners rely on such forums to receive help about their learning needs. Stack Overflow (SO) is an example of such a forum used by millions of programmers. The ability of users to receive timely answers to questions is crucial to the sustainability of such forums and for successful lifelong learning. In SO we have observed that the number of questions answered within 15 minutes have diminished with more questions taking a longer time to get answered or remaining unanswered in some cases. This suggests the need for an effective approach in predicting prospective helpers who can provide timely answers to the questions. In this paper, we seek to explore strategies to match helpers and help seekers. In particular we wish to use these strategies to predict which SO users will provide timely answers to questions asked in SO, and then compare these predictions to the users who actually answered the questions. In making these predictions we looked at 3 time frames of user data: 1 month, 3 months and 6 months. We used 5 basic strategies: frequency, knowledgeability, eagerness, willingness, recency; and we compared the success rates of each strategy in making predictions on 3 different success criteria: predicting the first answerer, predicting the answerer most liked by the asker of the question, and predicting the answerer rated most highly by other SO users. We then incorporated a timeliness measure, which takes into consideration how quickly the user provides answers to questions in the past, which helped us to achieve a higher success rate. The results of our study are an improvement over a similar previous study of SO and we hope will form the basis of methods for recommending peers in online forums who can provide just-intime help to lifelong learners as their knowledge needs evolve and change. [For the full proceedings, see ED596512.]
Predicting Short- and Long-Term Vocabulary Learning via Semantic Features of Partial Word Knowledge
Author: Nam, SungJin
Date: 2017-06-00
Type: Speeches/Meeting Papers
Subjects: Predictor Variables,Vocabulary Development,Semantics,Intelligent Tutoring Systems,Scores,Measures (Individuals),Knowledge Level,Middle School Students,Grade 6,Grade 7,Grade 8,Teaching Methods

We show how the novel use of a semantic representation based on Osgood's semantic differential scales can lead to effective features in predicting short- and long-term learning in students using a vocabulary learning system. Previous studies in students' intermediate knowledge states during vocabulary acquisition did not provide much information on which semantic knowledge students gained during word learning practice. Moreover, these studies relied on human ratings to evaluate the students' responses. To solve this problem, we propose a semantic representation for words based on Osgood's semantic decomposition of vocabulary [16]. To demonstrate our method can effectively represent students' knowledge in vocabulary acquisition, we build models for predicting the student's short-term vocabulary acquisition and long-term retention. We compare the effectiveness of our Osgood-based semantic representation to that provided by Word2Vec neural word embedding [13], and find that prediction models using features based on Osgood scale-based scores (OSG) perform better than the baseline and are comparable in accuracy to those using Word2Vec score-based models (W2V). By using more interpretable Osgood-based scales, our study results can help with better understanding of students' ongoing learning states and designing personalized learning systems that can address an individual's weak points in vocabulary acquisition. [For the full proceedings, see ED596512.]
Using a Single Model Trained across Multiple Experiments to Improve the Detection of Treatment Effects
Author: Patikorn, Thanaporn
Date: 2017-06-00
Type: Speeches/Meeting Papers
Subjects: Statistical Analysis,Outcomes of Treatment,Intervention,Randomized Controlled Trials,Electronic Learning,Middle School Students

In this work, we describe a new statistical method to improve the detection of treatment effects in interventions. We call our method TAME (Trained Across Multiple Experiments). TAME takes advantage of multiple experiments with similar designs to create a single model. We use this model to predict the outcome of the dependent variable in unseen experiments. We use the predictive accuracy of the model on the conditions of the experiment to determine if the treatment had a statistically significant effect. We validated the effectiveness of our model using a large-scale simulation study, where we showed that our model can detect treatment effects with 10% more statistical power than an ANOVA in certain settings. We also applied our model to real data collected from the ASSISTments online learning platform and showed that the treatment effects detected by our model were comparable to the effects detected by the ANOVA. [For the full proceedings, see ED596512.]
Online Learning Persistence and Academic Achievement
Author: Fang, Ying
Date: 2017-06-00
Type: Speeches/Meeting Papers
Subjects: Learning Processes,Academic Persistence,Correlation,Academic Achievement,Mathematics Instruction,Recordkeeping,Standardized Tests,Mastery Learning,Online Courses,Intelligent Tutoring Systems,Middle School Students,Teaching Methods,Achievement Tests,Scores,Pretests Posttests

Student persistence in online learning environments has typically been studied at the macro-level (e.g., completion of an online course, number of academic terms completed, etc.). The current examines student persistence in an adaptive learning environment, ALEKS (Assessment and LEarning in Knowledge Spaces). Specifically, the study explores the relationship between students' academic achievement and their persistence during learning. By using archived data that included their math learning log data and performance on two standardized tests, we first explored student learning behavior patterns with regard to their persistence during learning. Clustering analysis identified three distinctive patterns of persistence-related learning behaviors: (1) High persistence and rare topic shifting; (2) Low persistence and frequent topic shifting; and (3) Moderate persistence and moderate topic shifting. We further explored the association between persistence and academic achievement. No significant differences were observed between academic achievement and the different learning patterns. We interpret this result in addition to a preliminary exploration of topic mastery trends, to suggest that &quot;wheel-spinning&quot; behaviors coexist with persistence, and is ultimately not beneficial to learning. [For the full proceedings, see ED596512.]
Task and Timing: Separating Procedural and Tactical Knowledge in Student Models
Author: Cook, Joshua
Date: 2017-06-00
Type: Speeches/Meeting Papers
Subjects: Models,Bayesian Statistics,Data Use,Intelligent Tutoring Systems,Probability,Sequential Approach,Accuracy,Mathematical Logic,Learning Processes

BKT and other classical student models are designed for binary environments where actions are either correct or incorrect. These models face limitations in open-ended and data-driven environments where actions may be correct but non-ideal or where there may even be degrees of error. In this paper we present BKT-SR and RKT-SR: extensions of the existing BKT model that distinguish knowing &quot;how&quot; to apply a skill from knowing &quot;when.&quot; We compare their relative performance to that of classical BKT and PFA on data collected from Deep Thought, an open-ended propositional logic tutor. We develop basic performance curves for student outcomes to help us visually compare models predictions to data. We also introduce a new approach for finding a probability distribution of actions in ranked, multiple option environments with RKT and RKT-SR. Our results show that knowing when to use skills is more important than how in these open-ended contexts. In fact, including the &quot;how&quot; components may hurt performance if implemented naively. Furthermore we show that ranked models outperform binary-based models even under restrictive assumptions. [For the full proceedings, see ED596512.]
Towards Reliable and Valid Measurement of Individualized Student Parameters
Author: Liu, Ran
Date: 2017-06-00
Type: Speeches/Meeting Papers
Subjects: Data Analysis,Intelligent Tutoring Systems,Bayesian Statistics,Pretests Posttests,Educational Technology,Models,Reliability,Validity,Prior Learning,Goodness of Fit,High School Students,Geometry,Mathematics Instruction,Learning Processes,Identification,Mastery Learning,Individual Differences,Skill Development,Equations (Mathematics),Honors Curriculum,Inclusion,Mathematics Skills,General Education,Comparative Analysis

Research in Educational Data Mining could benefit from greater efforts to ensure that models yield reliable, valid, and interpretable parameter estimates. These efforts have especially been lacking for individualized student-parameter models. We collected two datasets from a sizable student population with excellent &quot;depth&quot; -- that is, many observations for each skill for each student. We fit two models, the Individualized-slope Additive Factors Model (iAFM) and Individualized Bayesian Knowledge Tracing (iBKT), both of which individualize for student ability and student learning rate. Estimates of student ability were reliable and valid: they were consistent across both models and across both datasets, and they significantly predicted out-of-tutor pretest data. In one of the datasets, estimates of student learning rate were reliable and valid: consistent across models and significantly predictive of pretest-posttest gains. This is the first demonstration that statistical models of data resulting from students' use of learning technology can produce reliable and valid estimates of individual student learning rates. Further, we sought to interpret and understand what differentiates a student with a high estimated learning rate from a student with a low one. We found that learning rate is significantly related to estimates of student ability (prior knowledge) and self-reported measures of diligence. Finally, we suggest a variety of possible applications of models with reliable estimates of individualized student parameters, including a more novel, straightforward way of identifying wheel spinning. [For the full proceedings, see ED596512.]
The Misidentified Identifiability Problem of Bayesian Knowledge Tracing
Author: Doroudi, Shayan
Date: 2017-06-00
Type: Speeches/Meeting Papers
Subjects: Bayesian Statistics,Research Problems,Models,Learning,Prediction,Markov Processes,Identification,Probability,Goodness of Fit

In this paper, we investigate two purported problems with Bayesian Knowledge Tracing (BKT), a popular statistical model of student learning: &quot;identifiability&quot; and &quot;semantic model degeneracy.&quot; In 2007, Beck and Chang stated that BKT is susceptible to an &quot;identifiability problem&quot;--various models with different parameters can give rise to the same predictions about student performance. We show that the problem they pointed out was not an identifiability problem, and using an existing result from the identifiability of hidden Markov models, we show that under mild conditions on the parameters, BKT is actually identifiable. In the second part of the paper, we discuss a problem that has been conflated with identifiability, but which actually does arise when fitting BKT models, &quot;semantic model degeneracy&quot;--the model parameters that best fit the data are inconsistent with the conceptual assumptions underlying BKT. We give some intuition for why semantic model degeneracy may arise by showing that BKT models fit to data generated from alternative models of student learning can have semantically degenerate parameters. Finally, we discuss the potential implications of these insights. [For the full proceedings, see ED596512. For a related grantee submission, see ED577166.]
On the Influence on Learning of Student Compliance with Prompts Fostering Self-Regulated Learning
Author: Lallé, Sébastien
Date: 2017-06-00
Type: Speeches/Meeting Papers
Subjects: Compliance (Psychology),Metacognition,Computer Software,Eye Movements,Time Management,Correlation,Hypermedia,Intelligent Tutoring Systems,Student Behavior,Cues,Learning Processes,Teaching Methods,Instructional Effectiveness,Science Instruction,Anatomy,Pretests Posttests,College Students

In this paper, we investigate the relationship between students' learning gains and their compliance with prompts fostering self-regulated learning (SRL) during interaction with MetaTutor, a hypermedia-based intelligent tutoring systems (ITS). When possible, we evaluate compliance from student explicit answers on whether they want to follow the prompts, When such answers are not available, &quot;we mine&quot; several student behaviors related to prompt compliance. These behaviors are derived from students' eye-tracking and interaction data (e.g., time spent on a learning page, number of gaze fixations on that page). Our results reveal that compliance with some, but not all SRL prompts provided by MetaTutor do influence learning. These results contribute to gain a better understanding of how students benefit from SRL prompts, and provides insights on how to further improve their effectiveness. For instance, prompts that do improve learning when followed could be the focus of adaptation designed to foster compliance for those students who would disregard them otherwise. Conversely, prompts that do not improve learning when followed could be improved based on further investigations to understand the reason for their lack of effectiveness. [For the full proceedings, see ED596512.]