ethanporcaro/eric-fulltext-meta huggingface dataset:
Detecting Creativity in an Open Ended Geometry Environment
Author: Shillo, Roi
Date: 2019-07-00
Type: Speeches/Meeting Papers
Subjects: Geometry,Mathematics Instruction,Visualization,Teaching Methods,Creative Thinking,Geometric Concepts,Computer Software,Prediction,Undergraduate Students,Creativity Tests,Classification

Creativity is a dynamic process which generates ideas that are both novel and of value. However there is little understanding in what drives creativity in students and how to help teachers or education experts to detect creative thinking. This paper begins to address this gap by providing a platform and experiments for studying how creative outcomes can ensue over time. The platform is an open ended environment for creating geometrical shapes that supports exploration and trial and error. We show that participants exhibit diversity in their usage patterns in the system, and in particular, some exhibiting 'creative leaps' in which they move from creating a sequence of shapes in one category to another, new category. We designed a visualization tool that aids understanding in detecting these aspects in students' work. We provide a basic computational model that is able to predict whether a student will create a new shape at a given point in time. The impact of this work is in beginning to provide tools for promoting creativity in students and directing their interactions in a way that facilitates the creative process. [For the full proceedings, see ED599096.]
Detecting Outlier Behaviors in Student Progress Trajectories Using a Repeated Fuzzy Clustering Approach
Author: Howlin, Colm P.
Date: 2019-07-00
Type: Speeches/Meeting Papers
Subjects: Data Collection,Student Behavior,Learning Strategies,Feedback (Response),College Students,Time Factors (Learning),Pacing,Cognitive Style,Cluster Grouping,Mathematics

Clustering of educational data allows similar students to be grouped, in either crisp or fuzzy sets, based on their similarities. Standard approaches are well suited to identifying common student behaviors; however, by design, they put much less emphasis on less common behaviors or outliers. The approach presented in this paper employs fuzzing clustering in the identification of these outlier behaviors. The algorithm is an iterative one, where clustering is applied, outliers identified, the data restricted to the outliers, and the process repeated. This approach produces a clustering that is crisp between each iteration and fuzzy within. It arose as a consequence of trying to cluster student progress trajectories in an adaptive learning platform. Included are results from applying the repeated fuzzy clustering algorithm to data from multiple courses and semesters at the University of Central Florida, (N=5,044). [For the full proceedings, see ED599096.]
Detecting Suggestions in Peer Assessments
Author: Zingle, Gabriel
Date: 2019-07-00
Type: Speeches/Meeting Papers
Subjects: Peer Evaluation,Formative Evaluation,Evaluation Methods,Classification,Data Analysis,Artificial Intelligence

Peer assessment has proven to be a useful strategy for increasing the timeliness and quantity of formative feedback, as well as for promoting metacognitive thinking among students. Previous research has determined that reviews that contain suggestions can motivate students to revise and improve their work. This paper describes a method for automatically detecting suggestions in review text. The quantity of suggestions can be treated as a metric for the helpfulness of review text. Even before a review is submitted, the system can tell a reviewer when a review is lacking in suggestions and consequently advise that they be added. This paper presents several neural-network approaches for detecting suggestions and compares them against traditional natural language processing (NLP) methods such as rule-based techniques, as well as past machine-learning approaches. Our network-based classifiers outperformed rule-based classifiers in every experiment. Our neural-network classifiers attained F1- scores in the low 90% range, outperforming the support vector machine (SVM) classifier whose F1-score was 88%. The naïve Bayes (NB) classifier had an F1-score of 84% and the rule-based classifier had an F1-score of 80%. As in other domains such as determining sentiment, we found that neural-network models perform better than the likes of naïve Bayes and support vector machines when classifying suggestions in text. [For the full proceedings, see ED599096.]
Detecting Wheel-Spinning and Productive Persistence in Educational Games
Author: Owen, V. Elizabeth
Date: 2019-07-00
Type: Speeches/Meeting Papers
Subjects: Educational Games,Persistence,Productivity,Student Behavior,Game Based Learning,Identification,Preschool Children,Kindergarten,Mathematics Instruction,Computer Assisted Instruction

Games in service of learning are uniquely positioned to offer immersive, interactive educational experiences. Well-designed games build challenge through a series of well-ordered problems or activities, in which perseverance is key for working through ingame failure and increasing game difficulty. Indeed, persistence through challenges during learning is beneficial not just in games but in other contexts as well, with grit and perseverance positively associated with academic performance and learning outcomes. However, recent studies suggest that not all persistence is positive, suggesting that many students end up &quot;wheel-spinning&quot;, spending considerable time on a topic without achieving mastery. Thus, it is vital to differentiate productive and unproductive persistence in order to understand emergent student progress, particularly in the context of learning games and personalized learning systems, in which individual pathways differ greatly based on student needs. Leveraging Educational Data Mining methods, this study builds a detector of wheel-spinning behavior (differentiated from productive persistence) in an adaptive, game-based learning system. With the ability to predict unproductive persistence early, this detection model can be used to intelligently adapt to students needing further support in-system, as well as informing in-person intervention in a classroom setting--thus supporting a personalized, engaging learning experience in both formal and informal learning environments. [For the full proceedings, see ED599096.]
Grade Prediction Based on Cumulative Knowledge and Co-Taken Courses
Author: Ren, Zhiyun
Date: 2019-07-00
Type: Speeches/Meeting Papers
Subjects: Grade Prediction,Undergraduate Students,Prior Learning,Courses,Models,Predictor Variables,Majors (Students),Knowledge Level

Over the past decade, low graduation and retention rates have plagued higher education institutions. To help students graduate on time and achieve optimal learning outcomes, many institutions provide advising services supported by educational technologies. Accurate grade prediction is an integral part of these services such as degree planning software, personalized advising systems and early warning systems that can identify students at-risk of dropping from their field of study. In this work, we present next-term grade prediction models based on students' cumulative knowledge and co-taken courses. The proposed models are based on a matrix factorization framework and incorporate a co-taken course interaction function to learn the influence from the co-taken courses on the target course. The co-taken course interaction function is formed by a neural network, which takes the knowledge difference between the co-taken courses and the target course as input, and outputs an influence value that will be used to predict students' grades on the target course. The experimental results on various datasets from a U.S. University demonstrate that the proposed models significantly outperform competitive baselines across different test sets. Furthermore, we analyze the proposed models' performance with different numbers of co-taken courses as well as different numbers of co-taken course subjects, and highlight with an application case study how a student might make decisions related to selection of courses. The codes are available at https://github.com/Zhiyun0411/EDM. [For the full proceedings, see ED599096.]
Grades Are Not Normal: Improving Exam Score Models Using the Logit-Normal Distribution
Author: Arthurs, Noah
Date: 2019-07-00
Type: Speeches/Meeting Papers
Subjects: Grades (Scholastic),Scores,Statistical Distributions,Models,Item Response Theory,Grading,Undergraduate Students

Understanding exam score distributions has implications for item response theory (IRT), grade curving, and downstream modeling tasks such as peer grading. Historically, grades have been assumed to be normally distributed, and to this day the normal is the ubiquitous choice for modeling exam scores. While this is a good assumption for tests comprised of equally-weighted dichotomous items, it breaks down on the highly polytomous domain of undergraduate-level exams. The logit-normal is a natural alternative because it is has a bounded range, can represent asymmetric distributions, and lines up with IRT models that perform logistic transformations on normally distributed abilities. To tackle this question, we analyze an anonymized dataset from Gradescope consisting of over 4000 highly polytomous undergraduate exams. We show that the logit-normal better models this data without having more parameters than the normal. In addition, we propose a new continuous polytomous IRT model that reduces the number of item-parameters by using a logit-normal assumption at the item level. [For the full proceedings, see ED599096.]
Grading Emails and Generating Feedback
Author: Unnam, Abhishek
Date: 2019-07-00
Type: Speeches/Meeting Papers
Subjects: Grading,Electronic Mail,Feedback (Response),Written Language,Business Communication,Writing Skills,Artificial Intelligence,Accuracy,Computer Software,Comparative Analysis,Writing Evaluation,Evaluators,Grammar,English (Second Language),Second Language Learning,Correlation,Text Structure

Email has become the most preferred form of business communication. Writing &quot;good&quot; email has become an essential skill required in the industry. &quot;Good&quot; email writing not only facilitates clear communication, but also makes a positive impression on the recipient, whether it be one's colleague or a customer. The aim of this paper is to demystify the components of a &quot;good&quot; email and to define a set of parameters by which to grade the quality of an email and provide detailed feedback. This can help candidates improve their email writing skills and also guide tutors. The email grading parameters encompass traditional attributes of written English (i.e. coherent and relevant content and correct grammar) but also include a unique set of characteristics that we may objectively identify as email etiquette. These characterstics comprise the metrics we use to evaluate the quality of the various constituent parts of an email. We grade the email using artificial intelligence, acting on semi-structured text. We use a mix of machine learning and rule-based systems to effectively grade an email on the specified parameters. Our system automatically grades email with accuracy comparable to human graders. [For the full proceedings, see ED599096.]
Implicit and Explicit Emotions in MOOCs
Author: Syed, Munira
Date: 2019-07-00
Type: Speeches/Meeting Papers
Subjects: Online Courses,Electronic Learning,Affective Behavior,Emotional Response,Outcomes of Education,Discussion Groups,Student Surveys,Interaction,Positive Attitudes,Introductory Courses,Statistics

Understanding the affect expressed by learners is essential for enriching the learning experience in Massive Open Online Courses (MOOCs). However, online learning environments, especially MOOCs, pose several challenges in understanding the different types of affect experienced by a learner. In this paper, we define two categories of emotions, explicit emotions as those collected directly from the student through self-reported surveys, and implicit emotions as those inferred unobtrusively during the learning process. We also introduce positivity as a measure to study the valence reported by students chronologically, and use it to derive insights into their emotion patterns and their association with learning outcomes. We show that implicit and explicit emotions expressed by students within the context of a MOOC are independent of each other, however, they correlate better with students' behavior compared to their valence. [For the full proceedings, see ED599096.]
Modeling Person-Specific Development of Math Skills in Continuous Time: New Evidence for Mutualism
Author: Ou, Lu
Date: 2019-07-00
Type: Speeches/Meeting Papers
Subjects: Mathematics Skills,Skill Development,Time,Models,Electronic Learning,Computation,Addition,Educational Technology,Elementary School Students

In this study, we fitted a mixed-effects nonlinear continuous-time mutualism model of skill development proposed by van der Maas et al. (2006) to naturally collected irregularly spaced time series data from an online adaptive practice system for mathematics called Math Garden. Results showed that the mutualism model provided a better fit to the data than a g-factor model. The paper illustrates continuous-time modeling of irregularly-spaced multivariate time series data that are increasingly prevalent in modern learning systems. [For the full proceedings, see ED599096.]
Planning Gamification Strategies Based on User Characteristics and DM: A Gender-Based Case Study
Author: Toda, Armando M.
Date: 2019-07-00
Type: Speeches/Meeting Papers
Subjects: Gender Differences,Game Based Learning,Preferences,Design,Student Characteristics,STEM Education,Computer Games,Competition,Cooperation,Social Influences,Data Use,Rewards

Gamification frameworks can aid in gamification planning for education. Most frameworks, however, do not provide ways to select, relate or recommend how to use game elements, to gamify a certain educational task. Instead, most provide a &quot;one-size-fits-all&quot; approach covering all learners, without considering different user characteristics, such as gender. Therefore, this work aims to adopt a data-driven approach to provide a set of game element recommendations, based on user preferences, that could be used by teachers and instructors to gamify learning activities. We analysed data from a novel survey of 733 people (male=569 and female=164), collecting information about user preferences regarding game elements. Our results suggest that the most important rules were based on four (out of nineteen) types of game elements: Objectives, Levels, Progress and Choice. From the perspective of user gender, for the female sample, the most interesting rule associated Objectives with Progress, Badges and Information (confidence=0.97), whilst the most interesting rule for the male sample associated also Objectives with Progress, Renovation and Choice (confidence=0.94). These rules and our descriptive analysis provides recommendations on how game elements can be used in educational scenarios. [For the full proceedings, see ED599096.]
Kappa Learning: A New Item-Similarity Method for Clustering Educational Items from Response Data
Author: Nazaretsky, Tanya
Date: 2019-07-00
Type: Reports - Evaluative
Subjects: Intelligent Tutoring Systems,Item Response Theory,Measurement,Testing,Test Items,Mastery Learning,Mathematics Tests,Computer Assisted Testing

Sequencing items in adaptive learning systems typically relies on a large pool of interactive question items that are analyzed into a hierarchy of skills, also known as Knowledge Components (KCs). Educational data mining techniques can be used to analyze students response data in order to optimize the mapping of items to KCs, with similarity-based clustering as one of the two main approaches for this type of analysis. However, current similarity-based methods make the implicit assumption that students' performance on items that belong to the same KC should be similar. This assumption holds if the latent trait (mastery of the underlying skill) is relatively fixed during students' activity, as in the context of testing, which is the primary context in which these methods were developed and applied. However, in adaptive learning systems that aim for learning, and address subject matters such as K-6 Math that consist of multiple sub-skills, this assumption does not hold. In this paper we propose a new item-similarity measure, termed Kappa Learning (KL), which aims to address this gap. KL identifies similarity between items under the assumption of learning, namely, that learners' mastery of the underlying skills changes as they progress through the items. We evaluate KL on data from a K-6 Math Intelligent Tutoring System, with experts' tagging as ground truth, and on simulated data. Our results show that clustering that is based on KL outperforms clustering that is based on commonly used similarity measures (Cohen's Kappa, Yule, and Pearson), and that KL is also superior in the task of discovering the number of KCs. [For the full proceedings, see ED599096.]
Evaluating Fairness and Generalizability in Models Predicting On-Time Graduation from College Applications
Author: Hutt, Stephen
Date: 2019-07-00
Type: Speeches/Meeting Papers
Subjects: Generalization,Predictive Measurement,College Applicants,Time to Degree,Graduation Rate,Student Characteristics,Academic Achievement,Scores,Standardized Tests,Extracurricular Activities,Student Participation,Work Experience,Foreign Students,Bachelors Degrees,Public Schools,Validity

We explore generalizability and fairness across sociodemographic groups for predicting on-time college graduation using a national dataset of 41,359 college applications. Our features include sociodemographics, institutional graduation rates, academic achievement, standardized test scores, engagement in extracurricular activities, and work experiences. We identify five latent classes based on available sociodemographic data and train Random Forest classifiers to successfully predict 4-year graduation. When individually trained and tested on each class using a split-half validation method, we achieved AUROCs between 0.629 and 0.694. We then evaluate how a model trained on the entire dataset performs on each latent class by performing a slicing analysis, finding a 6 to 10 percent improvement in AUROCs compared to the individual-class models. We explore fairness of our model by extending the slicing analysis to consider Absolute Between ROC Area (ABROCA), finding similar values for each of our latent classes. We contemplate how our results might be used to avoid perpetuating biases inherent in college application data. [Additional funding was provided by the Mindset Scholars Network. For the full proceedings, see ED599096.]
Exploring Neural Network Models for the Classification of Students in Highly Interactive Environments
Author: Käser, Tanja
Date: 2019-07-00
Type: Speeches/Meeting Papers
Subjects: Educational Environment,Interaction,Cluster Grouping,Models,Short Term Memory,Educational Games,Accuracy,Student Behavior,Prediction,Middle School Students,Grade 8,Sequential Approach,Discovery Learning,Intervention

Open-ended learning environments (OELEs) allow students to freely interact with the content and to discover important principles and concepts of the learning domain on their own. However, only some students possess the necessary skills for efficient and effective exploration. Guidance in the form of targeted interventions or feedback therefore has the potential to improve educational outcomes. A promising approach for adaptation in OELEs is the design of interventions based on the detection of characteristic learning behaviors through offline clustering, followed by a real-time classification of new students. In this paper, we explore the possibility of using recurrent neural network (RNN) models for this online classification task. We extensively evaluate the predictive performance of different variants of RNNs, namely long-short term memory models and gated recurrent units, and different architectures on a data set collected from an exploration-based educational game. We also compare the prediction accuracy of the different RNN models to the performance of traditional classifiers on the same data set. Our results demonstrate that RNNs perform similar or better than traditional methods regarding early classification and therefore constitute a promising alternative for the online classification of new students. [For the full proceedings, see ED599096.]
Generalizing Expert Misconception Diagnoses through Common Wrong Answer Embedding
Author: Kolb, John
Date: 2019-07-00
Type: Speeches/Meeting Papers
Subjects: Misconceptions,Expertise,Mathematics Teachers,Semantics,Generalization,STEM Education,Tutoring,Natural Language Processing,Educational Diagnosis

Misconceptions have been an important area of study in STEM education towards improving our understanding of learners' construction of knowledge. The advent of largescale tutoring systems has given rise to an abundance of data in the form of learner question-answer logs in which signatures of misconceptions can be mined. In this work, we explore the extent to which collected expert misconception diagnoses can be generalized to held-out questions to add misconception semantics. We attempt this generalization by way of a question-answer neural embedding trained on chronological sequences of learner answers. As part of our study, we collect natural language misconception diagnoses from math educators for a sampling of student answers to questions within four topics on Khan Academy. Drawing inspiration from machine translation, we use a multinomial logistic regression model to explore how well the expert misconception semantics, in the form of bag-of-words vectors, can be mapped onto the learned embedding space and interpolated. We evaluate the ability of the space to generalize expert diagnoses using three levels of cross-fold validation in which we measure the recall of predicted natural language diagnoses across rater, topics, and questions. We find that the embedding provides generalization performance substantially beyond baseline approaches. [For the full proceedings, see ED599096.]
Generalizability of Sensor-Free Affect Detection Models in a Longitudinal Dataset of Tens of Thousands of Students
Author: Jensen, Emily
Date: 2019-07-00
Type: Speeches/Meeting Papers
Subjects: Generalization,Longitudinal Studies,Psychological Patterns,Identification,Electronic Learning,Sample Size,Models,Mathematics Instruction,Middle School Students,High School Students,Multivariate Analysis

Recent work in predictive modeling has called for increased scrutiny of how models generalize between different populations within the training data. Using interaction data from 69,174 students who used an online mathematics platform over an entire school year, we trained a sensor-free affect detection model and studied its generalizability to clusters of students based on typical platform use and demographic features. We show that models trained on one group perform similarly well when tested on the other groups, although there was a small advantage obtained by training individual subpopulation models compared to a general (all-population) model. Lastly, we perform a series of simulations to show how generalizability is affected by sample size. These results agree with our initial analysis that individual subpopulation models yield a small advantage over all-population models. Additionally, we show that training sizes smaller than 1,500 yield unstable models which make generalizability difficult to interpret. We discuss applications of this work in the context of developing large-scale affect detection models for diverse populations. [For the full proceedings, see ED599096. For the corresponding grantee submission, see ED595455.]