ethanporcaro/eric-fulltext-meta huggingface dataset:
Can&apos;t Inflate Data? Let the Models Unite and Vote: Data-Agnostic Method to Avoid Overfit with Small Data
Author: Shimmei, Machi
Date: 2023-00-00
Type: Speeches/Meeting Papers
Subjects: Artificial Intelligence,Training,Natural Language Processing,Educational Research,Data,Classification,Accuracy,Models,Questioning Techniques,Program Effectiveness

We propose an innovative, effective, and data-agnostic method to train a deep-neural network model with an extremely small training dataset, called VELR (Voting-based Ensemble Learning with Rejection). In educational research and practice, providing valid labels for a sufficient amount of data to be used for supervised learning can be very costly and often impractical. The shortage of training data often results in deep neural networks being overfitting. There are many methods to avoid overfitting such as data augmentation and regularization. Though, data augmentation is considerably data dependent and does not usually work well for natural language processing tasks. Moreover, regularization is often quite task specific and costly. To address this issue, we propose an ensemble of overfitting models with uncertainty-based rejection. We hypothesize that misclassification can be identified by estimating the distribution of the class-posterior probability P(y|x) as a random variable. The proposed VELR method is data independent, and it does not require changes to the model structure or the re-training of the model. Empirical studies demonstrated that VELR achieved classification accuracy of 0.7 with only 200 samples per class on the CIFAR-10 dataset, but 75% of input samples were rejected. VELR was also applied to a question generation task using a BERT language model with only 350 training data points, which resulted in generating questions that are indistinguishable from human-generated questions. The paper concludes that VELR has potential applications to a broad range of real-world problems where misclassification is very costly, which is quite common in the educational domain. [For the complete proceedings, see ED630829.]
Early Prediction of Student Performance in a Health Data Science MOOC
Author: Rohani, Narjes
Date: 2023-00-00
Type: Speeches/Meeting Papers
Subjects: Prediction,Academic Achievement,Health Education,Data Science,MOOCs,At Risk Students,Interaction,Artificial Intelligence,Learning Processes

Massive Open Online Courses (MOOCs) make high-quality learning accessible to students from all over the world. On the other hand, they are known to exhibit low student performance and high dropout rates. Early prediction of student performance in MOOCs can help teachers intervene in time in order to improve learners' future performance. This is particularly important in healthcare courses, given the acute shortages of healthcare staff and the urgent need to train data-literate experts in the healthcare field. In this paper, we analysed a health data science MOOC taken by over 3,000 students. We developed a novel three-step pipeline to predict student performance in the early stages of the course. In the first step, we inferred the transitions between students' low-level actions from their clickstream interactions. In the second step, the transitions were fed into Artificial Neural Network (ANN) that predicted student performance. In the final step, we used two explanation methods to interpret the ANN result. Using this approach, we were able to predict learners' final performance in the course with an AUC ranging from 83% to 91%. We found that students who interacted predominately with lab, project, and discussion materials outperformed students who interacted predominately with lectures and quizzes. We used the DiCE counterfactual method to automatically suggest simple changes to the learning behaviour of low- and moderate-performance students in the course that could potentially improve their performance. Our method can be used by instructors to help identify and support struggling students during the course. [For the complete proceedings, see ED630829.]
Precise Unbiased Estimation in Randomized Experiments Using Auxiliary Observational Data
Author: Gagnon-Bartsch, J. A.
Date: 2019-00-00
Type: Reports - Research
Subjects: Randomized Controlled Trials,Educational Research,Prediction,Algorithms,Statistical Bias,Sampling,Sample Size,Error of Measurement,Causal Models,Statistical Inference,Bayesian Statistics,Experiments

Randomized controlled trials (RCTs) admit unconfounded design-based inference--randomization largely justifies the assumptions underlying statistical effect estimates--but often have limited sample sizes. However, researchers may have access to big observational data on covariates and outcomes from RCT non-participants. For example, data from A/B tests conducted within an educational technology platform exist alongside historical observational data drawn from student logs. We outline a design-based approach to using such observational data for variance reduction in RCTs. First, we use the observational data to train a machine learning algorithm predicting potential outcomes using covariates, and use that algorithm to generate predictions for RCT participants. Then, we use those predictions, perhaps alongside other covariates, to adjust causal effect estimates with a flexible, design-based covariate-adjustment routine. In this way there is no danger of biases from the observational data leaking into the experimental estimates, which are guaranteed to be exactly unbiased regardless of whether the machine learning models are &quot;correct&quot; in any sense or whether the observational samples closely resemble RCT samples. We demonstrate the method in analyzing 33 randomized A/B tests, and show that it decreases standard errors relative to other estimators, sometimes substantially. [This is the online version of an article published in &quot;Journal of Causal Inference.&quot; Additional funding was provided by the U.S. Department of Education's Graduate Assistance in Areas of National Need (GAANN) program.]
Auto-Scoring Student Responses with Images in Mathematics
Author: Baral, Sami
Date: 2023-00-00
Type: Speeches/Meeting Papers
Subjects: Mathematics Instruction,Mathematical Concepts,Problem Solving,Test Format,Mathematics Tests,Natural Language Processing,Artificial Intelligence,Learning Management Systems,Scoring,Computer Assisted Testing,Handwriting,Freehand Drawing,Responses,Computer Software

Teachers often rely on the use of a range of open-ended problems to assess students' understanding of mathematical concepts. Beyond traditional conceptions of student open-ended work, commonly in the form of textual short-answer or essay responses, the use of figures, tables, number lines, graphs, and pictographs are other examples of open-ended work common in mathematics. While recent developments in areas of natural language processing and machine learning have led to automated methods to score student open-ended work, these methods have largely been limited to textual answers. Several computer-based learning systems allow students to take pictures of hand-written work and include such images within their answers to open-ended questions. With that, however, there are few-to-no existing solutions that support the auto-scoring of student hand-written or drawn answers to questions. In this work, we build upon an existing method for auto-scoring textual student answers and explore the use of OpenAI/CLIP, a deep learning embedding method designed to represent both images and text, as well as Optical Character Recognition (OCR) to improve model performance. We evaluate the performance of our method on a dataset of student open-responses that contains both text- and image-based responses, and find a reduction of model error in the presence of images when controlling for other answer-level features. [For the complete proceedings, see ED630829.]
Automated Search for Logistic Knowledge Tracing Models
Author: Pavlik, Philip I., Jr.
Date: 2023-00-00
Type: Speeches/Meeting Papers
Subjects: Students,Models,Regression (Statistics),Alternative Assessment,Data Analysis,Cloze Procedure,Automation,Computer Software,Item Response Theory

This paper presents a tool for creating student models in logistic regression. Creating student models has typically been done by expert selection of the appropriate terms, beginning with models as simple as IRT or AFM but more recently with highly complex models like BestLR. While alternative methods exist to select the appropriate predictors for the regression-based models (e.g., stepwise selection or LASSO), we are unaware of their application to student modeling. Such automatic methods of model creation offer the possibility of better student models with either reduced complexity or better fit, in addition to relieving experts from the burden of searching for better models by hand with possible human error. Our new functions are now part of the preexisting R package LKT. We explain our search methods with two datasets demonstrating the advantages of using the tool with stepwise regression and regularization (LASSO) methods to aid in feature selection. For the stepwise method using BIC, the models are simpler (due to the BIC penalty for parameters) than alternatives like BestLR with little lack of fit. For the LASSO method, the models can be made simpler due to the fitting procedure involving a regularization parameter that penalizes large absolute coefficient values. However, LASSO also offers the possibility of highly complex models with exceptional fit. [For the complete proceedings, see ED630829.]
Effective Evaluation of Online Learning Interventions with Surrogate Measures
Author: Prihar, Ethan
Date: 2023-00-00
Type: Speeches/Meeting Papers
Subjects: Electronic Learning,Intervention,Instructional Effectiveness,Data Collection,Models,Measurement

There is a growing need to empirically evaluate the quality of online instructional interventions at scale. In response, some online learning platforms have begun to implement rapid A/B testing of instructional interventions. In these scenarios, students participate in series of randomized experiments that evaluate problem-level interventions in quick succession, which makes it difficult to discern the effect of any particular intervention on their learning. Therefore, distal measures of learning such as posttests may not provide a clear understanding of which interventions are effective, which can lead to slow adoption of new instructional methods. To help discern the effectiveness of instructional interventions, this work uses data from 26,060 clickstream sequences of students across 31 different online educational experiments exploring 51 different research questions and the students' posttest scores to create and analyze different proximal surrogate measures of learning that can be used at the problem level. Through feature engineering and deep learning approaches, next-problem correctness was determined to be the best surrogate measure. As more data from online educational experiments are collected, model based surrogate measures can be improved, but for now, next-problem correctness is an empirically effective proximal surrogate measure of learning for analyzing rapid problem-level experiments. [For the complete proceedings, see ED630829. Additional funding for this paper was provided by the U.S. Department of Education's Graduate Assistance in Areas of National Need (GAANN).]
Exploring the Effectiveness of Vocabulary Proficiency Diagnosis Using Linguistic Concept and Skill Modeling
Author: Ma, Boxuan
Date: 2023-00-00
Type: Speeches/Meeting Papers
Subjects: Vocabulary Development,Second Language Instruction,Second Language Learning,Language Proficiency,Item Response Theory,Comparative Analysis,Learning Processes,Artificial Intelligence,Task Analysis,Evaluation Methods

Vocabulary proficiency diagnosis plays an important role in the field of language learning, which aims to identify the level of vocabulary knowledge of a learner through his or her learning process periodically, and can be used to provide personalized materials and feedback in language-learning applications. Traditional approaches are widely applied for modeling knowledge in science or mathematics, where skills or knowledge concepts are well-defined and easy to associate with each item. However, only a handful of works focus on defining knowledge concepts and skills using linguistic characteristics for language knowledge proficiency diagnosis. In addressing this, we propose a framework for vocabulary proficiency diagnosis based on neural networks. Specifically, we propose a series of methods based on our framework that uses different linguistic features to define skills and knowledge concepts in the context of the language learning task. Experimental results on a real-world second-language learning dataset demonstrate the effectiveness and interpretability of our framework. We also provide empirical evidence with ablation testing to prove that our knowledge concept and skill definitions are reasonable and critical to the performance of our model. [For the complete proceedings, see ED630829.]
Secondary Students&apos; Metaphors for Learning English
Author: Özcan, Ahmet
Date: 2022-12-00
Type: Reports - Research
Subjects: Middle School Students,Grade 6,Grade 7,Grade 8,English (Second Language),Second Language Instruction,Second Language Learning,Foreign Countries,Student Attitudes,Language Usage,Figurative Language

The aim of this study is to find out the perceptions of secondary students about learning the English language through metaphor analysis. The research was carried out with 432 students studying in the 6th, 7th and 8th grades of 6 primary schools in Yozgat city centre. Qualitative method and survey model were used in the study; typical case sampling was used to form the study group. The data of the study were obtained with the data collection tool prepared by the researcher and finalized after expert opinion. In the data collection tool, students were asked to answer a semi-structured question reflecting their thoughts about learning English. The data obtained from the research were analyzed by content analysis method and descriptive analysis techniques were used to analyze the data. As a result of data analysis, metaphors were examined under five themes: challenging, progressive, beneficial, enjoyable, and intangible. The metaphors show that students generally have positive perceptions about learning English. These metaphors are substantial as they provide an in-depth understanding of students' perceptions and shed light on teachers and curriculum designers. [This paper was published in &quot;Journal of Research in Social Sciences and Language.&quot;]
Can the Paths of Successful Students Help Other Students with Their Course Enrollments?
Author: Wagner, Kerstin
Date: 2023-00-00
Type: Speeches/Meeting Papers
Subjects: College Freshmen,At Risk Students,Dropouts,Dropout Programs,Success,Academic Achievement,Course Selection (Students),Artificial Intelligence,Information Systems,Technology Uses in Education,Decision Support Systems,Low Achievement,Program Effectiveness,Incidence,Enrollment,Predictor Variables,Peer Influence

In this paper, we present an extended evaluation of a course recommender system designed to support students who struggle in the first semesters of their studies and are at risk of dropping out. The system, which was developed in earlier work using a student-centered design and which is based on the explainable k-nearest neighbor algorithm, recommends a set of courses that have been passed by the majority of the student's nearest neighbors who have completed their studies. The present evaluation is based on the data of students from three different study programs. One result is that the recommendations do lower the dropout risk. We also discovered that while the recommended courses differed from those taken by students who dropped out, they matched quite well with courses taken by students who completed the degree program. Although the course recommender system targets primarily students at risk, students doing well could use it. Furthermore, we found that the number of recommended courses for struggling students is less than the number of courses they actually enrolled in. This suggests that the recommendations given indicate a different and hopefully feasible path through the study program for students at risk of dropping out. [For the complete proceedings, see ED630829.]
Clustering to Define Interview Participants for Analyzing Student Feedback: A Case of Legends of Learning
Author: Karimov, Ayaz
Date: 2023-00-00
Type: Speeches/Meeting Papers
Subjects: Student Attitudes,Feedback (Response),Educational Games,Information Retrieval,Pattern Recognition,Data Analysis,Student Motivation,Academic Achievement,Educational Technology,Foreign Countries,Science Education,Cluster Grouping,Algorithms,Grade 5,Grade 6,Grade 7,Middle School Students,Teacher Attitudes

Within the last decade, different educational data mining techniques, particularly quantitative methods such as clustering, and regression analysis are widely used to analyze the data from educational games. In this research, we implemented a quantitative data mining technique (clustering) to further investigate students' feedback. Students played educational games within a week on the educational games platform, Legends of Learning and after a week, we asked them to fulfill the feedback survey about their feelings on the use of this platform. To analyze the collected data from students, firstly, we prepared clusters and selected one prototype student closest to the centroid of each cluster to interview. Interviews were held to explain the clusters more and due to time and resource limitations, we were unable to interview all (N=60) students, thus only the most representative students were interviewed. In addition to the students, we conducted an interview with the teacher as well to get her detailed feedback and observations on the usage of educational games. We also asked students to take an exam before and after the research to see the impact of games on their grades. Our results depict that though educational games can increase students' motivation, they may negatively impact some students' grades. And even though playing games made students feel interested and fun, they would not like to play them on a daily basis. Hence, using educational games for a certain duration such as subject revision weeks may positively influence students' grades and motivation. [For the complete proceedings, see ED630829.]
Is Your Model &quot;MADD&quot;? A Novel Metric to Evaluate Algorithmic Fairness for Predictive Student Models
Author: Verger, Mélina
Date: 2023-00-00
Type: Speeches/Meeting Papers
Subjects: Prediction,Models,Student Behavior,Academic Achievement,Online Courses,Algorithms,Bias,Foreign Countries,STEM Education,Social Studies,College Students

Predictive student models are increasingly used in learning environments due to their ability to enhance educational outcomes and support stakeholders in making informed decisions. However, predictive models can be biased and produce unfair outcomes, leading to potential discrimination against some students and possible harmful long-term implications. This has prompted research on fairness metrics meant to capture and quantify such biases. Nonetheless, so far, existing fairness metrics used in education are predictive performance-oriented, focusing on assessing biased outcomes across groups of students, without considering the behaviors of the models nor the severity of the biases in the outcomes. Therefore, we propose a novel metric, the Model Absolute Density Distance (MADD), to analyze models' discriminatory behaviors independently from their predictive performance. We also provide a complementary visualization-based analysis to enable fine-grained human assessment of how the models discriminate between groups of students. We evaluate our approach on the common task of predicting student success in online courses, using several common predictive classification models on an open educational dataset. We also compare our metric to the only predictive performance-oriented fairness metric developed in education, ABROCA. Results on this dataset show that: (1) fair predictive performance does not guarantee fair models' behaviors and thus fair outcomes; (2) there is no direct relationship between data bias and predictive performance bias nor discriminatory behaviors bias; and (3) trained on the same data, models exhibit different discriminatory behaviors, according to different sensitive features too. We thus recommend using the MADD on models that show satisfying predictive performance, to gain a finer-grained understanding on how they behave and regarding who and to refine models selection and their usage. Altogether, this work contributes to advancing the research on fair student models in education. Source code and data are in open access at https://github.com/melinaverger/MADD. [For the complete proceedings, see ED630829.]
KC-Finder: Automated Knowledge Component Discovery for Programming Problems
Author: Shi, Yang
Date: 2023-00-00
Type: Speeches/Meeting Papers
Subjects: Computer Science Education,Data Analysis,Programming,Coding,Assignments,Teaching Methods,Learning Analytics,Classification,Introductory Courses,Programming Languages,Goodness of Fit,Evaluation Methods,Artificial Intelligence,Models,Decision Making,Undergraduate Students,State Universities,Student Evaluation

Knowledge components (KCs) have many applications. In computing education, knowing the demonstration of specific KCs has been challenging. This paper introduces an entirely data-driven approach for: (1) discovering KCs; and (2) demonstrating KCs, using students' actual code submissions. Our system is based on two expected properties of KCs: (1) generate learning curves following the power law of practice; and (2) are predictive of response correctness. We train a neural architecture (named KC-Finder) that classifies the correctness of student code submissions and captures problem-KC relationships. Our evaluation on data from 351 students in an introductory Java course shows that the learned KCs can generate reasonable learning curves and predict code submission correctness. At the same time, some KCs can be interpreted to identify programming skills. We compare the learning curves described by our model to four baselines, showing that: (1) identifying KCs with naive methods is a difficult task; and (2) our learning curves exhibit a substantially better curve fit. Our work represents a first step in solving the data-driven KC discovery problem in computing education. [For the complete proceedings, see ED630829.]