ethanporcaro/eric-fulltext-meta huggingface dataset:
Housing Instability &amp; Educational Outcomes of San Mateo County Youth
Author: Stanford University, John W. Gardner Center for Youth and Their Communities (JGC)
Date: 2022-04-00
Type: Reports - Research
Subjects: School Districts,Outcomes of Education,Case Studies,Partnerships in Education,Homeless People,Place of Residence,Lunch Programs,Low Income Students,English Language Learners,Hispanic American Students,African American Students,Attendance,Barriers,Youth Programs,Equal Education,Disproportionate Representation,Best Practices,COVID-19,Pandemics,Identification,Race,Ethnicity,Student Characteristics,Kindergarten,Elementary Secondary Education,Counties,Longitudinal Studies

In partnership with the Chan Zuckerberg Initiative and 22 San Mateo County school districts, this report and research brief explore the ways in which local students experience housing instability, and how cross-sector strategies can better identify and serve these students. This study considers a rich set of school administrative data, which allow us to describe youth based on their dwelling status designations reported by the school districts, their free or reduced-price meal participation, and several demographic characteristics. We also examine the relationship between student dwelling status designation and educational outcomes. At a high level, our research finds that over 2,600 adolescents in San Mateo County experienced housing instability over the course of the three school years before the pandemic. Students experiencing housing instability were also: (1) disproportionately Latinx, Black, and/or English language learners; and (2) likely to miss more days of school than their housing stable peers. Based on these findings, we conclude that youth experiencing housing instability face a range of unique challenges, and we recommend a set of concrete actions that state policymakers, local and county leaders, school district leaders, and leaders from youth-serving organizations can take to help ensure positive and equitable outcomes. [This report was written with contributions from Francine Biscocho, Elise Dizon Ross, Kristin Geiser, Amy Gerstein, Leslie Patron, Jayme Pyne, Nicole Tognozzi, and Charlotte Woo. For the report brief, see ED624056.]
Automatic Classification of Learning Objectives Based on Bloom&apos;s Taxonomy
Author: Li, Yuheng
Date: 2022-00-00
Type: Speeches/Meeting Papers
Subjects: Educational Objectives,Taxonomy,Universities,Cognitive Ability,Foreign Countries,Classification,Teaching Methods,Learning Processes,Computational Linguistics,Artificial Intelligence,Course Evaluation

Learning objectives, especially those well defined by applying Bloom's taxonomy for Cognitive Objectives, have been widely recognized as important in various teaching and learning practices. However, many educators have difficulties developing learning objectives appropriate to the levels in Bloom's taxonomy, as they need to consider the progression of learners' skills with learning content as well as dependencies between different learning objectives. To remedy this challenge, we aimed to apply state-of-the-art computational techniques to automate the classification of learning objectives based on Bloom's taxonomy. Specifically, we collected 21,380 learning objectives from 5,558 different courses at an Australian university and manually labeled them according to the six cognitive levels of Bloom's taxonomy. Based on the labeled dataset, we applied five conventional machine learning approaches (i.e., naive Bayes, logistic regression, support vector machine, random forest, and XGBoost) and one deep learning approach based on pre-trained language model BERT to construct classifiers to automatically determine a learning objective's cognitive levels. In particular, we adopted and compared two methods in constructing the classifiers, i.e., constructing multiple binary classifiers (one for each cognitive level in Bloom's taxonomy) and constructing only one multi-class multi-label classifier to simultaneously identify all the corresponding cognitive levels. Through extensive evaluations, we demonstrated that: (i) BERT-based classifiers outperformed the others in all cognitive levels (Cohen's K up to 0.93 and F1 score up to 0.95); (ii) three machine learning models -- support vector machine, random forest, and XGBoost -- delivered performance comparable to the BERT-based classifiers; and (iii) most of the binary BERT-based classifiers (5 out of 6) slightly outperformed the multi-class multi-label BERT-based classifier, suggesting that separating the characterization of different cognitive levels seemed to be a better choice than building only one model to identify all cognitive levels at one time. [For the full proceedings, see ED623995.]
Building a Reinforcement Learning Environment from Limited Data to Optimize Teachable Robot Interventions
Author: Maidment, Tristan
Date: 2022-00-00
Type: Speeches/Meeting Papers
Subjects: Robotics,Cooperative Learning,Artificial Intelligence,Training,Reinforcement,Undergraduate Students,Student Attitudes,Simulation

Working collaboratively in groups can positively impact performance and student engagement. Intelligent social agents can provide a source of personalized support for students, and their benefits likely extend to collaborative settings, but it is difficult to determine how these agents should interact with students. Reinforcement learning (RL) offers an opportunity for adapting the interactions between the social agent and the students to better support collaboration and learning. However, using RL in education with social agents typically involves training using real students. In this work, we train an RL agent in a high-quality simulated environment to learn how to improve students' collaboration. Data was collected during a pilot study with dyads of students who worked together to tutor an intelligent teachable robot. We explore the process of building an environment from the data, training a policy, and the impact of the policy on different students, compared to various baselines. [For the full proceedings, see ED623995.]
Log Mining for Course Recommendation in Limited Information Scenarios
Author: Sanguino, Juan
Date: 2022-00-00
Type: Speeches/Meeting Papers
Subjects: Data Analysis,Learning Analytics,Student Interests,Student Needs,Vignettes,Independent Study,MOOCs,Computer Software,Barriers,Course Content,Evaluation Methods,Comparative Analysis,Student Characteristics,Learning Experience,Academic Persistence,Learning Management Systems,Learning Trajectories

Recommender systems in educational contexts have proven effective to identify learning resources that fit the interests and needs of learners. Their usage has been of special interest in online self-learning scenarios to increase student retention and improve the learning experience. In current recommendation techniques, and in particular, in collaborative filtering recommender systems, the quality of the recommendation is largely based on the explicit or implicit information obtained about the learners. On free massive online learning platforms, however, the information available about learners may be limited and based mostly on logs from website analytics tools such as Google Analytics. In this paper, we address the challenge of recommending meaningful content with limited information from users by using rating estimation strategies from a log system. Our approach posits strategies to mine logs and generates effective ratings through the counting and temporal analysis of sessions. We evaluate different rating penalty strategies and compare the use of per-user and global metrics for rating estimation. The results show that using the average number of lessons viewed per-user is better than using global metrics with a p-value under 0.01 for 4 of our 5 hypotheses, showing statistical significance. Additionally, the results show that functions that penalize the rating to a lesser degree behave better and lead to a better recommendation. [For the full proceedings, see ED623995.]
Mining Assignment Submission Time to Detect At-Risk Students with Peer Information
Author: Wang, Yuancheng
Date: 2022-00-00
Type: Speeches/Meeting Papers
Subjects: College Students,Assignments,Time,Scheduling,At Risk Students,Predictor Variables,Comparative Analysis,Academic Achievement,Prediction,Man Machine Systems,Artificial Intelligence,Peer Relationship

Doing assignments is a very important part of learning. Students' assignment submission time provides valuable information on study attitudes and habits which strongly correlate with academic performance. However, the number of assignments and their submission deadlines vary among university courses, making it hard to use assignment submission time as a feature to predict academic performance. In this paper, we propose a new method called Relative Assignment Submission Time (RAST) which uses the assignment submission information of peer students to improve the correlation with course grades. Experiments on real-life data of 20 courses show that RAST has a high correlation with students' academic performance. We also build a machine learning model using RAST as a feature to detect students who would suffer from poor grades. Our method outperforms the traditional method by up to 61% on f1-score. We believe that our proposed method can help other studies on assignment submission time to improve the prediction accuracy on academic performance and detecting at-risk students. [For the full proceedings, see ED623995.]
Graduation and Dropout Information Handbook. Updated
Author: Kansas State Department of Education
Date: 2022-08-00
Type: Reports - Descriptive
Subjects: Graduation Rate,Dropout Rate,Home Schooling,Students with Disabilities,High School Graduates,Dropouts,Transfer Students,Public Schools,Private Schools,Withdrawal (Education),Virtual Schools

The Kansas Graduate and Dropout Handbook provides a comprehensive overview of how the Kansas State Department of Education (KSDE) calculates and reports the Adjusted Cohort Graduation and Annual Dropout Rates. Sections include: (1) Introduction; (2) Terminology; (3) Data Sources; (4) Graduation Calculations; (5) Dropout Calculation; and (6) Dropout Graduation Summary Report (DGSR). [For the 2021-2022 handbook, see ED615185.]
Detecting SMART Model Cognitive Operations in Mathematical Problem-Solving Process
Author: Zhang, Jiayi
Date: 2022-00-00
Type: Speeches/Meeting Papers
Subjects: Mathematics Instruction,Teaching Methods,Problem Solving,Metacognition,Learning Strategies,Guidelines,Protocol Analysis,Models,Learning Analytics,Integrated Learning Systems,Scaffolding (Teaching Technique),Peer Relationship,Measurement,Middle School Students

Self-regulated learning (SRL) is a critical component of mathematics problem solving. Students skilled in SRL are more likely to effectively set goals, search for information, and direct their attention and cognitive process so that they align their efforts with their objectives. An influential framework for SRL, the SMART model, proposes that five cognitive operations (i.e., searching, monitoring, assembling, rehearsing, and translating) play a key role in SRL. However, these categories encompass a wide range of behaviors, making measurement challenging -- often involving observing individual students and recording their think-aloud activities or asking students to complete labor-intensive tagging activities as they work. In the current study, we develop machine-learned indicators of SMART operations, in order to achieve better scalability than other measurement approaches. We analyzed student's textual responses and interaction data collected from a mathematical learning platform where students are asked to thoroughly explain their solutions and are scaffolded in communicating their problem-solving process to their peers and teachers. We built detectors of four indicators of SMART operations (namely, assembling and translating operations). Our detectors are found to be reliable and generalizable, with AUC ROCs ranging from 0.76-0.89. When applied to the full test set, the detectors are robust against algorithmic bias, performing well across different student populations. [For the full proceedings, see ED623995.]
Evaluating the Explainers: Black-Box Explainable Machine Learning for Student Success Prediction in MOOCs
Author: Swamy, Vinitra
Date: 2022-00-00
Type: Speeches/Meeting Papers
Subjects: Artificial Intelligence,Academic Achievement,Grade Prediction,MOOCs,Teaching Methods,Models

Neural networks are ubiquitous in applied machine learning for education. Their pervasive success in predictive performance comes alongside a severe weakness, the lack of explainability of their decisions, especially relevant in humancentric fields. We implement five state-of-the-art methodologies for explaining black-box machine learning models (LIME, PermutationSHAP, KernelSHAP, DiCE, CEM) and examine the strengths of each approach on the downstream task of student performance prediction for five massive open online courses. Our experiments demonstrate that the families of explainers do not agree with each other on feature importance for the same Bidirectional LSTM models with the same representative set of students. We use Principal Component Analysis, Jensen-Shannon distance, and Spearman's rank-order correlation to quantitatively cross-examine explanations across methods and courses. Furthermore, we validate explainer performance across curriculum-based prerequisite relationships. Our results come to the concerning conclusion that the choice of explainer is an important decision and is in fact paramount to the interpretation of the predictive results, even more so than the course the model is trained on. Source code and models are released at http://github.com/epfl-ml4ed/evaluating-explainers. [For the full proceedings, see ED623995.]
Individual Fairness Evaluation for Automated Essay Scoring System
Author: Doewes, Afrizal
Date: 2022-00-00
Type: Speeches/Meeting Papers
Subjects: Scoring,Essays,Writing Evaluation,Comparative Analysis,Computer Assisted Testing,Ethics,Student Evaluation,Computational Linguistics,Text Structure,Learning Processes,Phrase Structure,Prediction,Models,Word Frequency

In Automated Essay Scoring (AES) systems, many previous works have studied group fairness using the demographic features of essay writers. However, individual fairness also plays an important role in fair evaluation and has not been yet explored. Initialized by Dwork et al., the fundamental concept of individual fairness is &quot;similar people should get similar treatment&quot;. In the context of AES, individual fairness means that &quot;similar essays should be treated similarly&quot;. In this work, we propose a methodology to measure individual fairness in AES. The similarity of essays can be computed using the distance of the text representation of essays. We compare several text representations of essays, from the classical text features, such as BOW and TFIDF, to the more recent deep-learning-based features, such as Sentence-BERT and LASER. We also show their performance against paraphrased essays to understand if they can maintain the ranking of similarities between the original and the paraphrased essays. Finally, we demonstrate how to evaluate the performance of automated scoring systems models with regard to individual fairness by counting the number of pairs of essays that satisfy the individual fairness equation and by observing the correlation of score difference with the distance of essays. Our analysis suggests that the Sentence-BERT, as the text representation of the essays, and Gradient Boosting, as the score prediction model, provide better results based on the proposed individual fairness evaluation methodology. [For the full proceedings, see ED623995.]
Investigating Growth of Representational Competencies by Knowledge-Component Model
Author: Rho, Jihyun
Date: 2022-00-00
Type: Speeches/Meeting Papers
Subjects: Learning Processes,Models,Introductory Courses,Engineering Education,Teaching Methods,Visual Aids,Course Content,Competence,Abstract Reasoning,Knowledge Level,Artificial Intelligence,Intelligent Tutoring Systems,Learning Trajectories,Learning Analytics

Instruction in many STEM domains heavily relies on visual representations, such as graphs, figures, and diagrams. However, students who lack representational competencies do not benefit from these visual representations. Therefore, students must learn not only content knowledge but also representational competencies. Further, as learning progresses, knowledge likely becomes more abstract, so that content knowledge may no longer be tied to a specific representation. This raises the question of whether students integrate representational competencies with content knowledge as learning progresses. The present study addresses this question by building knowledge-component models using log data collected from two studies in an introductory electrical engineering course. We compared knowledge-component models that separate representational competencies from content knowledge to knowledge-component models that integrate representational competencies with content knowledge. Our results show that as learning progressed, integrated knowledge-component models had better model fit. This finding indicates that over time, students' representational competencies become gradually integrated into content knowledge. Further, this suggests that different knowledge-component models might be needed at different times during a learning progression. [For the full proceedings, see ED623995.]
Investigating Multimodal Predictors of Peer Satisfaction for Collaborative Coding in Middle School
Author: Ma, Yingbo
Date: 2022-00-00
Type: Speeches/Meeting Papers
Subjects: Middle School Students,Cooperative Learning,Prediction,Peer Relationship,Scores,Coding,Computer Science Education,Student Attitudes,Task Analysis,Semantics,Dialogs (Language),Discourse Analysis,Eye Movements,Acoustics,Intonation,Suprasegmentals,Nonverbal Communication,Problem Solving,Programming,Programming Languages,Grade 7

Collaborative learning is a complex process during which two or more learners exchange opinions, construct shared knowledge, and solve problems together. While engaging in this interactive process, learners' satisfaction toward their partners plays a crucial role in defining the success of the collaboration. If intelligent systems could predict peer satisfaction early during collaboration, they could intervene with adaptive support. However, while extensive studies have associated peer satisfaction with factors such as social presence, communication, and trustworthiness, there is no research on automatically predicting learners' satisfaction toward their partners. To fill this gap, this paper investigates the automatic prediction of peer satisfaction by analyzing 44 middle school learners' interactions during collaborative coding tasks. We extracted three types of features from dialogues: (1) linguistic features indicating semantics; (2) acoustic-prosodic features including energy and pitch; and (3) visual features including eye gaze, head pose, facial behaviors, and body pose. We then trained several regression models to predict the peer satisfaction scores that learners received from their partners. The results revealed that head position and body location were significant indicators of peer satisfaction: lower head and body distances between partners were associated with more positive peer satisfaction. This work is the first to investigate the multimodal prediction of peer satisfaction during collaborative problem solving, and represents a step toward the development of real-time intelligent systems that support collaborative learning. [For the full proceedings, see ED623995.]
Item Response Theory-Based Gaming Detection
Author: Huang, Yun
Date: 2022-00-00
Type: Speeches/Meeting Papers
Subjects: Item Response Theory,Learner Engagement,Student Behavior,Student Motivation,Academic Achievement,Middle School Students,High School Students,Algebra,Learning Strategies

Gaming the system, a behavior in which learners exploit a system's properties to make progress while avoiding learning, has frequently been shown to be associated with lower learning. However, when we applied a previously validated gaming detector across conditions in experiments with an algebra tutor, the detected gaming was not associated with learning, challenging its construct validity. Our iterative exploratory data analysis suggested that some contextual factors that varied across and within conditions might contribute to this lack of association. We present a latent variable model, &quot;item response theory-based gaming detection&quot; (IRT-GD), that accounts for contextual factors and estimates latent gaming tendencies as the degree of deviation from normative behaviors across contexts. Item response theory models, widely used in knowledge assessment, account for item difficulty in estimating latent student abilities: students are estimated as having higher ability when they can get harder items correct than when they only get easier items correct. Similarly, IRT-GD accounts for contextual factors in estimating latent gaming tendencies: students are estimated as having a higher gaming tendency when they game in less commonly gamed contexts than when they only game in more commonly gamed contexts. IRT-GD outperformed the original detector on three datasets in terms of the association with learning. IRT-GD also more accurately revealed intervention effects on gaming and revealed a correlation between gaming and perceived competence in math. Our approach is not only useful for others wanting to apply a gaming assessment in their context but is also generally applicable in creating robust behavioral measures. [For the full proceedings, see ED623995.]
Evaluating Gaming Detector Model Robustness over Time
Author: Levin, Nathan
Date: 2022-00-00
Type: Speeches/Meeting Papers
Subjects: Intelligent Tutoring Systems,Artificial Intelligence,Models,Cheating,Student Behavior,Deception,Prediction,Reliability,Algorithms

Research into &quot;gaming the system&quot; behavior in intelligent tutoring systems (ITS) has been around for almost two decades, and detection has been developed for many ITSs. Machine learning models can detect this behavior in both real-time and in historical data. However, intelligent tutoring system designs often change over time, in terms of the design of the student interface, assessment models, and data collection log schemas. Can gaming detectors still be trusted, a decade or more after they are developed? In this research, we evaluate the robustness/degradation of gaming detectors when trained on older data logs and evaluated on current data logs. We demonstrate that some machine learning models developed using past data are still able to predict gaming behavior from student data collected 16 years later, but that there is considerable variance in how well different algorithms perform over time. We demonstrate that a classic decision tree algorithm maintained its performance while more contemporary algorithms struggled to transfer to new data, even though they exhibited better performance on unseen students in both New and Old data sets by themselves. Examining the feature importance values provides some explanation for the differences in performance between models, and offers some insight into how we might safeguard against detector rot over time. [For the full proceedings, see ED623995.]