ethanporcaro/eric-fulltext-meta huggingface dataset:
Estimating the Causal Effects of Khan Academy MAP Accelerator across Demographic Subgroups
Author: Grimaldi, Phillip
Date: 2022-00-00
Type: Speeches/Meeting Papers
Subjects: Educational Technology,Mastery Learning,Learning Management Systems,Middle School Students,Elementary School Students,School Districts,Mathematics Instruction,Standardized Tests,Mathematics Tests,Instructional Effectiveness,Disadvantaged,Learning Analytics,Attribution Theory,Comparative Analysis,Educational Benefits,Race,Gender Differences,English Language Learners,Outcomes of Education,Equal Education,Achievement Tests,Inferences,Hispanic American Students

As educational technology platforms become more and more commonplace in education, it is critical that these systems work well across a diverse range of student sub-groups. In this study, we estimated the effectiveness of MAP Accelerator; a large-scale, personalized, web-based, mathematics mastery learning platform. Our analysis placed a particular focus on students from historically under-resourced groups. Our sample comprised 181K students in grades 3-8 from 99 school districts across the United States, 52% of whom attended schools where the majority of the students are eligible for free or reduced price lunch (a proxy for high-poverty schools). Using a combination of system logs/platform telemetry data, standardized assessments, and publicly available school/district data, we estimated the causal effect of a year-long supplemental math intervention on students' standardized mathematics growth outcomes via a quasi-experimental design with controls and a modification of the difference-in-differences framework. Students who used the platform as recommended (30+ min/wk) during the (COVID-19 disrupted) 2020-2021 school year had math growth scores 0.26 standard deviations higher on average than similar students who used the platform for &lt;15 min/week. Further, positive benefits of the platform were observed across all genders, race/ethnic groups, and school poverty levels, but were not as large for districts with greater than 20% English language learners. Further analysis revealed that these students were predominantly Hispanic, and tended to make less progress on fewer skills than their peers given the same amount of practice time. We discuss the implications of these findings, and potential avenues to ensure more equitable outcomes for these students in the future. [For the full proceedings, see ED623995.]
From {Solution} Synthesis to {Student Attempt} Synthesis for Block-Based Visual Programming Tasks
Author: Singla, Adish
Date: 2022-00-00
Type: Speeches/Meeting Papers
Subjects: Programming,Computer Science Education,Task Analysis,Introductory Courses,Intelligent Tutoring Systems,Inferences,Misconceptions,Prediction,Specialists,Benchmarking,Performance Based Assessment,Artificial Intelligence,Comparative Analysis,Student Evaluation,Student Behavior

Block-based visual programming environments are increasingly used to introduce computing concepts to beginners. Given that programming tasks are open-ended and conceptual, novice students often struggle when learning in these environments. AI-driven programming tutors hold great promise in automatically assisting struggling students, and need several components to realize this potential. We investigate the crucial component of student modeling, in particular, the ability to automatically infer students' misconceptions for predicting (synthesizing) their behavior. We introduce a novel benchmark, StudentSyn, centered around the following challenge: For a given student, synthesize the student's attempt on a new target task after observing the student's attempt on a fixed reference task. This challenge is akin to that of program synthesis; however, instead of synthesizing a {&quot;solution&quot;} (i.e., program an expert would write), the goal here is to synthesize a {&quot;student attempt&quot;} (i.e., program that a given student would write). We first show that human experts (TutorSS) can achieve high performance on the benchmark, whereas simple baselines perform poorly. Then, we develop two neuro/symbolic techniques (NeurSS and SymSS) in a quest to close this gap with TutorSS. [For the full proceedings, see ED623995.]
Mining Artificially Generated Data to Estimate Competency
Author: Robson, Robby
Date: 2022-00-00
Type: Speeches/Meeting Papers
Subjects: Experiential Learning,Mathematical Models,Vignettes,Decision Making,Profiles,Learning Management Systems,Artificial Intelligence,Competence,Guidelines,Teamwork,Trainees

The context for this paper is the &quot;Synthetic Training Environment Experiential Learning -- Readiness&quot; (STEEL-R) project [1], which aims to estimate individual and team competence using data collected from synthetic, semi-synthetic, and live scenario-based training exercises. In STEEL-R, the &quot;Generalized Intelligent Framework for Tutoring&quot; (GIFT) orchestrates scenario sessions and reports data as experience API (xAPI)statements. These statements are translated into assertions about individual and team competencies by the &quot;Competency and Skills System&quot; (CaSS). Mathematical models use these assertions to estimate the competency states of trainees. This information is displayed in a dashboard that enables users to explore progression over time and informs decisions concerning advancement to the next training phase and which skills to address. To test, tune, and demo STEEL-R, more data was needed than was available from real-world training exercises. Since the raw data used to estimate competencies are captured in xAPI statements, a component called DATASIM was added. DATASIM simulated training sessions by generating xAPI statements that conformed to a STEEL-R &quot;xAPI Profile.&quot; This facilitated testing of STEEL-R and was used to create a demo that highlighted the ability to map data from multiple training systems to a single competency framework and to generate a display that team leaders can use to personalize and optimize training across multiple training modalities. This paper gives an overview of STEEL-R, its architecture, and the features that enabled the use of artificial data. The paper explains how xAPI statements are converted to assertions and how these are used to estimate trainee competency. This is followed by a section on xAPI Profiles and on the xAPI Profile used in STEEL-R. The paper then discusses how artificial data were generated and the challenges of modeling longitudinal development and team in these data. The paper ends with a section on future research. [For the full proceedings, see ED623995.]
Simulating Policy Changes in Prerequisite-Free Curricula: A Supervised Data-Driven Approach
Author: Baucks, Frederik
Date: 2022-00-00
Type: Speeches/Meeting Papers
Subjects: Curriculum Development,Educational Change,Educational Policy,Prerequisites,Foreign Countries,Graphs,Time to Degree,Prediction,College Students,Computer Science Education

Curriculum research is an important tool for understanding complex processes within a degree program. In particular, stochastic graphical models and simulations on related curriculum graphs have been used to make predictions about dropout rates, grades, and degree completion time. There exists, however, little research on changes in the curriculum and the evaluation of their impact. The available evaluation methods of curriculum changes assume pre-existing strict curriculum graphs in the form of directed acyclic graphs. These allow for a straightforward model-oriented probabilistic or graph topological investigation of curricula. But the existence of such graphs cannot generally be assumed. We present a novel generalizing approach in which a curriculum graph is constructed based on data, using measurable student flow. By applying a discrete event simulation, we investigate the impact of policy changes on the curriculum and evaluate our approach on a sample data set from a German university. Our method is able to create a comparably effective and individually verifiable simulation without requiring a curriculum graph. It can thus be extended to prerequisite-free curricula, making it feasible to evaluate changes to flexible curricula. [For the full proceedings, see ED623995.]
Modeling One-on-One Online Tutoring Discourse Using an Accountable Talk Framework
Author: Balyan, Renu
Date: 2022-00-00
Type: Speeches/Meeting Papers
Subjects: Tutoring,Guidelines,Mathematics Instruction,Computer Assisted Instruction,Teaching Methods,Classroom Communication,Learning Processes,Thinking Skills,Models,Classification,Discourse Analysis,Comparative Analysis,Computer Mediated Communication,Teacher Student Relationship,High School Students,Mathematics Tests,Academic Failure,Algebra,Videoconferencing

The National Council of Teachers of Mathematics (NCTM) has been emphasizing the importance of teachers' pedagogical communication as part of mathematical teaching and learning for decades. Specifically, NCTM has provided guidance on how teachers can foster mathematical communication that positively impacts student learning. A teacher may have different academic goals towards what needs to be achieved in a classroom, which require a variety of discourse-based tools that allow students to engage fully in mathematical thinking and reasoning. Accountable or academically productive talk is one such approach for classroom discourse that may ensure that the discussions are coherent, purposeful and productive. This paper discusses the use of a transformer model for classifying classroom talk moves based on the accountable talk framework. We investigate the extent to which the classroom Accountable Talk framework can be successfully applied to one-on-one online mathematics tutoring environments. We further propose a framework adapted from Accountable Talk, but more specifically aligned to one-on-one online tutoring. The model performance for the proposed framework is evaluated and compared with a small sample of expert coding. The results obtained from the proposed framework for one-on-one tutoring are promising and improve classification performance of the talk moves for our dataset. [For the full proceedings, see ED623995.]
Examining Inter-Rater Reliability of Evaluators Judging Teacher Performance: Proposing an Alternative to Cohen&apos;s Kappa. CEME Technical Report. CEMETR-2021-06
Author: Lambert, Richard G.
Date: 2021-06-00
Type: Reports - Research
Subjects: Interrater Reliability,Teacher Evaluation,Test Validity,Evaluation Methods,Teacher Effectiveness

The validity of the Kappa coefficient of chance-corrected agreement has been questioned when the prevalence of specific rating scale categories is low and agreement between raters is high. The researchers proposed the Lambda Coefficient of Rater-Mediated Agreement as an alternative to Kappa to address these concerns. Lambda corrects for chance agreement based on specific assumptions about raters and the rater-mediated assessment process including rater-specific tendencies for strict or lenient ratings. Actual ratings of teacher profiles from an inter-rater reliability exercise confirmed the shortcomings of Kappa. The rater data also demonstrated the robustness of Lambda-1, Lambda-2, Gwet's AC1, and Gwet's AC2 to the data conditions that are problematic for Kappa. All four alternative chance-corrected agreement coefficients showed less variability across the 45 raters than Kappa. However, AC-2 was undetermined for 39 of the 45 raters. Simulation data demonstrated the robustness of the Lambda Coefficient of Rater-Mediated Agreement to the data conditions that are problematic for Kappa. [This paper was presented to the virtual Annual Meeting of the National Council on Measurement in Education, June, 2021.]
Toward Better Grade Prediction via A2GP -- An Academic Achievement Inspired Predictive Model
Author: Qiu, Wei
Date: 2022-00-00
Type: Speeches/Meeting Papers
Subjects: Academic Achievement,Prediction,Grades (Scholastic),Models,At Risk Students,Identification,Undergraduate Students

Predicting student performance in an academic institution is important for detecting at-risk students and administering early-intervention strategies. We propose a new grade prediction model that considers three factors: temporal dynamics of prior courses across previous semesters, short-term performance consistency, and relative performance against peers. The proposed architecture comprises modules that incorporate the attention mechanism, a new short-term gated long short-term memory network, and a graph convolutional network to address limitations of existing works that fail to consider the above factors jointly. A weighted fusion layer is used to fuse learned representations of the above three modules--course importance, performance consistency, and relative performance. The aggregated representations are then used for grade prediction which, in turn, is used to classify at-risk students. Experiment results using three datasets obtained from over twenty thousand students across seventeen undergraduate courses show that the proposed model achieves low prediction errors and high F1 scores compared to existing models that predict grades and thereafter identifies at-risk students via a pre-defined threshold. [For the full proceedings, see ED623995.]
ISBE Strategic Plan: 2020-2021 Annual Status Report
Author: Illinois State Board of Education (ISBE)
Date: 2021-06-00
Type: Reports - Descriptive
Subjects: State Boards of Education,Strategic Planning,Equal Education,Educational Opportunities,Educational Objectives,Public Schools,Kindergarten,Elementary Secondary Education,COVID-19,Pandemics,School Community Relationship,Stakeholders,Barriers,Educational Cooperation,Partnerships in Education,Educational Environment,Teacher Effectiveness,Academic Achievement,Achievement Gains,Teaching Methods

The Illinois State Board of Education crafted their Strategic Plan as a roadmap to equity, defining the steps their agency will take to improve and increase opportunities and supports for Illinois students, especially those in the greatest need. They adopted the plan in November 2020, nine months into a global pandemic that continues to shape the world today. Despite the challenges caused by the pandemic, the agency, in partnership with educators, principals, administrators, parents, advocates, and students, accomplished a lot. The Strategic Plan also helped the agency maintain a laser-like focus on their goals as they addressed the impact of the pandemic. This report describes the agency's collective achievements that brought them closer than ever to reaching their goals in the areas of student learning, learning conditions, and elevating educators. [For &quot;Illinois State Board of Education 2020-2023 Strategic Plan,&quot; see ED624118.]
Where California High School Students Attend College
Author: Kurlaender, Michal
Date: 2018-12-00
Type: Reports - Research
Subjects: High School Students,Public Schools,College Attendance,Enrollment,Counties,Institutional Characteristics,Public Colleges,Private Colleges,Two Year Colleges,Geographic Location,Gender Differences,Socioeconomic Status,Disadvantaged Youth,Racial Differences,Ethnicity,Grade 11,Out of State Students

For as much as we know about the economic benefits of a college degree, California policymakers and educators have little information about the college destinations of high school graduates. To fill this information gap, we assembled a unique data set of three recent cohorts of public high school students matched with college enrollment data from the National Student Clearinghouse. This report, a product of a partnership with the California Department of Education, details where California public high school students attend college and how college attendance and destinations vary by county. Among California public high school students, 63% enroll in college following high school graduation; 26% enroll in a four-year college; and 37% enroll in a two-year college. Among California public high school students who enroll in college: 28% enroll in public four-year campuses in-state (18% at CSU, 10% at UC); 57% enroll in public two-year in-state community colleges; 4% enroll in private in-state colleges, predominately four-year institutions; and 11% enroll in out-of-state institutions. Only 7% of all high school students (or 11% of all students who enroll in college) leave the state to attend college after high school; these students reach every state in the nation. Substantial disparities in college enrollment exist by student subgroups: Female students are more likely to go to college, and more likely to attend four-year colleges, than male students. Students from socioeconomically disadvantaged backgrounds enroll in college at lower rates than their more advantaged peers. College enrollment, particularly by higher education sector, varies by race/ethnicity. [For the Appendix, see ED624117.]
Where California High School Students Attend College. Appendix
Author: Stanford University, Policy Analysis for California Education (PACE)
Date: 2018-12-00
Type: Reports - Research
Subjects: High School Students,Public Schools,College Attendance,Enrollment,Counties,Institutional Characteristics,Public Colleges,Private Colleges,Two Year Colleges,Geographic Location,Gender Differences,Socioeconomic Status,Disadvantaged Youth,Racial Differences,Ethnicity,Grade 11,Out of State Students

This is the appendix for &quot;Where California High School Students Attend College.&quot; The full report provides information on first-time college enrollment for the census of California public high school 11th grade students expected to graduate high school in the spring of 2014, 2015, and 2016, respectively. This Appendix provides tables of college enrollment statistics by institution type and by student subgroup for each of the 58 counties in California. [For the full report, see ED624116.]
Illinois State Board of Education 2020-2023 Strategic Plan
Author: Illinois State Board of Education (ISBE)
Date: 2020-00-00
Type: Reports - Descriptive
Subjects: State Boards of Education,Strategic Planning,Public Schools,Equal Education,School Safety,COVID-19,Pandemics,Teacher Effectiveness,Educational Quality,Educational Objectives,Educational Opportunities,Kindergarten,Elementary Secondary Education,School Community Relationship,Educational Cooperation,Stakeholders,Academic Achievement,Achievement Gains

This roadmap for Illinois public schools through school year 2022-23 defines the steps the Illinois State Board of Education (ISBE) will take to provide each and every student in the state with safe and engaging schools, excellent teachers, and equitable learning opportunities. This plan reflects both the realities and uncertainties are faced today due to the COVID-19 pandemic. The plan is built around three goals, which focus on student learning, learning conditions, and elevating educators. These goals are held together by four principles that guide the work at ISBE -- equity, quality, collaboration, and community -- and the agency's responsibility to tirelessly pursue educational equity for all of their students in all of their classrooms, schools, and districts.
Bridging Success: An Evaluation of a Program for Foster Care Alumni
Author: Quintana, Erica
Date: 2022-04-00
Type: Reports - Research
Subjects: Foster Care,Transitional Programs,College Students,Youth,Academic Aspiration,Program Effectiveness,School Holding Power,Graduation Rate,Student Recruitment,Program Evaluation

Arizona State University's (ASU) Bridging Success program, launched in 2015, is a campus-based support program for students with a background in foster care, also known as Foster Care Alumni (FCA). The program is housed within ASU's School of Social Work, which is part of the Watts College of Public Service and Community Solutions. The program serves youth in foster care who are considering post-secondary education and all ASU students who self-identify with a foster care background. Based on evaluation of the data for the first four cohorts (2015-2018), the program is having a positive impact on the lives and educational experiences of Foster Care Alumni. Important progress is being made to exceed recruitment goals, retain Foster Care Alumni students in higher education, and ultimately help them graduate at rates higher than national averages. Morrison Institute produced an initial assessment that provides baseline programmatic data for continued tracking over time and qualitative feedback from students who have been involved with the program as to the usefulness of different program components.
Sparse Factor Autoencoders for Item Response Theory
Author: PaaBen, Benjamin
Date: 2022-00-00
Type: Speeches/Meeting Papers
Subjects: Item Response Theory,Test Items,Item Analysis,Inferences,Learning Analytics,Computer Software,Accuracy,Correlation,Factor Analysis,Knowledge Level,Comparative Analysis,Multiple Choice Tests,Mathematics Tests,Vocational Education,Chemistry,Mathematics Skills

Item response theory (IRT) is a popular method to infer student abilities and item difficulties from observed test responses. However, IRT struggles with two challenges: How to map items to skills if multiple skills are present? And how to infer the ability of new students that have not been part of the training data? Inspired by recent advances in variational autoencoders for IRT, we propose a novel method to tackle both challenges: The Sparse Factor Autoencoder (SparFAE). SparFAE maps from test responses to abilities via a linear operator and from abilities to test responses via an IRT model. All parameters of the model offer an interpretation and can be learned in an efficient manner. In experiments on synthetic and real data, we show that SparFAE is similar in accuracy to other autoencoder approaches while being faster to learn and more accurate in recovering item-skill associations. [For the full proceedings, see ED623995.]
No Meaning Left Unlearned: Predicting Learners&apos; Knowledge of Atypical Meanings of Words from Vocabulary Tests for Their Typical Meanings
Author: Ehara, Yo
Date: 2022-00-00
Type: Speeches/Meeting Papers
Subjects: Language Tests,Vocabulary Development,Second Language Learning,Second Language Instruction,Artificial Intelligence,Definitions,Models,Prediction,Task Analysis,Classification,Item Response Theory,Semantics,Test Items,Factor Analysis,English (Second Language),Transfer of Training,Programming Languages,Computational Linguistics

Language learners are underserved if there are unlearned meanings of a word that they think they have already learned. For example, &quot;circle&quot; as a noun is well known, whereas its use as a verb is not. For artificial-intelligence-based support systems for learning vocabulary, assessing each learner's knowledge of such atypical but common meanings of words is desirable. However, most vocabulary tests only test the typical meanings of words, and the texts used in the test questions are too short to apply readability formulae. We tackle this problem by proposing a novel dataset and a flexible model. First, we constructed a reliable vocabulary test in which learners answered questions regarding typical and atypical meanings of words. Second, we proposed a simple but powerful method for applying flexible and context-aware masked language models (MLMs) to learners' answers in the abovementioned vocabulary test results. This is a personalized prediction task, in which the results vary among learners for the same test question. By introducing special tokens that represent each learner, our method can reduce the personalized prediction task to a simple sequence classification task in which MLMs are applicable. In the evaluation, item response theory (IRT)-based methods, which cannot leverage the semantics of test questions, were used as baselines. The experimental results show that our method consistently and significantly outperformed the IRT-based baselines. Moreover, our method is highly interpretable because one can obtain the learners' language abilities from the first principal component scores of the token embeddings representing each learner. [For the full proceedings, see ED623995.]