DOI: 10.3886/e202181v1-159901
title: Online Credit Recovery Study: Effects on High School Students' Proximal and Distal Outcomes

description: The American Institutes for Research conducted a multisite randomized study that tested an online learning model for
credit recovery at 24 high schools in Los Angeles, California in 2018 and 2019.
The study focused on first-year high school students who failed Algebra 1 or
English 9 (their ninth-grade English course) and retook the course during the
summer before their second year of high school. Within each participating
school, we used a lottery to determine whether each student was placed in either
the school’s typical teacher-directed class (business-as-usual control condition) or a
class that used an online learning model (treatment condition). For the online
learning model, an online provider supplied the main course content, and the
school provided a subject-appropriate, credentialed in-class teacher who could
supplement the digital content with additional instruction.<br><br>The study compared outcomes of students assigned to the treatment condition to outcomes of students assigned to the
control condition. Analyses focused both on proximal outcomes (ex: student course
experiences, content knowledge, and credit recovery rates) and distal outcomes
(ex: on-time graduation and cumulative credits earned by the end of the 4th year of
high school). We estimated average treatment effects for the intent-to-treat
sample using regression models that control for student characteristics and
randomization blocks. We conducted separate analyses for students who failed
Algebra 1 and students who failed at least one semester of their English 9
course.<br><br>This ICPSR data deposit includes our final analytical dataset and three supplemental files. Data come from three sources: (1) extant district data on
student information and academic outcomes, (2) end-of-course surveys of
students’ and teachers’ experiences, and (3) end-of-course test of students’
content knowledge. Data fields include:<br>Sample information: term, school (anonymized), teacher (anonymized), course, randomization block, student cohort, treatment statusDemographics: sex, race/ethnicity, National School Lunch Program status, inclusion in the Gifted/Talented program, Special Education status, and English language learner statusPre-treatment information (treatment group only): 9th grade GPA, 9th grade attendance rate, number of 9th grade courses failed, 8th grade test scoresOnline course engagement information: percentage of online course completed, average score on online activities, minutes spent in online platformStudent survey data: responses a survey administered at the end of the course for treatment and control students. Questions cover degree of student engagement with the course, perceptions of teacher support and course difficulty, and clarity of course expectations.End-of-course test data: answers and scores on an end-of-course assessment administered to treatment and control students to evaluate content knowledge (Algebra 1 or English 9). The test did not count towards the final course grade and included 17-20 multiple choice questions.Academic outcomes: grade in credit recovery course, credits attempted/earned in each year of high school, GPA in each year of high school, credits/GPA in math and ELA in each year of high school, indicator for on-time high school graduation, 10th grade PSAT scoresTeacher survey and logs: teacher-reported logs on the use of different instructional activities and responses to surveys about course pacing, content, goals, and degree of student support
description: Response Rates: The
response rate for the survey was 66% for Algebra 1 (60% for online classes and
73% for teacher-directed classes) and 59% for English 9 (57% for online classes
and 61% for teacher-directed classes). The response rate for the test was 66%
for Algebra 1 (59% for online classes and 72% for teacher-directed classes) and
59% for English 9 (57% for online classes and 60% for teacher-directed
classes). For the PSAT outcome, Grade 10 students in the district are supposed
to take the PSAT. Missing data for the PSAT measures could be due to non-response
or students in the ITT sample leaving the district prior to the PSAT testing
date. For the on-time high school graduation analysis, all students were coded
as graduating from the district or not based on the district’s administrative
data.
description: Students in 24 Los Angeles high schools who entered 9th grade in the 2017-18 school year (Cohort 1) or the 2018-19 school year (Cohort 2), failed their Algebra 1 and/or at least one semester of their English 9 course, and enrolled in a credit recovery course the summer between their 9th and 10th grade year.