Addressing Flawed Research in Developmental Education.

During the past 5 years, community college developmental education has been the subject of considerable debate. Research centers such as MDRC and the Community College Research Center (CCRC) have conducted many studies on all aspects of developmental education. Policy analysis organizations such as the Education Commission of the States, the National Governors Association, and Complete College America have published a number of reports on remedial courses. Foundations, such as the Lumina Foundation, the Bill and Melinda Gates Foundation, the Rockefeller Foundation, the Carnegie Foundation, and the Kresge Foundation, have also funded a variety of demonstration and research projects.The result has been a plethora ofliterature and research marked by varying qualities of methodology and data analysis accompanied by sometimes conflicting and sometimes consistent conclusions. This literature has been used to produce countless policy recommendations for states and institutions. These policy recommendations are also of varying quality. Some are based on reasonable interpretations of the available data, some represent a misunderstanding of the available data, and some simply ignore the available data. The purpose of this paper is to explore the continuum of recent developmental education studies, their data, the conclusions their researchers have made based on that data, the premises upon which those conclusions are based, and the unfortunate end to which some people are taking their arguments. Included are alternative interpretations and recommendations based on the data from this recent research, as well as some rigorous research often left out of the discussion.A Misunderstanding of Developmental Education's PurposeOne of the most prevalent themes in recent research into developmental education is that remedial programs are not effective because students who take this coursework do not perform better than nonremedial students in subsequent comparisons, over and above the comparison of gatekeeper course pass rates (the term "gatekeeper course" is frequently used in this literature and it refers to college-level English or math, the typical first-year mainstream course a student takes at any college). What this means is that recent researchers believe if developmental courses are effective, then students who take these courses should do better than students who never need to take developmental courses, not only in subsequent gatekeeper courses but also in other areas such as persistence or retention (how long a student stays in a class or in college), total number of credits, overall GPA, transfer rates, graduation rates, and labor market outcomes (wages after graduation). Therefore, when remedial students do not ultimately perform better than nonremedial students in all or most of these categories, many researchers conclude that developmental courses do not actually help students in any observable way.One of the most cited sources which makes this argument is Paco Martorell and Isaac McFarlin's "Help or Hindrance? The Effect of College Remediation on Academic and Labor Market Outcomes. " This self-published document was distributed in July of 2007 and revised in April 2010. Martorell, who works forthe RAND Corporation, ran a regression discontinuity (RD) design study to analyze thousands of Texas high school students who took placement examinations to enter the state's colleges and universities in the 1990s.An RD design, in this instance, is a study which compares students who scored just below a placement test's cut-offline (and thus required remediation) to students who scored just above the cut-offline (and thus did not require remediation). For example, if one student scored between 65-69 on ACT's COMPASS* and was required to take a developmental course, and another scored between 70-74 and was allowed to enter a gatekeeper course, then an RD study would compare both of these students in subsequent assessments after the first one took any remedial courses necessary. …