Adequate Yearly Progress, Small Schools, and Students with Disabilities: The Importance of Confidence Intervals When Making Judgments about AYP

Abstract Indicators of school-level achievement, such as the percentage of students who are proficient in a particular content area, are subject to random year-to-year variation in much the same way that the results of an opinion poll will vary from one random sample to another. This random variation, which is more pronounced for a small school, should be taken into account by education officials when evaluating school progress in a policy climate of high stakes. To do otherwise is to risk the false identification of a failing school, whether for all students combined or for the subgroup of students with disabilities. In this article, I describe the application of confidence intervals to the evaluation of "adequate yearly progress" for No Child Left Behind (NCLB). Throughout, I demonstrate the particular relevance of confidence intervals for small schools in general and, more specifically, for the (smaller still) subgroup of students with disabilities. The ambitious agenda of the No Child Left Behind Act of 2001 (NCLB) sets unprecedented challenges for public schools in die United States. And these challenges are particularly daunting for states that have a sizable rural population, where the major precepts of NCLB are often at variance with the reality of rural education (e.g., Reeves, 2003; Tompkins, 2003). To be sure, NCLB provisions regarding school choice, teacher qualifications, technical assistance, supplemental education services, and the evaluation of adequate yearly progress are daunting for any school. But these provisions are considerably more difficult for schools that are small and geographically isolated-that is, for the many schools in this country that reside in rural communities. Among the most ambitious NCLB mandates is that, by 2014, all students must reach proficiency on "challenging academic content standards and challenging student achievement standards." To monitor progress in this regard, the state determines whether each school is making "adequate yearly progress" (AYP) in reading/ language arts and mathematics.1 This is accomplished by first establishing a proficiency "starting point" for each content area, based on 2001-2002 state test scores. The NCLB starting-point criterion that most states have adopted is the 2OtIi percentile school for the particular content area. That is, you first rank-order all schools according to percent proficient2 (e.g., in mathematics) and then count up from the bottom until 20% of the student population is reached. The percentage of students who are proficient in that school is the 2001-2002 starting point for the state. Now subtract this percentage from 100% and divide by 12 and you have the average annual gain required for all students to be proficient in 2014. For illustration, suppose that in your state the mathematics starting point for high schools is 28%. As Figure 1 shows, an average annual gain of 6% would be required to reach 100% proficiency in mathematics by 2014. This trajectory provides the basis for evaluating a school's progress toward that goal. For example, if at least 40% of students in your high school were proficient in 2004, your school "met" or "made" AYP that year. Figure 1 illustrates a linear trajectory with annual increases. Many states employ alternatives to such simplicity. Consider the trajectory adopted by Maine for high school mathematics (Figure 2), which permits slower growth initially and, in later years, requires more rapid (seemingly Herculean) growth toward 100% proficiency in 2014. As most readers doubtless know, NCLB also requires disaggregated evaluations of AYP by various subgroups in the school population: the economically disadvantaged, major racial and ethnic groups, students with limited English proficiency, and of particular interest to readers of Rural Special Education Quarterly, students with disabilities. A school fails to make AYP if any one of these subgroups falls short of the AYP target in either reading/ language arts or mathematics. …