An Examination of Variability as a Function of Passage Variance in CBM Progress Monitoring

Abstract. This study examined the effects of controlling the level of difficulty on the sensitivity of repeated curriculum-based measurement (CBM). Participants included 99 students in Grades 2 through 5 who were administered CBM reading passage probes twice weekly over an 11-week period. Two sets of CBM reading progress monitoring materials were compared: (a) grade level material that was controlled for difficulty, and (b) uncontrolled randomly selected material from graded readers. Students' rate of progress in each progress monitoring series was summarized for slope, standard error of estimate, and standard error of slope. Results suggested that controlled reading passages significantly reduced measurement error as compared to uncontrolled reading passages, leading to increased sensitivity and reliability of measurement. ********** Curriculum-based measurement (CBM) is a set of standardized and specific measurement procedures that can be used to index student performance in the basic academic skill areas of reading, spelling, written expression, and mathematics calculation (Deno, 1985; Deno & Fuchs, 1987; Fuchs & Deno, 1991; Shinn, 1989). As a variant of curriculum-based assessment (CBA), CBM uses dynamic indicators in the basic skill areas for making educational decisions such as screening, instructional planning, and program evaluation (Shinn & Bamonto, 1998). When used within a problem-solving model (Deno, 2002), the primary purposes of CBM are to: (a) obtain point estimates of basic skill performance to identify and certify potential academic weaknesses, and (b) monitor student responsiveness to instruction over time in a formative manner. When used to index student progress in a formative manner, CBM has shown to be highly sensitive to student change over time (Fuchs, 1986, 1989, 1993; Fuchs & Fuchs, 1986a, 1986b). In addition to being sensitive to the effects of instruction, however, CBM has also been shown to be influenced by variables other than instruction. For example, the basic CBM datum in reading (i.e., oral reading fluency or rate) can be affected by variables such as who administers the reading passages and where the reading passages are administered (Derr & Shapiro, 1989; Derr-Minneci & Shapiro, 1992) and the level in the curriculum used for probe development (Dunn & Eckert, 2002; Hintze, Daly, & Shapiro, 1998). When used in a time-series manner, formative decision making and evaluation may be affected by how many data points are available for inspection or the context in which students are being evaluated (e.g., is progress being judged within individual or according to some group standard?) (Shinn, Powell-Smith, & Good, 1996); the nature of the curriculum used for assessment (Hintze & Shapiro, 1997; Hintze, Shapiro, & Lutz, 1994); or by the number of data points used for determining slope (Good & Shinn, 1990; Hintze, Owen, Shapiro, & Daly, 2000; Shinn, Good, & Stein, 1989). In addition to these environmental and decision-making variables, one of the key considerations when adopting CBM for progress monitoring is the manner in which the actual probes are developed. In reading, for example, it has been demonstrated that the material from which reading passage probes are selected and the difficulty of selected material can have significant effects on resultant CBM outcomes. Hintze et al. (1994) found that the type of curriculum used in the sampling process could significantly alter the type of growth that might be observed over time for a student. Specifically, reading curricula that were characterized by uncontrolled readability and vocabulary proved too difficult for students and thus were insensitive to growth over time. In a follow-up study, Hintze and Shapiro (1997) found that by purposively selecting text and controlling for readability and vocabulary content from otherwise uncontrolled material, sensitive progress monitoring growth information could be obtained that mirrored the type of growth that would be seen in controlled text. …