A Critical Analysis of the Role of Measurement on the Validity of Emotional and Behavioral Disorders (EBD) Research

"Measurement is science" (Messick, 1988, p.43) and "Science is measurement" (Siegel, 1964, p. 11)In November of 2001, a special issue of Behavior Disorders entitled, "The Meaning of Science and Empirical Rigor in the Social Sciences," was published. This special issue discussed the current status of science in educational research for students with EBD as reviewed by an impressive cadre of colleagues. However, as both Messick (1988) and Siegel (1964) point out, ultimately we cannot talk about science without comparable emphasis on measurement. As the EBD field continues on the present course of expanded and increasingly complex applied scientific inquiry, we need to further examine our past and future research efforts to ensure the validity of our findings, securing that the outcomes of our research truly reflect an enhanced understanding of the individuals and settings in which we purport to investigate. Simply put, we need to examine the meaning and validity of our research findings as they relate to our ability to understand and intervene on behalf of children and youth with EBD in school and community settings.One critically important and relevant area to the examination of research validity is measurement. Accordingly, in this paper we discuss the role of measurement in the evolution of EBD research and the influence of measurement on the validity of our research findings. We begin with an overview of recent national policy and legislative events providing the impetus for highlighting evidence-based practices. Next, we examine the impact of measurement on the availability of evidencebased practices, including a historical perspective of measurement trends in EBD research, followed by a critical analysis of how measurement influences the validity of our research and, in turn, the children, families, and teachers for whom the work is targeted.Defining Evidence-Based Practices Through National InitiativesAlthough always considered part and parcel of solid scientific inquiry, the importance of measurement in EBD research has become even greater due to recent national events. Preeminent among these, the No Child Left Behind (NCLB) Act (U.S. Department of Education, 2002b; 2002c) has brought to the forefront the ever-present need to translate research to practice by further defining evidence-based practices for schools and related settings (OSEP, 2003). One priority NCLB emphasizes is the use of effective, research-based programs and practices in the selection, development, and implementation of instructional strategies as a means to improve education (U.S. Department of Education, 2002b; 2002c). The No Child Left Behind Act includes frequent references to accountability, quality teachers, results, and scientifically based research (U.S. Department of Education, 2002b). Although the degree to which NCLB applies to special education is still somewhat unclear, the renewed focus on the research-to-practice gap is undeniable (Council for Exceptional Children, 2003).The mandate from NCLB to employ only those practices that are scientifically based has prompted several new initiatives by federal and professional organizations to operationally define scientifically based practices, frequently referred to as evidence-based practices. For example, as an extension of NCLB, the Institute : of Educational Sciences (IES) was developed to assist in managing the newly conceptualized What Works Clearinghouse. Through IES, the What Works Clearinghouse has developed its own standards for determining the soundness of scientific research for educators, going so far as to indicate a "preferred (or gold) standard" for research design and methodology (Mageau, 2004). In a related initiative to further define evidence-based practices, the Council for Exceptional Childrens' Division of Research (CEC-DR) created working groups to identity quality indicators across four distinct research methodologies: (1) group design experimental research; (2) single-subject design research; (3) qualitative research; and (4) correlational research (Odom, 2004). …