Science Performance Assessment and English Learners: Results from an Elementary Reform Initiative

This post-hoc exploratory analysis examines patterns of student learning as measured by locally developed performance assessments. The assessments are embedded within inquiry-based units of instruction implemented in a multi-year, multi-district, NSF-funded science education reform initiative. The sample consisted of scores from 834 fifth grade students on three performance assessments given in one of the initiative’s participating district’s 14 elementary schools during the 2004-2005 school year. Mean scores were used as the basis for comparison. The results indicate that, across all three assessments, the majority of students achieved at the proficient level as defined by initiative-developed rubrics. The mean scores of English Learners overall were essentially the same as those of their non-English Learner peers (less than .02 point difference on a 4 point scale). Comparisons between non-English Learners and English Learner subgroups (Exit, Limited English Proficient or LEP, Non-English Proficient or NEP) revealed that only NEP students showed significant difference (p<.01) in performance relative to their non-English Learner counterparts on one of the three assessments. Overall, student level demographic variables explained only a small proportion of the variance in the scores for all three assessments. The results indicate the efficacy of the initiative’s reform model which includes aligned curriculum, instruction and assessment along with coordinated teacher professional development on each of those components. The results also lend support to the use of performance assessments over selected response assessments as viable measures of inquirybased science, especially for English Learners. Science Assessment and English Learners 3 Science Performance Assessment and English Learners: Results from an Elementary Reform Initiative This study was motivated by the confluence of three significant factors on the US K-12 educational landscape: an emphasis on inquiry-based science instruction, the value of performance assessment for measuring student learning in such contexts, and the rise of English Learners as a segment of the school age population. The first two factors speak to a desirable alignment between instruction and assessment while the third acknowledges current and projected demographic realities. Seminal documents such as the National Science Education Standards (National Research Council, 1996) tout the need for science teaching that centrally involves “guiding students in active and extended scientific inquiry” (p. 52). Such inquiry-based science instruction fosters student learning of conceptual understanding and sophisticated process skills. Given their propensity to favor “recognition and recall,” the authors of Inquiry and the National Science Education Standards (National Research Council, 2000) note the potential of traditional assessments, such as multiple-choice tests, to “pose a serious obstacle to inquiry-based teaching” (p. 75). Conversely, performance assessment’s ability to tap into complex thinking and skills leads many to regard it as being more closely aligned with the learning outcomes associated with inquiry-based science (Kim, Park, Kang & Noh, 2000; Lee, 1999; National Research Council, 2000). Consideration of the students being assessed in inquiry-based science classrooms leads to the issue of linguistic diversity. English Learners (Non-native speakers of English who are developing their proficiency in English) constitute a sizeable and growing portion of the US KScience Assessment and English Learners 4 12 student population. The website for the National Clearinghouse for English Language Acquisition and Language Instruction Educational Programs (NCELA) states the following regarding English Learners in US public elementary and secondary schools: their number has more than doubled in the past fifteen years (school years 1989/1990-2004/2005), their rate of enrollment has increased at nearly seven times the rate of total enrollment; in 2004/2005 (most recent year for which data are listed) their estimated number totaled nearly 5,120,000 – approximately 10.5% of the total public school enrollment – a figure that represents a 56.2% increase over that reported for school year 1994/1995 (NCELA, 2007). With continued immigration and the ongoing growth of the US Hispanic population, these trends are expected to continue into the near future. The rhetoric of current reform exhorts that high quality science education is for all students, including English Learners (American Association for the Advancement of Science, 1989; National Research Council, 1996). Institutions such as the National Science Foundation have supported this vision by funding large-scale efforts designed to implement inquiry-based science, particularly with diverse or underserved student populations. Reform efforts strive to alignment between instruction and assessment, often by linking inquiry-based curriculum and instruction with performance assessment. Given the above demographic data and the inclusive nature of many science education reform efforts, one can infer that in contexts where inquirybased science instruction is measured via performance assessment, English Learners are increasingly engaging in such assessments. Using performance assessments to measure student learning in inquiry-based science classrooms holds promise and peril. As discussed above, many consider performance assessments to be more congruent with the teaching practices called for in current conceptions of Science Assessment and English Learners 5 educational reform. Noting the bias in traditional assessments, proponents of performance assessment argue its potential to narrow achievement gaps between ethnic, socioeconomic, and gender groups (Lee, 1999). Nevertheless, challenges associated with the use of performance assessments include difficulty and costliness in development and implementation (Baker, 1997). Science performance assessments in particular may be up to 100 times more expensive than multiple-choice tests (Stecher, 1995) and three times more expensive than open-ended writing assessments (Stecher & Klein, 1997). Moreover, studies have found that while girls tend to have higher overall mean scores than boys, patterns of achievement gaps among ethnic and socioeconomic groups are generally the same with both performance assessment and traditional tests (Klein et al., 1997; Lee, 1999; Lynch, 2000; Stecher & Klein, 1997). As Lee (1999) concluded, “The limited available research does not support the contention that performance assessment is more equitable with diverse students than traditional multiple-choice tests” (p. 103). A similar conclusion can be drawn when focusing on assessment of English Learners in content areas such as science. In fact, there is evidence indicating inequitable aspects of performance assessments when used with this student population. Studies have documented significant links between students’ level of proficiency in English and their performance on content-based assessments (Abedi, 2002, Abedi & Lord, 2001; Abedi, Lord, Hofstetter, & Baker, 2000). Understandably, low levels of proficiency in English have been shown to correspond with low levels of achievement. However, confounding variables include the linguistic demands of the assessment which may be mitigated by the use of accommodations such as customized dictionaries (Abedi, 2001). Science Assessment and English Learners 6 When looking exclusively at science performance assessment with English Learners, the research base is quite sparse. Nevertheless, there are indications of similar linkages between English proficiency and student scores (see for example Shaw, 1997). Once again, the limited findings are insufficient to determine a consensus on the relative merit of using performance assessments to measure the achievement of English Learners in inquiry-based science classrooms. The present study was conducted to address these gaps in the literature. While not comparative (i.e., traditional versus performance) in nature, the purpose of the study was to investigate a context in which performance assessments were indeed being used to measure the achievement of English Learners who in fact were taught science via inquiry-based instruction. Accordingly, this post-hoc analysis examined the learning of fifth grade English Learners in inquiry-based science classrooms as measured by a set of classroom-based performance assessments. Specifically, the study investigated the following questions: Regarding 5 grade students’ scores on three classroom-based science performance assessments: 1. What are the patterns of performance for all students, for English Learners in general, and for English Learners at different levels of proficiency in English? 2. What are key similarities and differences in the patterns of performance for the above groups? 3. To what extent does status as an English Learner impact student performance? Research Design Context. This study is based on the work of students taught science by teachers who were participants in a recently completed multi-year, multi-district, NSF-funded science education Science Assessment and English Learners 7 reform initiative known as STEP-uP (Science Teacher Enhancement Program unifying the Pikes Peak region). STEP-uP efforts to improve student learning included the development of performance assessments integrated with inquiry-based curriculum units taught at each grade level, K-5. A prominent feature of the STEP-uP model is the engagement of participating teachers in professional development on the curriculum units as well as the assessments. This study focused on the scores of 5 grade students in a single STEP-uP district, referred to as the Abacus School District (ASD), during the 2004-2005 school year. Assessments. STEP-uP-developed performance assessments were used to measure student learning for the three sci