Testing the Test: A Study of PARCC Field Trials in Two School Districts. Policy Brief
Author: Rennie Center for Education Research and Policy
Date: 2015-03-00
Type: Reports - Research
Subjects: Computer Assisted Testing,Educational Assessment,School Districts,Case Studies,College Readiness,Career Readiness,Public Schools,Interviews,Focus Groups,Performance Based Assessment,Pretests Posttests,Surveys,Access to Computers,Laptop Computers,Handheld Devices,Computers,Scheduling,Testing,Staff Utilization,Faculty Development,Orientation,Student Experience,Elementary School Students,Middle School Students,High School Students,Elementary School Teachers,Middle School Teachers,Secondary School Teachers

The potential use of computer-based assessments has raised concerns from educators, policymakers, and parents about information technology infrastructure in school districts and the preparation of staff and students to use new technologies for assessment purposes, and the potential impact of testing activities on core school functions, particularly teaching and learning. This case study documents lessons learned in the district- or school-wide administration of computer-based student assessments in the Burlington and Revere School Districts during the spring 2014 Partnership for Assessment of Readiness for College and Careers (PARCC) field test. The case study focused on three research questions: (1) What was the preparation process for district- and school-wide implementation of an online student assessment, including decision making on scheduling, staffing, professional development, technology and hardware, and test materials management?; (2) What successes and challenges were discovered through administration of computer-based assessments? What role did test-taking instructions, appropriateness of physical space, facility with technology, as well as the quality of professional development play in these discoveries?; and (3) What was the reported experience of student test-takers? What were educators' (e.g., school leaders, teachers, test administrators) perspectives on students' test-taking experience? To adequately answer these research questions, the Rennie Center team conducted the following research activities in both Burlington Public Schools and Revere Public Schools: interviews with district leadership and district technology staff; focus groups with principals, teachers, and students from all testing schools (i.e., elementary, middle, and high schools), accounting for nearly 170 leaders, teachers, and students; survey of staff prior to the administration of the Performance-based Assessment (PBA) tests (i.e., pre-test survey); and survey of staff after the administration of the PBA tests (i.e., post-test survey). Almost 400 staff members, across the two districts, responded to the pre-test survey and about 250 to the post-test survey. The overall response rate for the pre-test survey was about 60%; 99% of Revere educators in the testing schools completed a survey. In Burlington, where all educators across the district received the questionnaire, 55% of educators completed a pre-test survey. For the post-test survey, response rates were about 42% overall (88% in testing schools in Revere; and 34% in Burlington). At Burlington High School, fewer teachers were directly involved in test administration since only 10th grade students were tested in English language arts and math, but not in other subjects. The findings section presents themes that were common across respondents regardless of role or district affiliation, and offers more detailed descriptions of specific implementation issues. Key findings from the PARCC educator survey is appended. [Additional support for this project was provided by the Massachusetts Teachers Association and the Massachusetts Association of School Superintendents.]