MEMO To: Dr. Mitchell Chester, Commissioner of Elementary and Secondary Education, Massachusetts Department of Elementary and Secondary Education CC: Carrie Conaway, Associate Commissioner of Planning, Research, and Delivery Systems, Massachusetts Department of Elementary and Secondary Education James Peyser, Massachusetts Secretary of Education Laura Slover, Chief Executive Officer of PARCC Inc/Partnership for the Assessment of Readiness for College and Careers Francine Markowitz, Program Associate: Policy, Research and Design of PARCC Inc/Partnership for the Assessment of Readiness for College and Careers From: Michael J. Petrilli and Amber Northern, the Thomas B. Fordham Institute Date: October 30, 2015 Re: Evaluation of the Content and Quality of the 2014 MCAS and relative to the CCSSO Criteria for High Quality Assessments We are pleased to provide this special report as requested by the Massachusetts Department of Elementary and Secondary Education. We hope that it will provide useful information for planning purposes in Massachusetts relative to future assessments of English language arts/literacy and mathematics for students in grades 3–8. The data and analyses provided herein are a subset of the results of an evaluation that examined the content and quality of four assessments relative to the CCSSO Criteria for High Quality Assessments in ELA/literacy and mathematics. The Thomas B. Fordham Institute convened expert panels in summer 2015 to evaluate operational grade five and eight assessments (end of elementary and middle school), while the Human Resources Research Organization (HumRRO) conducted a similar evaluation at the high school level (reported separately). This special report includes takeaways from the Fordham study for two of the four assessment programs: the 2014 MCAS and the Partnership for Assessment of Readiness for College and Careers (PARCC). The full report, to be released in January 2016, will add results from Smarter Balanced and ACT Aspire, as well as detailed supplementary analyses (that do not impact the final ratings contained herein). The report was co-authored by Nancy Doorey and Morgan Polikoff, whose biographies appear at the end. Should you have any questions about the results that follow, do not hesitate to contact us. Kindly, Michael J. Petrilli Amber M. Northern President Senior Vice President for Research
Table 1. Final English Language Arts/Literacy Ratings and Summary Statements, Grades Five and Eight (MCAS and PARCC) Content Sub-Criteria MCAS Rating MCAS Summary Statement PARCC Rating PARCC Summary Statement B.6 Vocabulary and language skills: Tests place sufficient emphasis on academic vocabulary and language conventions as used in real-world activities. L On “emphasizing vocabulary and language skills,” the rating is Limited/Uneven Match. Vocabulary items are sufficient and generally aligned to the criterion; however, the grade five items need more words at the Tier 2 level. Furthermore, a lack of program documentation means that the quality of vocabulary assessments cannot be substantiated across forms. MCAS does not meet the criterion for assessing language skills, which call for them to be assessed within writing assessments that mirror real- world activities including editing and revision. E On “emphasizing vocabulary and language skills,” the rating is Excellent Match. The test contains an adequate number of high-quality items for both language use and Tier 2 vocabulary and awards sufficient score points, according to the program’s documentation, to both of these areas. B.7 Research and inquiry: Assessments require students to demonstrate the ability to find, process, synthesize and organize information from multiple sources. W On “assessing research and inquiry”, the rating is Weak Match. The assessment has no test questions devoted to research. E On “assessing research and inquiry,” the rating is Excellent Match. The research items require analysis, synthesis, and/or organization and the use of multiple sources, therefore meeting the criterion for Excellent. B.8 Speaking and listening: Over time, and as assessment advances allow, the assessments measure speaking and listening communication skills. W On assessing “speaking and listening,” the rating is Weak Match. The program does not assess speaking or listening at this time. Because this criterion is to be met “over time, as assessment advances allow,” this rating is not included in the overall rating for Content. W On assessing “speaking and listening,” the rating is Weak The program does not assess speaking or listening at this time. Because this criterion is to be met “over time, as assessment advances allow,” this rating is not included in the overall rating for Content.