Since the point of systemic reform is send clearer signals, teachers soon learn what the desired practices are, or at least what the code words are, and in high stakes environments may feel pressured to indicate compliance. Surveys may exaggerate the extent of the changes in practice. There are worse problems with student performance data. Many states and districts do not assess all areas in which we seek to set standards or do not do so adequately. New measures are under development but are not yet trouble free.
Performance measures are expensive and plagued with unresolved technical problems.
This is also true of portfolios. The results of more conventional paper and pencil measures are not trusted by reformers who argue that they are not sensitive to the outcomes they care about, and furthermore, they undermine their efforts to alter practice. We may be in the midst of a great transition in assessment but evaluators need valid, reliable, and stable measures to examine changes over time. Policy makers and grantmakers have trouble waiting two or three years, but we know it takes more time than that to put the policies and supports in place and achieve broad scale changes in practice. Five years seems far too short to expect the broad scale changes reformers believe are needed to produce significant gains in achievement.