# Presentation: 547726

## Models for Aligning Assessments to Standards

- Consortia Conference Call
- August 23, 2005
- Regie Stites
-  SRI International

## Assessment Limits

- Some (more) Chinese sayings:
- “Viewing the heavens through a bamboo tube, Measuring the ocean with a spoon.”
- “Riding a horse, viewing flowers.”
- “Frog sitting in a well, viewing the sky.”

## Accountability Demands

- An American saying:
- “What gets tested, gets taught.”
- When West meets East, we’ve got a world of trouble unless ...
- everyone understands the limits of assessment, and
- we achieve reasonably good alignment of standards, assessment, and instruction.

## Two Directions for Alignment

- Horizontal – align content standards to assessments, match content and depth
- Vertical – align accountability tests to educational systems (curricula, materials content, instruction, student outcomes, stakeholder opinions)

## How to Align Tests and Standards:Three Methods

- Sequential – develop content standards and curriculum, then develop standards-based tests
- Expert review – use expert judgments to evaluate alignment, then select/reject test or explicate alignment
- Content analysis – use content criteria to guide formal analysis, then select/reject test or explicate alignment

## How to Analyze Alignment: Four Expert Models

- ‘Webb’* – *procedures and criteria developed by Norman Webb
- ‘SEC’ – Surveys of Enacted Curriculum
- ‘Achieve’ – alignment analysis service provided by Achieve, Inc.
- ‘CBE’ – technical assistance from the Council for Basic Education
- See ‘Dimensions of Comparison’ on page 6 of Models for Alignment Analysis and Assistance to States, CCSSO (2002)

## How Alignment Analysis Helps

- Provides feedback for revision of assessments or standards
- Provides information to help align instruction with assessments and standards
- Helps everyone understand the proper limits for interpreting results of accountability tests

## Metaphors for Alignment Goals

- Alignment as Congruence – assessment and standards content mirror each other
- Alignment as a Set of Correspondences – assessment content samples from standards content
- Alignment as a Bridge –curriculum content connects standards and assessments
- Alignment as Gravitational Pull – standards, assessments, and instruction all reflect common elements of a reform agenda
- [from Baker, 2004]

## Achieve’s Alignment Protocol:Four Dimensions

- Content centrality – test item match to content of related standard
- Performance centrality – test item match to the type of performance (cognitive demand) of related standard
- Challenge – source and level of challenge for sets of items
- Balance and range – match between emphasis and coverage of sets of items and standards

## Achieve’s Alignment Protocol:Three Stages

- Stage 1: Item-by-item analysis
- 	1(a) Confirming the test blueprint
- 	1(b) Content centrality
- 	1(c) Performance centrality
- Stage 2: Challenge
- 	2(a) Source of challenge (each item)
- 	2(b) Level of challenge (sets of items mapped to each standard)
- Stage 3: Balance and range (sets of items mapped to each standard)
- 	3(a) Balance (emphasis)
- 	3(b) Range (coverage)

## Achieve’s Alignment Protocol:Ratings

- Stage 1: Content and performance centrality ratings =
- 2 clearly consistent; 1A not specific enough;
- 1B somewhat consistent; 0 inconsistent
- Stage 2: Source of challenge rating =
- 1 appropriate source(s); 0 inappropriate source(s)
- Stage 2: Level of challenge rating = narrative evaluation
- Stage 3: Balance rating = narrative evaluation
- Stage 3: Range rating = fraction of total objectives (indicators) mapped to standard assessed by at least one item
- [above .67 = good, between .50 and .66 = acceptable]

## Achieve’s Alignment Protocol:Results

- Elementary English Language Arts
- Content centrality (% of items rated 2)
- State A = 100; State B = 89; State C = 66; State D = 46; State E = 29
- Performance centrality (% of items rated 2)
- State A = 100; State B = 79; State C = 62; State D = 79; State E = 21
- Source of challenge (% of items rated ‘appropriate’)
- State A = 90; State B = 76; State C = 89; State D = 88; State E = 79
- Average range scores
- State A = .27; State B = .75; State C = .73; State D = .52; State E = .31

## Some Resources

**Achieve alignment procedures**
- Robert Rothman et al. (May 2002). *Benchmarking and alignment of standards and testing*.  _[http://www.cresst.org/Reports/TR566.pdf](http://www.cresst.org/Reports/TR566.pdf)_
**Metaphors for alignment**
- Eva Baker. (December 2004). *Aligning curriculum, standards, and assessments: Fulfilling the promise of school reform*.  _[http://www.cresst.org/reports/r645.pdf](http://www.cresst.org/reports/r645.pdf)_
**Test publishers see more alignment than teachers**
- Chad Buckendahl et al. (2000).  *Alignment of standardized achievement tests to state content standards: A comparison of publishers’ and teachers’ perspectives*. _[http://www.unl.edu/BIACO/NCME/buckendahlcw2.pdf](http://www.unl.edu/BIACO/NCME/buckendahlcw2.pdf)_
**‘****Nine requirements’ for a responsible state assessment system**
- Commission on Instructionally Supportive Assessment. (October 2001).  *Building tests to support instruction and accountability: A guide for policymakers*. _[http://www.aasa.org/issues_and_insights/assessment/Building_Tests.pdf](http://www.aasa.org/issues_and_insights/assessment/Building_Tests.pdf)_

## Resources Continued

**Horizontal and Vertical Alignment**Betsy Chase and Sasha Zucker, Harcourt Assessments, Inc.
**Rethinking Issues of Alignment **Under No Child Left BehindSri Ananda, WestEd.

## Discussion Points

- What are your goals for alignment? What do you want to learn from an alignment analysis? What will you do with the results of an alignment analysis?
- help educators and stakeholders interpret results of accountability tests?
- influence instructional content and practice?
- shape policy?