# Presentation: 013084

## Power of Reviewing Standards

- Consortia Conference Call
- March 15, 2005
- Susan Pimentel

**Notes:**
<number>

## Welcome Evaluation!

- Benefits:
- Ensures standards are appropriate, clear, accurate and complete
- Builds ownership and awareness
- Provides ideas about rollout of standards. professional development, etc.

## Make it Multi-Faceted!

- 1.  _Internal Review_: Determine how the standards drafts align to the research and other documents as required (e.g., GED, K-12 standards, NCTM)  {Alignment}
- _Field Review_: Ensure standards are understandable and acceptable to field who will be charged with standards’ implementation. {Validity}
- _External Expert Review_: Focus here is primarily  on the sufficiency and understandability of the content.
-       {Validity}

## Tips for Internal Review

- 1.  Ask members of writing teams to conduct.
- 2.  Get specific about how standards  align to research or other docs.  Are they:
- Congruent?
- Aligned?
- Contradictory?
- 3.  Document  extent of alignment (develop written rationale or correlation chart).

## Tips for Field Review: Focus Groups

- Focus Group:
- Involve instructors, program directors, etc.
- Keep groups small (10-15)
- Plan on 3 hours
- Be prepared to walk through the docs
- Employ members of writing teams to run
- Send invitations and drafts well in advance of  session
- Hold sessions around key regions of the state

## Focus Groups, cont’d.

- No defensiveness allowed!! Time to listen and inquire
- Develop feedback forms/surveys
- Document conversations/feedback
- Report back to focus groups (meet again if possible)

## Tips for Field Review:Survey

- Test survey with members of your writing team
- Mail out to programs w/personal appeal to directors
- Make it web-based as well
- Collect quantitative and qualitative information
- Ask about more than just the quality of the standards (e.g., roll out ideas, prof. dev.)
- Consider using a general question survey and standards specific survey
*See attached draft survey.*

## Combo is Best!

- Pros of Focus Group: Personal and in-depth inquiry possible (but time consuming and can only accommodate small numbers)
- Pros of Surveys: Able to reach large numbers and time and cost efficient (but can’t inquire, explain, discuss and promote greater understanding).
- Combo allows gives you depth and breadth; allows you to gather rich qualitative and quantitative information.

## Tips for External Expert Review

- Develop a set of questions for the expert reviewer.
- Include key aspects of the academic domain? Any gaps?
- Clear and specific enough to inform instruction?
- Adherence to research?
- Appropriate rigor?
- Reasonable and manageable?
- Proper  sequence or progression of skills?
- Be clear about how specific you want the reviewer to be (e.g., general evaluation and recommendations or specific suggestions for wording changes and  refinements).
- Need  to give “experts” a month or more lead-time.

## Pros of Expert Review

- They are:
- Current on research
- Familiar with expectations across the nation (and perhaps the world!) so they offer a broad perspective
- Independent
- Able to recognize common pitfalls
- Offer specific recommendations about how to strengthen your standards
- Able to provide standards with a “stamp of approval” that you can present to legislators, policy makers, etc.

## Prepare Teams for Review

*Reviewers tend to tell you what they don’t like rather than what they do like.*
*2.   Read between the lines: Standards mean change and change can be scary so important to hold on to your principles (e.g., must align to the research, prepare adults for success).*
*3.   Prepare for conflicts in the reviews and determine means to grapple with the contradictions (go back to why you asked a group or individual to give you feedback).*

## Prepare Teams, cont’d.

*4.   Let teams know that feedback offers a set of suggestions, not mandates.*
*Having said that, let teams know  when it is just a matter of style or the change won’t diminish the draft, make the change even if it isn’t their preferred choice. It promotes ownership.*
*Set the expectation that teams will review every suggestion carefully and either adjust the standards or note a rationale for not accepting the feedback.*
- The End