Appendices

PAGE

Appendix A: White Paper: Moving Toward Standards-Based Education A--1

Appendix B: Glossary B--1

Appendix C: Tools and Templates C--1

Appendix D: References and Resources D--1

Appendix A

White Paper: Moving Toward Standards-Based Education

# Roots of the Standards-Based Movement

The standards-based reform movement began in the 1980s with the public
response to discouraging reports on the quality of American public
education, such as *A Nation at Risk* (National Commission on Excellence
in Education 1983), which unfavorably compared American school
performance to that of other industrialized countries. High-level
attention was given to public school reform by then-president George H.
W. Bush and the agreement of the nation's governors, written up as *The
National Education Goals Panel Report: Building a Nation of Learners*
(1991). The goals set forth in that document were ambitious, visionary,
and sweeping. The signers pledged to improve academic achievement and
ensure readiness to learn, adult literacy, and safe schools. President
Clinton followed up in 1994 with Goals 2000 legislation, thus taking a
further step toward national standards and accountability mechanisms. In
2001, President George W. Bush authorized the No Child Left Behind
legislation that further emphasized the focus on accountability and
assessment.

Simultaneously, the changing labor market and economy in the 1980s
signaled a call to address school reform. A report from the National
Center on Education and the Economy (1990), *America's Choice: High
Skills or Low Wages?*, focused attention at the policy and corporate
levels on the need for high school reform as well as retraining workers
and opportunities for upgrading skills. The U.S. Department of Labor
published the report of the Secretary's Commission on Achieving
Necessary Skills (U.S. Department of Labor 1991), setting priorities of
"high performance workplaces"---those that require flexible thinking,
continual training, employee empowerment, responsibility for quality and
production management, and teamwork. The training necessary to achieve
these workplace skills was considered at odds with the traditional high
school curriculum and performance expectations, which emphasized
individual work and isolated subject areas. Similar concerns about the
mismatches between schooling goals and new market realities inform the
more recent report from the Educational Testing Service, *Standards for
What? The Economic Roots of K--16 Reform* (Carnevale and Desrochers
2003).

These reports have publicly involved the business community and the U.S.
Department of Labor in the school reform conversation. This involvement
can be seen in the move in vocational and adult education toward a focus
on workforce development and increased calls for accountability. This
move was cemented when adult education funding was included in the 1998
Workforce Investment Act, pending reauthorization as the Workforce
Reinvestment and Adult Education Act. This link with labor secures a
strong voice for business and industry in education, especially in
secondary and adult education---a voice very familiar with the language
of standards and
accountability.![](media/image1.wmf){width="1.8395833333333333in"
height="0.575in"}

## What Adult Education Can Learn From the K--12 Experience

Adult education and literacy programs enter an era of standards-based
education when leaders in the K--16 educational world are calling for
"mid-course corrections" to the standards reforms (Darling-Hammond
2003). The timing of this entrance affords adult education the advantage
of learning from earlier successes as well as from missteps and
misalignments (Stites 1999).

The standards-based education movement has led to reforms---initiated in
K--12 education, such as more transparent public accountability
mechanisms, higher expectations for students and schools regardless of
local economics---and an increased reliance on standardized assessments
and accountability systems that can support comparisons across schools,
districts, and states.

These reform efforts and the data derived from their study have yielded
positive outcomes, such as the following:

-   incontrovertible evidence of the positive impact on student learning
    from well-trained teachers (Darling-Hammond 1999)

-   invigoration of research on best practices for professional
    development (Cochran-Smith and Lytle 1999; Darling-Hammond and Sykes
    1999; Loucks-Horsley et al. 1998)

-   public and policy involvement in educational accountability (Dutro
    and Valencia 2004; Merrow 2001)

-   higher expectations for all students in spite of their ethnicity,
    economics, ability status, etc. (American Federation of Teachers
    2003; Darling-Hammond 2003)

As a result of these outcomes, public education advocacy groups began
discussing the allocation of resources necessary to build the capacity
for schools and districts to implement standards-based education and for
students to be able to succeed in classes with high standards.

Darling-Hammond (2003) suggests three areas to which the standards-based
education movement needs to pay attention to accomplish the "mid-course
correction" that can rectify some of the traditional and growing
inequalities in the American education system and increase the capacity
of institutions to offer high quality education:

-   *quality and alignment* of standards, curriculum guidance, and
    assessments

-   *appropriate use of assessments* to improve instruction rather than
    punish students and schools

-   development of systems that ensure *equal and adequate opportunity
    to learn*

Similar to most change efforts, however, the reforms brought about
through the implementation of standards have resulted in controversy and
unintended consequences. Critics decry:

-   overreliance on standardized tests and high-stakes testing (Kohn
    2000; Merrow 2001)

-   standards that are unrealistic and cumbersome "wish lists" created
    by content area experts (Marzano and Kendall 1998; Popham 2001)

-   early abandonment of the opportunity to learn standards that focused
    on equitable inputs and resources (NCTE and IRA 1996; Stites 1999)

-   misguided sequence of adopting assessment tools before aligning
    standards and curriculum (Merrow 2001)

These mistakes have taken an uneven toll on educationally vulnerable
students (Darling-Hammond 2003) and disadvantaged schools. They have
also been seen as a threat to teachers' creativity and to local
responsiveness to cultural interests and needs. It is critical that
adult educators consider these criticisms and address them openly as the
field moves forward with standards-based education; these criticisms
will no doubt resurface in local communities and programs.

Adult education has the opportunity to implement standards-based
education and strong accountability systems with a maturity balanced by
others' experiences and educators' social vision, born of long history
of social activism and work with marginalized learners. Standards-based
education can be a call to action, not merely from the top policy makers
down to the field, but from the field *out* to all stakeholders and
students---a call to dialogue on what constitutes quality adult
education.

# References

American Federation of Teachers. 2003. *Setting strong standards*
(Educational Issues Department No. 39-0175). Washington, DC: American
Federation of Teachers.
[http://www.aft.org/pubs-reports/downloads/teachers/\
settingstrong.pdf](http://www.aft.org/pubs-reports/downloads/teachers/settingstrong.pdf)
(accessed December 2007).

Carnevale, A. P., and D. M. Desrochers. 2003. *Standards for what? The
economic roots of K--16 reform.* Princeton, NJ: Educational Testing
Service.

Cochran-Smith, M., and S. L. Lytle. 1999. The teacher research movement:
A decade later. *Educational Researcher,* 28 (7): 15--25.

Darling-Hammond, L. 2003. Standards and assessments: Where we are and
what we need. *Teachers College Record,* February 16.
[http://www.tcrecord.org/\
Content.asp?ContentID=11109](http://www.tcrecord.org/Content.asp?ContentID=11109)
(accessed December 2007).

Darling-Hammond, L. 1999. *Teacher quality and student achievement: A
review of state policy evidence.* Seattle, WA: University of Washington,
Center for the Study of Teaching and Policy.
[http://depts.washington.edu/ctpmail/PDFs/\
LDH_1999.pdf](http://depts.washington.edu/ctpmail/PDFs/LDH_1999.pdf)
(accessed December 2007).

Darling-Hammond, L., and G. Sykes, eds. 1999. *Teaching as the learning
profession: Handbook of policy and practice.* San Francisco:
Jossey-Bass.

Dutro, E., and S. W. Valencia. 2004. *The relation between state and
district literacy standards: Issues of alignment, influence, and
utility.* Seattle, WA: University of Washington, Center for the Study of
Teaching and Policy. [http://depts.washington.edu/ctpmail/PDFs/\
LitStandards-EDSV-01-04.pdf](http://depts.washington.edu/ctpmail/PDFs/LitStandards-EDSV-01-04.pdf)
(accessed December 2007).

Kohn, A. 2000. *The case against standardized testing: Raising the
scores, ruining the schools*. Portsmouth, NH: Heinemann.

Loucks-Horsley, S., P. W. Hewson, N. Love, and K. E. Stiles. 1998.
*Designing professional development for teachers of science and
mathematics.* Thousand Oaks, CA: Corwin Press.

Marzano, R. J., and J. S. Kendall. 1998. *Awash in a sea of standards*.
Aurora, CO: Mid-continent Research for Education and Learning.
[http://www.mcrel.org/topics/\
products/120](http://www.mcrel.org/topics/products/120/)/ (accessed
December 2007).

Merrow, J. 2001. Undermining standards. *Phi Delta Kappan,* May,
652--659.

National Center on Education and the Economy. 1990. *America's choice:
High skills or low wages?* Washington, DC: National Center on Education
and the Economy.

National Commission on Excellence in Education. 1983. *A nation at
risk*: *The imperative for educational reform* (Publication no. ED 224
839). Washington, DC: Government Printing Office.

National Council of Teachers of English (NCTE), and International
Reading Association (IRA). 1996. *Standards for the English language
arts.* Urbana, IL: NCTE.

National Education Goals Panel. 1991. *The National Education Goals
Panel report: Building a nation of learners*. Washington, DC: National
Education Goals Panel. <http://govinfo.library.unt.edu/negp> (accessed
December 2007).

Popham, W. J. 2001. Whittling wish-list standards down to reality.
*Education Digest,* 66 (7): 8. <http://pqasb.pqarchiver.com/eddigest>
(accessed December 2007).

Stites, R. 1999. A user's guide to standards-based educational reform:
From theory to practice. *Focus on Basics,* 3 (C). Cambridge, MA:
National Center for the Study of Adult Literacy and Learning (NCSALL).
<http://www.ncsall.net/?id=352> (accessed December 2007).

U.S. Department of Labor. Secretary's Commission on Achieving Necessary
Skills (SCANS). 1991. *What work requires of schools* (Publication no.
ED 332 054). Washington, DC: U.S. Government Printing Office.

Appendix B

Glossary

Adult Basic Education/Adult Secondary Education (ABE/ASE)

Adult basic education (ABE) and literacy instruction emphasizes basic
skills development in reading, writing, mathematics, and problem solving
for adults below the eighth grade proficiency level. Adult secondary
education (ASE) instruction helps learners prepare to receive a high
school credential or for successful entry into employment or
postsecondary education and training.

Adult Education Content Standards Warehouse

The Adult Education Content Standards Warehouse is an online repository
of adult education content standards documents in English language
acquisition, mathematics, and reading. Users can research and retrieve
documents by a specific sponsor or author, or they can explore specific
content within any given standards. The warehouse is funded by the U.S.
Department of Education and can be accessed at
<http://www.adultedcontentstandards.ed.gov/>.

Academic (or Basic) Skills-Based Approach

An academic skills-based approach focuses on skill development in
reading, writing, mathematics, and English language acquisition. Basic
skills curricula usually consist of a sequence of skills that are
introduced and practiced at higher levels of complexity as learners
advance within the program.

Accountability

Accountability considers the extent to which an individual, group, or
institution is held responsible for meeting specified outcome measures.
Accountability systems require programs to provide substantiated
evidence of student achievement (e.g., educational achievement, entry
into employment, receipt of a high school credential) as a condition of
funding.

Advisory Committee

An advisory committee often comprises leaders or experts in a content
area who represent various stakeholder perspectives and provide overall
guidance to a project.

Alignment

Alignment is a documented connection among standards, teaching,
learning, and assessment. Alignment is essential to fairness in an
accountability system. Only when the components are aligned can programs
expect to see higher outcomes and sustainable program improvement.

American Diploma Project (ADP)

The American Diploma Project (ADP) is a joint project launched by
Achieve, Inc., the Education Trust, and the Thomas B. Fordham
Foundation. The goals are to (1) determine the English and mathematics
skills that high school graduates need to be successful in college and
the workplace and (2) help states incorporate those skills into
standards, assessments, and high school graduation requirements. ADP
outlines college and workplace readiness benchmarks in English and
mathematics and provides samples of how academic standards are used in
the real world. See [http://www.achieve.org](http://www.achieve.org/).

Assessment

Assessment is process for monitoring and evaluating student performance
and achievement. Assessment methods include standardized tests and
classroom-based measures such as observations, projects, interviews,
portfolios, quizzes, etc. Assessments can be conducted at the
individual, classroom, school, district, state, and national levels.

Benchmarks

Benchmarks describe the set of skills and knowledge learners need to
develop and achieve to master a content standard. (See also
*Indicators.*)

Bias

When the content or language of a document reflects a prejudice or
stereotype of a particular group, it may be considered "biased."
Potential bias may hinder learning and can be found in references to
age, gender, racial/ethnic, cultural, disability, socioeconomic,
community (rural, urban, or suburban), and/or linguistic groups in the
population of adult learners to be served by the content standards.

CASAS (Comprehensive Adult Student Assessment System)

CASAS is a widely used system for adult education that includes life
skill competencies, standardized assessments, curriculum and training
resources, and instruments for program evaluation. CASAS assessments can
be used with both native and non-native speakers of English to measure
basic skills in reading, math, listening, writing, and speaking within
functional contexts. See <http://casas.org/casasnewweb/index.cfm>
(accessed December 2007).

Competency-Based/Life Skills Approach

A competency-based approach focuses on the functional use of reading,
writing, mathematics, and speaking skills in adult contexts.
Competency-based or survival curricula offer a list of competencies in
topic areas such as consumerism, health, or employment. Learners
identify important competencies and then develop the basic reading,
math, and language skills they need to complete a real-life or
functional task.

Congruence

Congruence occurs when a content standard coincides exactly with the
document(s) with which it must align.

Content Area

A content area is a subject or discipline such as reading, mathematics,
science, or English language acquisition.

Content Standards

Content standards describe what learners should know and be able to do
within a specific content area.

Contradiction

Contradiction occurs when a content standard is inconsistent with or in
opposition to the document(s) with which it must align.

Coordinating Committee

A coordinating committee provides oversight and leadership throughout
the standards-based initiative---planning, developing, reviewing, and
implementing the standards. The coordinating committee is responsible
for making recommendations to the state.

Curriculum

Curricula provide *detailed* outlines of the knowledge and skills for
different instructional levels (e.g., a scope and sequence) and serve as
a road map for teachers in planning lessons. Curricula often provide
suggestions for teaching techniques, learning activities, textbooks, and
materials.

Curriculum Framework

Frameworks are *broad* outlines of the knowledge and skills that
programs use in developing local curricula. Frameworks can guide the
development of curriculum but do not specify how to teach.

EFF (Equipped for the Future)

EFF is a standards-based reform initiative that collaboratively "mapped
out" the roles and critical responsibilities for adults as members of
families, communities, and the workforce. The National Institute for
Literacy collaborated with hundreds of people in the development of a
framework that educators and learners could use in planning learning
experiences that meet real-world needs in the 21st century. See
[http://eff.cls.utk.edu](http://eff.cls.utk.edu/).

Environmental Scan

An environmental scan is a process for obtaining information, thoughts,
and opinions from a wide range of people and programs. It includes
literature reviews, formal or informal surveys, focus groups and
individual interviews, and reviews of data and documents in the field.
Information gathered can help inform decision making and determine
project activities.

ESL (English as a Second Language)

ESL programs focus on teaching English language and literacy skills to
non-native speakers of English. Other commonly used terms include ESOL
(English for speakers of other languages), ELA (English language
acquisition), and ELL (English language learners).

Evaluation

Evaluation involves reviewing, comparing, and judging the quality of
work based on established criteria. Summative evaluation is usually done
at specified "end points" through formal and often standardized
measures. In contrast, formative evaluation is an ongoing process that
assesses understanding and skills through a variety of formal and
informal assessment strategies.

Exemplars

Exemplars are samples of student work that illustrate the type and
complexity of performance expected at different instructional levels.

Field Test

Field tests are small-scale trials to assess how effectively new
products, initiatives, materials, or approaches can be used in a real
context. Data collected from the programs that participate in a field
test can be used to inform revisions and implementation procedures.

Focus Groups

Focus groups are structured interviews with 8--12 individuals in which a
facilitator guides discussion around a set topic. Focus groups allow the
facilitator to ask probing questions to gain an understanding of the
participants' reactions, opinions, and suggestions.

GED (General Educational Development \[test\])

The GED test measures a learner's knowledge and academic skills in
reading, mathematics, science, socials studies, and writing. A
certificate is given to learners who attain a passing score on the GED
test. A GED is recognized in some states and by some employers as a high
school credential.

Indicators

Indicators are measurable behaviors that reflect the skills and
knowledge learners need to develop and achieve to master a content
standard. (See also *Benchmarks.*)

International Reading Association (IRA)

The International Reading Association is a professional organization for
teachers of reading to learners of all ages. NCTE and IRA developed a
set of national reading standards called *Standards for the English
Language Arts.* See [http://www.reading.org](http://www.reading.org/).

National Council of Teachers of English (NCTE)

The National Council of Teachers of English is a professional
organization for teachers of English and English language arts. NCTE and
IRA developed a set of national reading standards called *Standards for
the English Language Arts.* See
[http://www.ncte.org](http://www.ncte.org/).

National Council of Teachers of Mathematics (NCTM)

The National Council of Teachers of Mathematics is a professional
organization for teachers of mathematics. NCTM has developed a set of
national mathematics standards called *Principles and Standards for
School Mathematics,* which provides guidelines for excellence in
mathematics education. See [http://www.nctm.org](http://www.nctm.org/).

National Reporting System (NRS)

The National Reporting System is an outcomes-based accountability system
for state-administered, federally funded adult education programs. The
NRS was designed to meet accountability requirements for adult education
programs required by Title II of the Workforce Investment Act. See
[http://www.nrsweb.org](http://www.nrsweb.org/).

Opportunity-to-Learn Standards\
Opportunity-to-learn standards describe or specify the instructional
conditions and resources necessary for adult learners to learn and
achieve content and performance standards. When provided with sufficient
opportunity (e.g., equitable access, appropriate curriculum and
materials, adequate facilities, and trained teachers), learners can
achieve the necessary knowledge and skills stated in the content
standards.

Outcomes

Outcomes are measures of achievement that result from participation in
adult education. Within adult education, the NRS outcomes include
measures of (1) educational gain in literacy skills; (2) entry into
employment, postsecondary education, and training; and (3) high school
completion. Adult education programs often track secondary outcomes such
as participating in children's education, voting, and obtaining a
driver's license.

Participatory or Learner-Centered Approach

A participatory approach focuses on the expressed needs and interests of
learners. Participatory approaches build on learners' prior knowledge
and often use problem-posing techniques to construct meaning generated
from texts and situations that adults encounter in life.

Performance-Based Assessment

Performance-based assessments are real-life or simulated tasks that
require learners to apply knowledge and skills to demonstrate
achievement of the indicators or content standards. Performance-based
assessments can be in the form of projects, presentations, tests, or
writing tasks.

Performance Standards

Performance standards describe how well or to what extent learners meet
the content standards.

Performance Descriptions

Performance descriptions state what students should know and the ways
they can demonstrate their knowledge and skills.

Portfolio

A portfolio is a purposeful collection of a learner's work to
demonstrate acquisition and application of knowledge and skills.
Portfolios are used to document and assess performance, achievement, or
progress. They often include learners' self-assessments.

Program Standards

Program standards describe the design, operation, and management of
programs and services rather than individuals' skills and performances.
Program standards address a full range of issues related to educational
program design and delivery, including administration, staffing,
assessment, curriculum, instruction, professional development, support
services, intake, and orientation.

Progress Monitoring

Progress monitoring is the ongoing review and assessment of a learner's
knowledge and skills. Continual classroom monitoring allows instructors
to identify strategies and materials that will help learners gain the
knowledge and skills to meet the content standards and to meet their
goals.

Reliability

Reliability refers to the degree to which the results of an assessment
are consistent when conducted over time and by different people, or
across different tasks that measure the same thing.

Rubric

Rubrics are tools that define or describe the criteria for assessing
learners' competence on assigned tasks and performances. Rubrics often
contain short, narrative descriptions at various levels on a continuum
(e.g., emergent, developing, proficient, advanced) and can be used to
assess student work as a whole or to assess components of student work.
Rubrics can be shared with learners so they understand the criteria
necessary for improving their performance. Rubrics are often used to
assess written work, oral presentations, project-based products, etc.

SCANS (Secretary's Commission on Achieving Necessary Skills)

The Secretary's Commission was formed by the U.S. Secretary of Labor to
define the skills all high school graduates should have for employment.
SCANS identified competencies in five key areas (Resources,
Interpersonal, Information Management, Systems, and Technology) and the
foundation of skills that lie at the heart of job performance (Basic
Skills, Thinking Skills, and Personal Qualities). See
<http://www.bused.org/scans.html>.

Stakeholders

Stakeholders are the people (or groups of people) with a vested interest
in a program or project. Adult education stakeholders include learners,
teachers, administrators, school staff, advocacy organizations,
community members, higher education institutions, and employers who have
a significant interest in public education. Broad stakeholder input is
essential for the successful development and implementation of content
standards.

Standards-Based Education (SBE)

Standards-based education is a reform effort that defines what is
important for learners to know and be able to do (content standards) and
aligns assessment, curriculum, instruction, and professional
development. SBE provides a systemic model for educational improvement.

Standardized Tests

Standardized tests are formal methods of assessing student performance
that use the same content, task-scoring procedures, and reporting
procedures for all learners. Standardized tests have empirically
determined, quantifiable measures of reliability and studies of their
validity. Such tests are popular accountability methods because they
allow for comparison across states and programs.

Study Circles

Study circles bring adult education practitioners (e.g., teachers,
counselors, administrators, or others) together in small group settings
to learn about research findings and theories and to explore how the
findings can be applied to practice and policy. Study circles are often
used for ongoing professional development.

Teachers of English to Speakers of Other Languages, Inc. (TESOL)

Teachers of English to Speakers of Other Languages, Inc., is a
professional organization for teachers of English to non-native English
speaking learners. TESOL has developed a set of national standards for
Pre-K--12 settings. See [http://www.tesol.org](http://www.tesol.org/).

Validity

Validity refers to the extent to which a measure reflects the underlying
concept of what it is supposed to measure. Effective assessments must
demonstrate their validity through empirical studies that involve
comparing their measures with a related measure derived from another
source (e.g., another assessment, expert judgment).

Appendix C

Tools and Templates

Sample Budget Worksheet for Content Standards Initiative C--2

Moderator Guidelines for Focus Group Discussion C--4

Sample Focus Group Protocol C--5

Indiana Adult Education Content Standards: Development Team Information
and\
Application C--8

Generic Content Standards Team Application C--10

Criteria for Reviewing Content Standards Documents C--11

Kentucky Content Standards Review Survey C--18

Standards Usability Checklist Instructor Feedback Form C--22

Sample Implementation Timeline C--23

Arizona Implementation Timeline C--25

Sample Action Plan---Implementation C--29

# Sample Budget Worksheet for Content Standards Initiative

A budget worksheet will help state staff think about and plan for the
potential costs and resources to support the standards initiative.

  --------------------------------------------------------------------------
  Expense                    Budget Calculation  Estimated     Funding
                                                 Cost          Source or
                                                               Charge Code
  -------------------------- ------------------- ------------- -------------
  *Personnel*                                                  

  State Staff                                                  

  Project oversight                                            

  Coordination of tasks                                        
  throughout the initiative                                    
  (e.g., developing work                                       
  plan, gathering resources,                                   
  participating in                                             
  environmental scan,                                          
  working with writing                                         
  teams, overseeing the                                        
  review and implementation                                    
  processes)                                                   

  Professional development                                     

  Support staff                                                

  Local Program Staff                                          

  Pay or stipends for                                          
  writing team members                                         

  Honoraria for focus groups                                   
  or reviews                                                   

  Field test participation                                     
  (i.e., local efforts to                                      
  plan, coordinate, and                                        
  gather information for                                       
  reporting)                                                   

  Professional development                                     

  Consultants/Contractors\                                     
  (fees and honoraria)                                         

  Advisors                                                     

  Facilitators                                                 

  Reviewers                                                    

  Preparation of                                               
  professional development                                     
  materials                                                    

  Editing and designing                                        

  *Travel Expenses (mileage,                                   
  airfare, lodging, per                                        
  diem, etc.)*                                                 

  Consultant(s)                                                

  Focus groups                                                 

  Meeting participants                                         

  Reviewers                                                    

  Field test sites                                             

  Professional development                                     
  events and meetings                                          

  *Meeting Expenses*                                           

  Rooms for meetings                                           

  Equipment                                                    

  Refreshments                                                 

  *Materials and Supplies*                                     

  Reference materials                                          

  Supplies (folders,                                           
  markers, paper, etc.)                                        

  *Communication Expenses*                                     

  Software for online                                          
  meetings and sharing                                         
  documents                                                    

  Telephone                                                    

  Electronic survey tool                                       

  Postage                                                      

  Photocopying                                                 
  --------------------------------------------------------------------------

#  Moderator Guidelines for Focus Group Discussion

The focus group protocol covers the following seven steps.[^1] Focus
groups can be used for the environmental scan to gather information
about standards from stakeholders and for reviewing the standards
document.

## 1. ***Introduction***

Present a welcome statement, purpose of the project and the group, and
group guidelines or ground rules.

## 2. ***Warm Up***

Make brief introductions.

## 3. ***Clarification of Terms***

A. Explain any terms that might have multiple meanings to the
participants. Consult the glossary of this guide for ideas.

B. Ask for any final questions on format before beginning.

## 4. ***Questions***

Sequence general questions and follow-up probing questions that might be
used. General questions should precede more complex ones. Share
questions with co-moderators or aides. Do not try to cover too much
ground, and have a question or two in reserve for less vocal groups.

## 5. ***Wrap Up***

Summarize the major themes (ideas, concerns, recommendations, etc.) that
emerged from the conversation.

## 6. ***Member Check***

Ask for a quick prioritizing of the themes. Ask if anything is missing
that needs to be added.

## 7. ***Closing Statement***

Thank participants. Explain how the information will be used, and tell
them who they can contact for additional information. Remind
participants that the comments and responses should remain anonymous.

# Sample Focus Group Protocol

## Stakeholder Input: Practitioners: Instructors, Tutors, and Instructional Specialists

This sample focus group protocol may be used during the environmental
scan to gather information from instructors, tutors, and instructional
specialists. It could also be adapted for a focus group to review the
standards document. This sample protocol follows the same format as the
Moderator Guidelines for Focus Group Discussion.

## Introduction

Good afternoon, and welcome to our discussion today about developing and
implementing content standards. We want to thank you for joining us, and
we appreciate that you have taken time away from your busy schedule. My
name is \_\_\_\_\_\_\_\_\_\_\_\_\_\_, and I will be moderating this
discussion. Along with me is
\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_, who will be recording
this discussion. We are with the \[name of organization\].

This focus group is one of a series of groups we will be holding around
the state with different individuals who have a stake in adult
education. We also will be conducting focus groups with learners;
professional development staff; state-level program managers; local
program administrators; and representatives of the business community,
higher education community, and faith-based organizations. As we begin
the process of standards development, it is important to understand the
different perspectives of all stakeholders.

The purpose of this discussion group is to get your feedback on
establishing standards in adult education programs. I have a few
questions that I would like the group to discuss, and my role is to keep
the discussion on track. I encourage you to give your opinions freely,
as this information will help state staff set priorities for standards,
understand resources available, and think about future needs for
professional development. There is no right or wrong answer to any
question.

Before we begin, here are a few guidelines:

-   Please, only one person speaks at a time.

-   We want to hear from everyone, but not everyone needs to answer
    every question.

-   Although we are using first names here, no names will be attached to
    any comments. Your individual responses will be confidential. We ask
    that you help us honor this commitment by not repeating comments
    made by others outside of this group.

-   The discussion will last about one hour.

-   We have a full agenda, so I apologize in advance if I have to cut
    the discussions off at any point. I don't want to be impolite, but I
    may have to interrupt and bring us back to the main topic if we get
    too far afield. Or I may have to break in and move us along to the
    next questions so that we have time to go through all the topics we
    need to discuss.

## Warm Up

Let's start by introducing ourselves, giving your name, your program
area, and what you like best in your role as an adult educator. I will
begin. My name is \_\_\_\_\_\_\_\_\_\_\_. I taught low intermediate
level ABE and am now an instructional specialist in reading. What I like
best as a reading specialist is when learners tell me they can read and
understand the notices their children bring home from school. It makes
them feel proud of their abilities.

## Clarification of Terms

We are going to be talking about different kinds of standards, and we
need to be using a common vocabulary. To keep our communication clear,
I've posted the definitions we will be using today and that we are using
in our work groups. \[Direct attention to a posted list of words and
definitions. See the definitions from chapter 1 and appendix B.\]

Are there any questions about definitions or guidelines before we begin?

## Questions

1\. What have you heard about standards-based education?\
(PROBE: What are the goals of standards-based education? Do you have any
concerns about content standards?)

2\. What should the content standards in reading and mathematics include
to enable learners to be successful?\
(PROBE: As members of the community and within the family? For
employment? For education?)

3\. What kind of support or resources do you think will be needed to
implement content standards in your program?\
(PROBE: What professional development will be needed? What materials
will be needed?)

4\. What suggestions do you have for the process of developing/field
testing/implementing the content standards?\
(PROBE: Who would you include in the process? What should be the state
role?)

## Wrap Up

Unfortunately, we are almost out of time. I'd like to review what were
noted as the main points raised in this group. \[Summarize points.\]
Have I missed anything that you think is critical?

## Member Check

Can we quickly go around the room and do a check on how you would
prioritize this list? This is just a quick response. Tell us which two
issues you would identify as the most important from your perspective.

## Closing Statement

Thank you for participating in this discussion group. Your input has
been valuable for helping us move forward with standards-based
education. I want to remind you that we have committed ourselves to
anonymity of responses outside of the group. We appreciate your time and
want you to know that we will take all this information into account as
we move forward with our standards-base initiative.

# Indiana Adult Education Content Standards: Development Team Information and Application

Dear Professional Development Facilitator (PDF):

Research indicates that content or curricula must align with performance
expectations or outcomes to be effective. Content standards provide the
framework by defining the scope of what is to be taught. Using this
framework, adult educators can individualize curricula and the
instructional strategies to address the specific needs of adult
learners.

Twelve states have developed content standards for adult education.
Fifteen more, including Indiana**,** have been invited to participate in
the Content Standards Consortia for the purpose of drafting content
standards for their respective states in either ESL or ABE/ASC. Indiana
will be drafting content standards for ABE/ASC in Math and Reading
during the FY 2005 program year. Our participation in this new
initiative reflects our agreement with research-based information
regarding the importance of content standards. We will address ESL
standards in the near future.

The state team members driving this new initiative are Linda Warner,
Cindy Conway, and Lynne Ames. The first meeting of the Consortia took
place in Washington, D.C., October 7 and 8, 2004. The primary outcome of
that meeting was the development of a state-specific Content Standards
Action Plan. Indiana's plan is being provided for your review and
information.

States that have already engaged in the content standards development
process have learned many lessons. The Consortia states wish to benefit
from those lessons that, in turn, have become primary goals. *They
include but are not limited to:*

-   building consensus and engaging stakeholders

-   involving program personnel to include administrators, teachers, and
    students

-   reviewing the research base and being realistic about what's
    important in adult education

-   integrating content standards within ongoing adult education systems

-   creating a user-friendly document

We are establishing a content standards development (writing) team and
would like to offer you and the instructors in your program the
opportunity to be involved with this worthwhile project. Please complete
and/or distribute the attached application and submit as indicated.
Those participating on the development team will help draft Indiana's
Adult Education Content Standards for Reading and Math. The action plan
outlines five team meetings (2 or 3 days each) and may require some
additional personal time.

There will also be other opportunities as the project moves forward.
These include editing the draft standards and field-testing the
standards. Please discuss these with your administrator. Decide what
role or roles you and your program can play in this important
initiative.

# Indiana Adult Education Content Standards: Development Team Information and Application

#### I would like to serve on the content standards development (writing) team.

  --------------------------- -------------------------------------------
  Name:                       

  Program:                    

  Address:                    

  City, state, ZIP:           

  Telephone number:           

  E-mail address:             

  Number of years in adult    
  education:                  

  Areas of expertise/teaching 
  experience:                 

  (Use reverse side if        
  needed)                     

                              

                              

                              

  Other experience with       
  content area:               
  areastandards:              

  (Example: K--12, NCTM,      
  etc.)                       

                              

                              

  Please consider me for:     Math Standards Development -- Yes / No
                              (Circle one)

  Please consider me for:     Reading Standards Development -- Yes / No
                              (Circle one)

  Please consider me for:     Either Math or Reading -- Yes / No (Circle
                              one)

  The best day(s) for me to   M -- T -- W -- Th -- F -- Sat -- Sun --
  meet is/are:                (Circle all that apply)

                              

  **I am unable to serve on   
  the content standards       
  development (writing) team; 
  however ...**               

  Please consider me for:     Editing the Draft Standards -- Yes / No
                              (Circle one)

  Please consider my program  Field-Testing the Draft Standards -- Yes /
  for:                        No (Circle one)
  --------------------------- -------------------------------------------

#### 

#### Please complete and return to:

#### Name, Address, Phone

# Generic Content Standards Team Application

This form can be adapted for local and state use to recruit standards
team members.

## Contact Information

Name:

Address:

Phone:

E-mail:

Best days/times to contact:

## Employment in Adult Education or Related Field

Position and dates:

Major responsibilities:

## Relevant Education and Training (Include Content Area Expertise)

### Related Experience

A. Standards, curriculum, and assessment

B. Working on teams or on special initiatives

### Why are you interested in serving? What do you hope to gain and contribute during this initiative? 

### Availability (Timeframe fall 2005 through spring 2006)

Best days of the week for meetings:

Best times of the day for meetings:

# Criteria for Reviewing Content Standards Documents

**Purpose:** This tool is designed to help states and programs review
content standards. State planning and writing teams are encouraged to
use the tool to stimulate discussion about the criteria and to look for
specific examples within standards documents. A preliminary review of
other standards will help states make initial decisions about how their
standards will be conceived, written, and organized. Later, state
writing teams can use this tool to review and revise their draft
standards. External reviewers can also use the tool as they evaluate and
provide feedback on draft standards documents.

The following pages offer criteria for reviewing standards and
indicators. For each area, determine the extent to which the criteria
are present. Choices are: no evidence, some evidence, or substantial
evidence. The review may be conducted individually or as a team. If
conducted individually, come together as a team to share ratings and try
to reach consensus. This process will enable team members to discuss the
criteria and how important each standard or indicator is to their state.
Transfer the ratings to this cover page to facilitate comparison with
other standards document and to identify general strengths and
weaknesses.

State Publication date

Document name

Content area Levels

Reviewed by: Date:

To what extent are criteria evident? (Select rating)

  --------------------- --------------- --------------- -----------------
                        No evidence     Some evidence   Substantial
                                                        evidence

  **Rigorous**                                          

  **Specific**                                          

  **Comprehensive and                                   
  coherent**                                            

  **Measurable**                                        

  **Clear and                                           
  intelligible**                                        

  **Manageable**                                        
  --------------------- --------------- --------------- -----------------

Concerns and questions Useful features for the state

## Rigorous

Rigorous standards contain the essential concepts, knowledge, and skills
that can be applied in a variety of contexts. They reflect high-level
skills that will allow learners to meet the demands of the 21st century
and set high expectations for all learners at appropriate levels.

*In reviewing standards and indicators*, consider the following:

+-------------------------+-------------------------+-----+-----+-----+
| Content standards and   | Look for . . .          | No  | S   | Su  |
| indicators              |                         | ev  | ome | bst |
|                         |                         | ide | ev  | ant |
|                         |                         | nce | ide | ial |
|                         |                         |     | nce | ev  |
|                         |                         |     |     | ide |
|                         |                         |     |     | nce |
+-------------------------+-------------------------+-----+-----+-----+
| Require higher order    | -   The inclusion of    |     |     |     |
| skills.                 |     verbs such as       |     |     |     |
|                         |     analyze,            |     |     |     |
|                         |     synthesize,         |     |     |     |
|                         |     compare, contrast,  |     |     |     |
|                         |     generalize, etc.    |     |     |     |
+-------------------------+-------------------------+-----+-----+-----+
| Reflect the knowledge   | -   Skills and concepts |     |     |     |
| and skills necessary    |     that prepare adults |     |     |     |
| for adults to succeed   |     for success in the  |     |     |     |
| in the 21st century.    |     community, at home, |     |     |     |
|                         |     on the job, etc.    |     |     |     |
+-------------------------+-------------------------+-----+-----+-----+
| Include knowledge and   | -   Knowledge and       |     |     |     |
| skills necessary for    |     skills included in  |     |     |     |
| high school credential  |     GED, SCANS,         |     |     |     |
| and entry into          |     American Diploma    |     |     |     |
| postsecondary           |     Project, College    |     |     |     |
| education.              |     Placement tests,    |     |     |     |
|                         |     etc.                |     |     |     |
+-------------------------+-------------------------+-----+-----+-----+

Comments and useful features:

Concerns and questions:

## Specific

For any set of standards to guide instruction and the development of
curricula and assessments, the standards must be specific enough to
assure a common understanding of the expectations at each level.
Standards should provide sufficient contextual detail and a strong sense
of what learners and teachers are expected to do.

*In reviewing standards and indicators*, consider the following:

+------------------------+-------------------------------+---+---+---+
| Content standards and  | Look for . . .                | N | S | S |
| indicators             |                               | o | o | u |
|                        |                               | e | m | b |
|                        |                               | v | e | s |
|                        |                               | i | e | t |
|                        |                               | d | v | a |
|                        |                               | e | i | n |
|                        |                               | n | d | t |
|                        |                               | c | e | i |
|                        |                               | e | n | a |
|                        |                               |   | c | l |
|                        |                               |   | e | e |
|                        |                               |   |   | v |
|                        |                               |   |   | i |
|                        |                               |   |   | d |
|                        |                               |   |   | e |
|                        |                               |   |   | n |
|                        |                               |   |   | c |
|                        |                               |   |   | e |
+------------------------+-------------------------------+---+---+---+
| Provide sufficient     | -   Sample reading passages   |   |   |   |
| detail to guide        |     and math problems that    |   |   |   |
| curriculum development |     exemplify the type and    |   |   |   |
| and instruction.       |     complexity of the         |   |   |   |
|                        |     standards at each level   |   |   |   |
|                        |     of ABE, ASE, or ELA.      |   |   |   |
|                        |                               |   |   |   |
|                        | -   Statements about the      |   |   |   |
|                        |     essential knowledge and   |   |   |   |
|                        |     skills without dictating  |   |   |   |
|                        |     how to teach.             |   |   |   |
+------------------------+-------------------------------+---+---+---+
| Provide sufficient     | -   Example for a Beginning   |   |   |   |
| contextual detail and  |     ELA standard: "Provide    |   |   |   |
| a strong sense of what |     basic personal            |   |   |   |
| learners and teachers  |     information (name,        |   |   |   |
| are expected to do.    |     address, phone) orally    |   |   |   |
|                        |     and in writing."          |   |   |   |
|                        |                               |   |   |   |
|                        | -   Example for a Beginning   |   |   |   |
|                        |     mathematics standard:     |   |   |   |
|                        |     "Use words, numbers, or   |   |   |   |
|                        |     pictures to represent     |   |   |   |
|                        |     equivalent fractions."    |   |   |   |
+------------------------+-------------------------------+---+---+---+
| Include indicators or  | -   Active verbs and clear    |   |   |   |
| benchmarks with        |     expectations to create    |   |   |   |
| sufficient detail to   |     assessments.              |   |   |   |
| develop assessments    |                               |   |   |   |
| for classroom use and  |                               |   |   |   |
| for accountability.    |                               |   |   |   |
+------------------------+-------------------------------+---+---+---+

Comments and useful features:

Concerns and questions:

## Comprehensive and Coherent

A comprehensive set of standards reflects current research and has a
balanced focus on the essential knowledge in the content area without
any significant gaps. If standards are to lead to a common core of
learning across the state, they must provide a clear progression of
skills with increasing levels of difficulty.

*In reviewing standards and indicators*, consider the following:

+------------------------+-------------------------------+---+---+---+
| Content standards and  | Look for . . .                | N | S | S |
| indicators             |                               | o | o | u |
|                        |                               | e | m | b |
|                        |                               | v | e | s |
|                        |                               | i | e | t |
|                        |                               | d | v | a |
|                        |                               | e | i | n |
|                        |                               | n | d | t |
|                        |                               | c | e | i |
|                        |                               | e | n | a |
|                        |                               |   | c | l |
|                        |                               |   | e | e |
|                        |                               |   |   | v |
|                        |                               |   |   | i |
|                        |                               |   |   | d |
|                        |                               |   |   | e |
|                        |                               |   |   | n |
|                        |                               |   |   | c |
|                        |                               |   |   | e |
+------------------------+-------------------------------+---+---+---+
| Represent the breadth  | -   An inclusive set of       |   |   |   |
| and depth of the       |     standards with no obvious |   |   |   |
| knowledge and skills   |     gaps or omissions.        |   |   |   |
| within the discipline. |                               |   |   |   |
+------------------------+-------------------------------+---+---+---+
| Include both knowledge | -   Focus on knowledge and    |   |   |   |
| and skills.            |     skills (not irrelevant    |   |   |   |
|                        |     concepts and beliefs).    |   |   |   |
|                        |                               |   |   |   |
|                        | -   Example: Mathematics      |   |   |   |
|                        |     standards that focus on   |   |   |   |
|                        |     procedural knowledge,     |   |   |   |
|                        |     conceptual understanding, |   |   |   |
|                        |     and problem solving.      |   |   |   |
+------------------------+-------------------------------+---+---+---+
| Reflect current        | -   The underlying research   |   |   |   |
| education theory and   |     base that informed the    |   |   |   |
| research in the        |     development of the        |   |   |   |
| content area.          |     standards.                |   |   |   |
|                        |                               |   |   |   |
|                        | -   Example: Reading          |   |   |   |
|                        |     standards that include    |   |   |   |
|                        |     phonemic awareness,       |   |   |   |
|                        |     phonics, word analysis,   |   |   |   |
|                        |     vocabulary development,   |   |   |   |
|                        |     fluency, and              |   |   |   |
|                        |     comprehension.            |   |   |   |
+------------------------+-------------------------------+---+---+---+
| Follow a sound         | -   Standards that are        |   |   |   |
| progression of skills  |     distinct for each level   |   |   |   |
| and knowledge from     |     and are not redundant.    |   |   |   |
| level to level with    |                               |   |   |   |
| increasing levels of   |                               |   |   |   |
| difficulty.            |                               |   |   |   |
+------------------------+-------------------------------+---+---+---+

Comments and useful features:

Concerns and questions:

## Measurable

Content standards have to be sufficiently detailed to provide clear
expectations of what learners know and are able to do at different
levels. They specify results that can be measured and communicated to
stakeholders.

*In reviewing standards and indicators*, consider the following:

+------------------------+-------------------------------+---+---+---+
| Content standards and  | Look for . . .                | N | S | S |
| indicators             |                               | o | o | u |
|                        |                               | e | m | b |
|                        |                               | v | e | s |
|                        |                               | i | e | t |
|                        |                               | d | v | a |
|                        |                               | e | i | n |
|                        |                               | n | d | t |
|                        |                               | c | e | i |
|                        |                               | e | n | a |
|                        |                               |   | c | l |
|                        |                               |   | e | e |
|                        |                               |   |   | v |
|                        |                               |   |   | i |
|                        |                               |   |   | d |
|                        |                               |   |   | e |
|                        |                               |   |   | n |
|                        |                               |   |   | c |
|                        |                               |   |   | e |
+------------------------+-------------------------------+---+---+---+
| Reflect learning that  | -   Active verbs that focus   |   |   |   |
| can be observable or   |     on learning results such  |   |   |   |
| verifiable in a        |     as demonstrate, complete, |   |   |   |
| definable way.         |     explain, write,           |   |   |   |
|                        |     calculate, describe, etc. |   |   |   |
|                        |                               |   |   |   |
|                        | -   Process words that are    |   |   |   |
|                        |     measurable. Avoid words   |   |   |   |
|                        |     such as investigate,      |   |   |   |
|                        |     explore, participate, or  |   |   |   |
|                        |     listen.                   |   |   |   |
|                        |                               |   |   |   |
|                        | -   Verbs that can be         |   |   |   |
|                        |     measured precisely. Avoid |   |   |   |
|                        |     words such as increase,   |   |   |   |
|                        |     expand, extend, begin,    |   |   |   |
|                        |     etc.                      |   |   |   |
+------------------------+-------------------------------+---+---+---+
| Differentiate learning | -   Indicators or benchmarks  |   |   |   |
| to measure each        |     that require a            |   |   |   |
| learner's progress     |     demonstrated performance  |   |   |   |
| from level to level.   |     (ideally with criteria    |   |   |   |
|                        |     for rating or an          |   |   |   |
|                        |     exemplar).                |   |   |   |
+------------------------+-------------------------------+---+---+---+

Comments and useful features:

Concerns and questions:

## Clear and Intelligible

Content standards are meaningful to students, teachers, and the general
public when they are clearly written. Multiple audiences will use the
standards, so they need to send a straightforward message about what
students know and are able to do.

In reviewing standards and indicators, consider the following:

+------------------------+-------------------------------+---+---+---+
| Content standards and  | Look for . . .                | N | S | S |
| indicators             |                               | o | o | u |
|                        |                               | e | m | b |
|                        |                               | v | e | s |
|                        |                               | i | e | t |
|                        |                               | d | v | a |
|                        |                               | e | i | n |
|                        |                               | n | d | t |
|                        |                               | c | e | i |
|                        |                               | e | n | a |
|                        |                               |   | c | l |
|                        |                               |   | e | e |
|                        |                               |   |   | v |
|                        |                               |   |   | i |
|                        |                               |   |   | d |
|                        |                               |   |   | e |
|                        |                               |   |   | n |
|                        |                               |   |   | c |
|                        |                               |   |   | e |
+------------------------+-------------------------------+---+---+---+
| Use language that is   | -   Enough contextual detail  |   |   |   |
| written clearly enough |     so that multiple          |   |   |   |
| for all stakeholders   |     audiences can understand  |   |   |   |
| to understand.         |     what learners are         |   |   |   |
|                        |     expected to do.           |   |   |   |
|                        |                               |   |   |   |
|                        | -   Sample activities and     |   |   |   |
|                        |     passages that exemplify   |   |   |   |
|                        |     the type and complexity   |   |   |   |
|                        |     of knowledge and skills   |   |   |   |
|                        |     required at each level.   |   |   |   |
+------------------------+-------------------------------+---+---+---+
| Use language that is   | -   Minimal use of technical  |   |   |   |
| free of jargon.        |     language.                 |   |   |   |
+------------------------+-------------------------------+---+---+---+
| Use language that is   | -   Nondiscriminatory         |   |   |   |
| sensitive to all adult |     language.                 |   |   |   |
| learner populations    |                               |   |   |   |
| and is free of bias.   | -   Language and examples     |   |   |   |
|                        |     that reflect the          |   |   |   |
|                        |     diversity of adult        |   |   |   |
|                        |     learners in a positive    |   |   |   |
|                        |     manner.                   |   |   |   |
|                        |                               |   |   |   |
|                        | -   Language that does not    |   |   |   |
|                        |     exclude some groups due   |   |   |   |
|                        |     to style, grammar, or     |   |   |   |
|                        |     vocabulary choice.        |   |   |   |
+------------------------+-------------------------------+---+---+---+

Comments and useful features:

Concerns and questions:

## Manageable

Manageability of standards considers both the quantity and the
presentation of the standards. The number of standards must reflect what
is feasible to teach and learn within the time constraints of the adult
education system. Content standards need to be organized in a
user-friendly format for all stakeholders.

*In reviewing standards and indicators*, consider the following:

+------------------------+-------------------------------+---+---+---+
| Content standards and  | Look for . . .                | N | S | S |
| indicators             |                               | o | o | u |
|                        |                               | e | m | b |
|                        |                               | v | e | s |
|                        |                               | i | e | t |
|                        |                               | d | v | a |
|                        |                               | e | i | n |
|                        |                               | n | d | t |
|                        |                               | c | e | i |
|                        |                               | e | n | a |
|                        |                               |   | c | l |
|                        |                               |   | e | e |
|                        |                               |   |   | v |
|                        |                               |   |   | i |
|                        |                               |   |   | d |
|                        |                               |   |   | e |
|                        |                               |   |   | n |
|                        |                               |   |   | c |
|                        |                               |   |   | e |
+------------------------+-------------------------------+---+---+---+
| Include realistic      | -   Attainable number of      |   |   |   |
| scope and quantity for |     standards, indicators, or |   |   |   |
| each level given time  |     benchmarks for each       |   |   |   |
| constraints.           |     level.                    |   |   |   |
+------------------------+-------------------------------+---+---+---+
| Present information in | -   Definitions of terms and  |   |   |   |
| a user-friendly layout |     levels.                   |   |   |   |
| and format.            |                               |   |   |   |
|                        | -   Consistency in            |   |   |   |
|                        |     presentation (and         |   |   |   |
|                        |     possibly coding for easy  |   |   |   |
|                        |     referral to an            |   |   |   |
|                        |     indicator).               |   |   |   |
|                        |                               |   |   |   |
|                        | -   Tables of contents and    |   |   |   |
|                        |     page numbers.             |   |   |   |
+------------------------+-------------------------------+---+---+---+
| Are available and      | -   Electronic and print      |   |   |   |
| accessible in multiple |     versions.                 |   |   |   |
| formats.               |                               |   |   |   |
|                        | -   Accessibility for users   |   |   |   |
|                        |     with disabilities.        |   |   |   |
+------------------------+-------------------------------+---+---+---+

Comments and useful features:

Concerns and questions:

SAMPLE Content Standards Review Survey

# Kentucky Content Standards Review Survey

Please complete the following information prior to responding to the
survey. Duplicate for other staff members.

**County:**

**Your primary responsibilities (at least 50 percent of time). Please
check most applicable:**

\_\_\_\_\_ Instructor---what subject(s)

\_\_\_\_\_ Program Director

\_\_\_\_\_ Instructor's Aide

\_\_\_\_\_ Other (Please specify: )

**Your educational attainment. Please check highest educational
achievement:**

\_\_\_\_\_ High school diploma or GED

\_\_\_\_\_ Associate's degree

\_\_\_\_\_ Bachelor's degree

\_\_\_\_\_ Master's degree

\_\_\_\_\_ Doctorate degree

\_\_\_\_\_ Other (Please specify: )

## Overall Impression of the Kentucky Adult Education Content Standards

After reviewing the entire content standards document, please consider
the document as a whole and circle the response that best summarizes
your opinion about each of the following statements.

1\. The standards document is *written* in a manner that is easily
understood.

  ----------------- ----------------- ----------------- -----------------
                                                        

  4\                3\                2\                1\
  Strongly agree    Agree             Disagree          Strongly disagree
  ----------------- ----------------- ----------------- -----------------

> Comments (Why? or Why not?):

2\. The standards document is *formatted* in a manner that is easily
understood.

  ----------------- ----------------- ----------------- -----------------
                                                        

  4\                3\                2\                1\
  Strongly agree    Agree             Disagree          Strongly disagree
  ----------------- ----------------- ----------------- -----------------

> Comments (Why? or Why not?):

***3. The introduction to the standards document is useful.***

  ----------------- ----------------- ----------------- -----------------
                                                        

  4\                3\                2\                1\
  Strongly agree    Agree             Disagree          Strongly disagree
  ----------------- ----------------- ----------------- -----------------

> Comments (Why? or Why not?):

4\. The standards document is specific enough to guide curriculum
development and selection.

  ----------------- ----------------- ----------------- -----------------
                                                        

  4\                3\                2\                1\
  Strongly agree    Agree             Disagree          Strongly disagree
  ----------------- ----------------- ----------------- -----------------

> Comments (Why? or Why not?):

5\. The standards document is specific enough to guide instructional
strategies.

  ----------------- ----------------- ----------------- -----------------
                                                        

  4\                3\                2\                1\
  Strongly agree    Agree             Disagree          Strongly disagree
  ----------------- ----------------- ----------------- -----------------

> Comments (Why? or Why not?):

6\. The standards identify knowledge and skills required of and
appropriate for adults transitioning to skilled work and postsecondary
education.

  ----------------- ----------------- ----------------- -----------------
                                                        

  4\                3\                2\                1\
  Strongly agree    Agree             Disagree          Strongly disagree
  ----------------- ----------------- ----------------- -----------------

> Comments (Why? or Why not?):

7\. There are critical knowledge and skills missing from the standards
document.

  ----------------- ----------------- ----------------- -----------------
                                                        

  4\                3\                2\                1\
  Strongly agree    Agree             Disagree          Strongly disagree
  ----------------- ----------------- ----------------- -----------------

> Please identify missing standards:

8\. Resources/materials that you currently use will enable you to
implement these content standards.

  ----------------- ----------------- ----------------- -----------------
                                                        

  4\                3\                2\                1\
  Strongly agree    Agree             Disagree          Strongly disagree
  ----------------- ----------------- ----------------- -----------------

> Comments (Why? or Why not?):

9\. Curricula that you currently use will enable you to implement these
content standards.

  ----------------- ----------------- ----------------- -----------------
                                                        

  4\                3\                2\                1\
  Strongly agree    Agree             Disagree          Strongly disagree
  ----------------- ----------------- ----------------- -----------------

> Comments (Why? or Why not?):

10\. Assessments that you currently use will enable you to assess these
content standards.

  ----------------- ----------------- ----------------- -----------------
                                                        

  4\                3\                2\                1\
  Strongly agree    Agree             Disagree          Strongly disagree
  ----------------- ----------------- ----------------- -----------------

> Comments (Why? or Why not?):

## Guided Comments

1\. What *difficulties, concerns,* or *issues*, if any, did you incur
while reviewing the standards?

2\. What additional *aids/resources/appendices* would you suggest to
enhance the document?

3\. What additional *glossary and/or terms* would you like to see
defined?

4\. Rate yourself on a scale of 1 to 5 on your *comfort level* with
teaching the standards.\
("0" = Not at all à "5" = Very comfortable) Why?

5\. What types of *professional development and assistance* would you
like to have available upon implementation of the standards? Please
check what applies:

\_\_\_\_\_ Online courses

\_\_\_\_\_ Web sites

\_\_\_\_\_ Videos

\_\_\_\_\_ Books

\_\_\_\_\_ Resources (please specify)

\_\_\_\_\_ Sample lesson plans

\_\_\_\_\_ Visiting coaches/coaching visits

\_\_\_\_\_ Workshops with follow-up

\_\_\_\_\_ Other:

6\. Describe *how you will use* these standards.

## Other Comments

# Standards Usability Checklist Instructor Feedback Form

Instructors can complete this form during the validity and field test
reviews to assess the characteristics of the standards and identify
areas for standards-based professional development.

+-----------------------------------------------------------------------+
| Program Area:                                                         |
|                                                                       |
| Instructional Level:                                                  |
+-----------------------------------------------------------------------+

Are the standards ...

  ----- --------------------------------------------------- -------- --------
                                                            YES      NO

  1\.   Challenging to learners at this level?              \_\_\_   \_\_\_

  2\.   Specific enough to be addressed by classroom        \_\_\_   \_\_\_
        instruction?                                                 

  3\.   Easy to understand?                                 \_\_\_   \_\_\_

  4\.   Measurable by classroom assessments?                \_\_\_   \_\_\_

  5\.   Manageable within the classroom setting?            \_\_\_   \_\_\_

  6\.   Appropriate in scope for learners at this level?    \_\_\_   \_\_\_
  ----- --------------------------------------------------- -------- --------

  -----------------------------------------------------------------------
  What additional knowledge, skills, or resources do you think
  instructors will need to implement the standards?

  -----------------------------------------------------------------------

# Sample Implementation Timeline

Implementing content standards is not a single, distinct activity that
begins after planning, drafting, and reviewing standards. The following
timeline illustrates how a state might phase in standards-based
education over a three-year period. It outlines the policy decisions,
activities, and who might have lead responsibility for the activities.

+----------------+----------------------------+------------------------+
| **PHASE**      | **POLICIES AND             | **LEAD                 |
|                | ACTIVITIES**               | RESPONSIBILITY**       |
+================+============================+========================+
| **I            | **Policy Decisions:        | **State office staff** |
| mplementation\ | Approve draft content      |                        |
| Phase 1**      | standards, phase in across |                        |
|                | programs, and ensure state |                        |
|                | procedures are in place to |                        |
|                | begin implementation.**    |                        |
+----------------+----------------------------+------------------------+
| Quarter 1      | -   Update Strategic Plan  | -   State office staff |
|                |     and set aside          |                        |
|                |     incentive funding for  |                        |
|                |     full scale             |                        |
|                |     implementation of      |                        |
|                |     standards              |                        |
|                |                            |                        |
|                | -   Prepare standards      |                        |
|                |     document for state     |                        |
|                |     approval and           |                        |
|                |     dissemination          |                        |
|                |                            |                        |
|                | -   Update assessment      |                        |
|                |     policies to reflect    |                        |
|                |     standards-based reform |                        |
+----------------+----------------------------+------------------------+
| Quarter 2      | -   Prepare promotional    | -   State office staff |
|                |     information for        |     and Content        |
|                |     different audiences    |     Standards          |
|                |     (educators, learners,  |     Coordinating       |
|                |     business and community |     Committee          |
|                |     partners, legislators, |                        |
|                |     institutional boards)  |                        |
|                |                            |                        |
|                | -   Begin dissemination of |                        |
|                |     content standards      |                        |
|                |                            |                        |
|                | -   Identify sites and     |                        |
|                |     technical assistance   |                        |
|                |     needs to begin Phase 1 |                        |
|                |     implementation         |                        |
+----------------+----------------------------+------------------------+
| Quarter 3      | -   Provide professional   | -   Professional       |
|                |     development to program |     development        |
|                |     administrators and     |     providers          |
|                |     adult education staff  |                        |
|                |     through regional       | -   Local program      |
|                |     meetings/conferences,  |     administrators     |
|                |     online resources,      |                        |
|                |     training of trainers,  |                        |
|                |     mentors, etc.          |                        |
|                |                            |                        |
|                | -   Begin Phase 1          |                        |
|                |     implementation         |                        |
+----------------+----------------------------+------------------------+
| Quarter 4      | -   Develop curriculum and | -   Phase 1 site       |
|                |     resources with Phase 1 |     instructional      |
|                |     sites                  |     staff and          |
|                |                            |     professional       |
|                |                            |     development        |
|                |                            |     providers          |
+----------------+----------------------------+------------------------+
| **I            | **Policy Decisions:        | **State office staff** |
| mplementation\ | Provide incentive grants   |                        |
| Phase 2**      | for local programs to      |                        |
|                | voluntarily implement      |                        |
|                | content standards.**       |                        |
+----------------+----------------------------+------------------------+
| Quarter 1      | -   Repeat Phase 1         | -   Professional       |
|                |     professional           |     development        |
|                |     development for        |     providers          |
|                |     additional programs to |                        |
|                |     initiate               | -   Local              |
|                |     standards-based        |     instructional      |
|                |     education              |     staff              |
|                |                            |                        |
|                | -   Create resource        |                        |
|                |     exchange for           |                        |
|                |     curricula, lessons,    |                        |
|                |     assessments, and       |                        |
|                |     materials              |                        |
+----------------+----------------------------+------------------------+
| Quarter 2      | -   Continue professional  | -   Professional       |
|                |     development            |     development        |
|                |                            |     providers          |
|                | -   Conduct site visits to |                        |
|                |     gather information and | -   State office staff |
|                |     provide technical      |                        |
|                |     assistance             |                        |
+----------------+----------------------------+------------------------+
| Quarter 3      | -   Prepare RFP/grant      | -   State office staff |
|                |     applications mandating |                        |
|                |     full integration of    | -   State office staff |
|                |     content standards      |                        |
|                |     across the state       | -   Professional       |
|                |                            |     development        |
|                | -   Conduct statewide      |     providers          |
|                |     electronic survey on   |                        |
|                |     the use of the         |                        |
|                |     standards              |                        |
|                |                            |                        |
|                | -   Conduct focus groups   |                        |
|                |     as part of site visits |                        |
|                |     and technical          |                        |
|                |     assistance to identify |                        |
|                |     future professional    |                        |
|                |     development needs and  |                        |
|                |     monitoring strategies  |                        |
+----------------+----------------------------+------------------------+
| Quarter 4      | -   Analyze evaluations    | -   State office staff |
|                |     and data from          |                        |
|                |     professional           | -   Professional       |
|                |     development, surveys,  |     development        |
|                |     focus groups, and      |     providers          |
|                |     technical assistance   |                        |
+----------------+----------------------------+------------------------+
| **I            | **Policy Decision: Require | **State office staff** |
| mplementation\ | all programs to adopt      |                        |
| Phase 3 Full   | content standards in new   |                        |
| Scale**        | grant award cycle.**       |                        |
+----------------+----------------------------+------------------------+
| Quarter 1      | -   Offer Summer           | -   Professional       |
|                |     Institute: possible    |     development        |
|                |     topics (1)             |     providers          |
|                |     differentiating        |                        |
|                |     professional           |                        |
|                |     development for new    |                        |
|                |     and experienced staff  |                        |
|                |     and (2) monitoring     |                        |
|                |     standards              |                        |
|                |     implementation         |                        |
+----------------+----------------------------+------------------------+
| Quarter 2      | -   Begin program          | -   State office staff |
|                |     monitoring for Phase 1 |                        |
|                |     sites                  | -   Professional       |
|                |                            |     development        |
|                | -   Continue professional  |     providers          |
|                |     development            |                        |
+----------------+----------------------------+------------------------+
| Quarter 3      | -   Continue technical     | -   State office staff |
|                |     assistance and         |                        |
|                |     resource development   | -   Professional       |
|                |                            |     development        |
|                |                            |     providers          |
+----------------+----------------------------+------------------------+
| Quarter 4      | -   Conduct preliminary    | -   State office staff |
|                |     evaluation of the      |                        |
|                |     implementation to plan |                        |
|                |     mid-course corrections |                        |
+----------------+----------------------------+------------------------+

# Arizona Implementation Timeline

<table>
<colgroup>
<col style="width: 15%" />
<col style="width: 3%" />
<col style="width: 3%" />
<col style="width: 3%" />
<col style="width: 3%" />
<col style="width: 3%" />
<col style="width: 3%" />
<col style="width: 3%" />
<col style="width: 3%" />
<col style="width: 3%" />
<col style="width: 3%" />
<col style="width: 3%" />
<col style="width: 3%" />
<col style="width: 3%" />
<col style="width: 3%" />
<col style="width: 3%" />
<col style="width: 3%" />
<col style="width: 3%" />
<col style="width: 3%" />
<col style="width: 3%" />
<col style="width: 3%" />
<col style="width: 3%" />
<col style="width: 3%" />
<col style="width: 3%" />
<col style="width: 3%" />
<col style="width: 3%" />
<col style="width: 3%" />
<col style="width: 3%" />
<col style="width: 3%" />
</colgroup>
<thead>
<tr class="header">
<th colspan="29"><p>ADE = Arizona Department of Education ECE = early
childhood education PS = performance standards TA = technical
assistance</p>
<p>AE = adult education NRS = National Reporting System S/SS =
science/social science tech = technology</p>
<p>CS = content standards PD = professional development SBE =
standards-based education</p></th>
</tr>
<tr class="odd">
<th><strong> </strong></th>
<th colspan="4"><strong>1998</strong></th>
<th colspan="4"><strong>1999</strong></th>
<th colspan="4"><strong>2000</strong></th>
<th colspan="4"><strong>2001</strong></th>
<th colspan="4"><strong>2002</strong></th>
<th colspan="4"><strong>2003</strong></th>
<th colspan="4"><strong>2004</strong></th>
</tr>
</thead>
<tbody>
<tr class="odd">
<td><strong>Policy Decisions</strong></td>
<td><strong>Q1</strong></td>
<td><strong>Q2</strong></td>
<td><strong>Q3</strong></td>
<td><strong>Q4</strong></td>
<td><strong>Q1</strong></td>
<td><strong>Q2</strong></td>
<td><strong>Q3</strong></td>
<td><strong>Q4</strong></td>
<td><strong>Q1</strong></td>
<td><strong>Q2</strong></td>
<td><strong>Q3</strong></td>
<td><strong>Q4</strong></td>
<td><strong>Q1</strong></td>
<td><strong>Q2</strong></td>
<td><strong>Q3</strong></td>
<td><strong>Q4</strong></td>
<td><strong>Q1</strong></td>
<td><strong>Q2</strong></td>
<td><strong>Q3</strong></td>
<td><strong>Q4</strong></td>
<td><strong>Q1</strong></td>
<td><strong>Q2</strong></td>
<td><strong>Q3</strong></td>
<td><strong>Q4</strong></td>
<td><strong>Q1</strong></td>
<td><strong>Q2</strong></td>
<td><strong>Q3</strong></td>
<td><strong>Q4</strong></td>
</tr>
<tr class="even">
<td>Adopt CS</td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td>x</td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
</tr>
<tr class="odd">
<td>Release CS</td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td>x</td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
</tr>
<tr class="even">
<td>Realign levels with NRS</td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td>x</td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
</tr>
<tr class="odd">
<td>Deadline to implement SBE</td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td>x</td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
</tr>
<tr class="even">
<td>Assessment policy criteria</td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td>x</td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
</tr>
<tr class="odd">
<td>Implement ECE standards</td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td>x</td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
</tr>
<tr class="even">
<td>Deadline for curriculum alignment (S/SS)</td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td>x</td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
</tr>
<tr class="odd">
<td>Implement assessment policy</td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td>x</td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
</tr>
<tr class="even">
<td>Implement tech CS</td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td>x</td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
</tr>
<tr class="odd">
<td>Purchase AE assessments</td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td>x</td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
</tr>
<tr class="even">
<td><strong>Development/Writing</strong></td>
<td><strong>Q1</strong></td>
<td><strong>2</strong></td>
<td><strong>3</strong></td>
<td><strong>4</strong></td>
<td><strong>1</strong></td>
<td><strong>2</strong></td>
<td><strong>3</strong></td>
<td><strong>4</strong></td>
<td><strong>1</strong></td>
<td><strong>2</strong></td>
<td><strong>3</strong></td>
<td><strong>4</strong></td>
<td><strong>1</strong></td>
<td><strong>2</strong></td>
<td><strong>3</strong></td>
<td><strong>4</strong></td>
<td><strong>1</strong></td>
<td><strong>2</strong></td>
<td><strong>3</strong></td>
<td><strong>4</strong></td>
<td><strong>1</strong></td>
<td><strong>2</strong></td>
<td><strong>3</strong></td>
<td><strong>4</strong></td>
<td><strong>1</strong></td>
<td><strong>2</strong></td>
<td><strong>3</strong></td>
<td><strong>4</strong></td>
</tr>
<tr class="odd">
<td>Writing team CS<br />
(5 areas)</td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
</tr>
<tr class="even">
<td>Design team</td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
</tr>
<tr class="odd">
<td>Focus groups (S/SS)</td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td>x</td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
</tr>
<tr class="even">
<td>Market (ongoing team)</td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
</tr>
<tr class="odd">
<td>PS</td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
</tr>
<tr class="even">
<td>Focus groups (PS)</td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td>x</td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
</tr>
<tr class="odd">
<td>Writing team (S/SS)</td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
</tr>
<tr class="even">
<td>Writing team (ECE)</td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td>x</td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
</tr>
<tr class="odd">
<td>Assessment team (ECE)</td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
</tr>
<tr class="even">
<td>Writing team (tech)</td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
</tr>
<tr class="odd">
<td>Focus groups (tech)</td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
</tr>
<tr class="even">
<td>Revise ELAA CS</td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
</tr>
<tr class="odd">
<td>Assessment team (ELAA)</td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td>x</td>
</tr>
<tr class="even">
<td>Revise CS (6 areas)</td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
</tr>
<tr class="odd">
<td><strong>Professional Development</strong></td>
<td><strong>Q1</strong></td>
<td><strong>2</strong></td>
<td><strong>3</strong></td>
<td><strong>4</strong></td>
<td><strong>1</strong></td>
<td><strong>2</strong></td>
<td><strong>3</strong></td>
<td><strong>4</strong></td>
<td><strong>1</strong></td>
<td><strong>2</strong></td>
<td><strong>3</strong></td>
<td><strong>4</strong></td>
<td><strong>1</strong></td>
<td><strong>2</strong></td>
<td><strong>3</strong></td>
<td><strong>4</strong></td>
<td><strong>1</strong></td>
<td><strong>2</strong></td>
<td><strong>3</strong></td>
<td><strong>4</strong></td>
<td><strong>1</strong></td>
<td><strong>2</strong></td>
<td><strong>3</strong></td>
<td><strong>4</strong></td>
<td><strong>1</strong></td>
<td><strong>2</strong></td>
<td><strong>3</strong></td>
<td><strong>4</strong></td>
</tr>
<tr class="even">
<td>PD team</td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
</tr>
<tr class="odd">
<td>Curriculum alignment workshop</td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td>x</td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
</tr>
<tr class="even">
<td>Summer instititute</td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td>x</td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
</tr>
<tr class="odd">
<td>GED institute</td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
</tr>
<tr class="even">
<td>Focus groups (ECE)</td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td>x</td>
<td></td>
<td>x</td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
</tr>
<tr class="odd">
<td>Assessment workshops</td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
</tr>
<tr class="even">
<td>Local directors training</td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td>x</td>
<td></td>
<td></td>
<td></td>
<td>x</td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
</tr>
<tr class="odd">
<td><strong>Implementation</strong></td>
<td><strong>Q1</strong></td>
<td><strong>2</strong></td>
<td><strong>3</strong></td>
<td><strong>4</strong></td>
<td><strong>1</strong></td>
<td><strong>2</strong></td>
<td><strong>3</strong></td>
<td><strong>4</strong></td>
<td><strong>1</strong></td>
<td><strong>2</strong></td>
<td><strong>3</strong></td>
<td><strong>4</strong></td>
<td><strong>1</strong></td>
<td><strong>2</strong></td>
<td><strong>3</strong></td>
<td><strong>4</strong></td>
<td><strong>1</strong></td>
<td><strong>2</strong></td>
<td><strong>3</strong></td>
<td><strong>4</strong></td>
<td><strong>1</strong></td>
<td><strong>2</strong></td>
<td><strong>3</strong></td>
<td><strong>4</strong></td>
<td><strong>1</strong></td>
<td><strong>2</strong></td>
<td><strong>3</strong></td>
<td><strong>4</strong></td>
</tr>
<tr class="even">
<td>Focus groups (assessment)</td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td>x</td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
</tr>
<tr class="odd">
<td>TA from ADE</td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
</tr>
<tr class="even">
<td>Standards specialists</td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
</tr>
<tr class="odd">
<td>Assessment teams</td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
</tr>
<tr class="even">
<td>Assessment recommendations</td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td>x</td>
<td></td>
<td></td>
<td></td>
<td>x</td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
</tr>
<tr class="odd">
<td>Assessment training (resource and ADE staff)</td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td>x</td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
</tr>
<tr class="even">
<td>Implement standardized assessment</td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td>x</td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
</tr>
<tr class="odd">
<td>Implement tech standards</td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td>x</td>
</tr>
</tbody>
</table>

#  

# Sample Action Plan---Implementation

This plan provides an example of steps that might be include in a state
implementation plan.

Date developed/updated:

Objective(s):

+--------------------+---------+--------+--------+-------+----------+
| Key Action Steps\  | Pe      | Pot    | Pro    | Proj  | Ch       |
| *(with subtasks)*  | rson(s) | ential | jected | ected | allenges |
|                    | Resp    | C      | Ou     | Timef | and      |
|                    | onsible | ollabo | tcomes | rame\ | S        |
|                    |         | rators |        | *     | olutions |
|                    |         | and    |        | (star |          |
|                    |         | Res    |        | t/end |          |
|                    |         | ources |        | da    |          |
|                    |         |        |        | tes)* |          |
+--------------------+---------+--------+--------+-------+----------+
| **Communicate      | State   | Coordi | State  | Ja    | Re       |
| goals for          | d       | nating | and    | nuary | sistance |
| standards-based    | irector | com    | local  | to    | to       |
| education**        |         | mittee | p      | March | change,  |
|                    |         | me     | rogram |       | a)       |
| -   Develop        |         | mbers, | staff  |       | Provide  |
|     communication  |         | profes | unde   |       | prof     |
|     strategies     |         | sional | rstand |       | essional |
|                    |         | devel  | goals  |       | dev      |
| -   Develop        |         | opment | and    |       | elopment |
|     strategies for |         | staff, | can    |       | to       |
|     dissemination  |         | w      | plan   |       | discuss  |
|     of standards   |         | riting | for    |       | change   |
|                    |         | team   | im     |       | process. |
|                    |         | m      | plemen |       |          |
|                    |         | embers | tation |       | b\)      |
|                    |         |        |        |       | Provide  |
|                    |         |        |        |       | support  |
|                    |         |        |        |       | to       |
|                    |         |        |        |       | programs |
+--------------------+---------+--------+--------+-------+----------+
| **Assess           |         |        |        |       |          |
| professional       |         |        |        |       |          |
| development        |         |        |        |       |          |
| needs**            |         |        |        |       |          |
|                    |         |        |        |       |          |
| -                  |         |        |        |       |          |
| -                  |         |        |        |       |          |
+--------------------+---------+--------+--------+-------+----------+
| **Develop and      |         |        |        |       |          |
| adapt professional |         |        |        |       |          |
| development        |         |        |        |       |          |
| modules**          |         |        |        |       |          |
|                    |         |        |        |       |          |
| -                  |         |        |        |       |          |
| -                  |         |        |        |       |          |
+--------------------+---------+--------+--------+-------+----------+
| **Provide          |         |        |        |       |          |
| technical          |         |        |        |       |          |
| assistance**       |         |        |        |       |          |
|                    |         |        |        |       |          |
| -                  |         |        |        |       |          |
| -                  |         |        |        |       |          |
+--------------------+---------+--------+--------+-------+----------+
| **Monitor local    |         |        |        |       |          |
| implementation of  |         |        |        |       |          |
| standards-based    |         |        |        |       |          |
| education**        |         |        |        |       |          |
|                    |         |        |        |       |          |
| -                  |         |        |        |       |          |
| -                  |         |        |        |       |          |
+--------------------+---------+--------+--------+-------+----------+

Appendix D

References and Resources

Adult Education Content Standards Warehouse.
<http://www.adultedcontentstandards.ed.gov/>.

American Council on Education. GED Testing Service. 1999. *Alignment of
national and state standards.* Washington, DC: American Council on
Education.

American Diploma Project, an initiative of Achieve, Inc., the Education
Trust, and the Thomas B. Fordham Foundation. n.d. *Achieve initiatives:
American diploma project.* <http://www.achieve.org/node/604> (accessed
December 2007).

American Federation of Teachers. 2003. *Setting strong standards*
(Educational Issues Department No. 39-0175). Washington, DC: American
Federation of Teachers.
<http://www.aft.org/pubs-reports/downloads/teachers/settingstrong.pdf>
(accessed December 2007).

Ananda, S. 2000. *Equipped for the Future assessment report: How
instructors can support adult learners through performance-based
assessments* (ERIC No. ED 443999)*.* Washington, DC: National Institute
for Literacy. <http://eff.cls.utk.edu/PDF/ananda_eff.pdf> (accessed
December 2007).

Arizona Department of Education. Adult Education Services. 2004.
*English language acquisition for adults (ELAA) standards.*
<http://www.ade.state.az.us/adult-ed/adult_ed_standards.asp> (accessed
December 2007).

Black, P., C. Harrison, C. Lee, B. Marshall, and D. Wiliam. 2003.
*Assessment for learning.* Berkshire, England: Open University Press.

Black, P., and D. Wiliam. 1998. Inside the black box: Raising standards
through classroom assessment. *Phi Delta Kappan.*
<http://pdkintl.org/kappan/kbla9810.htm> (accessed December 2007).

Carnevale, A. P., and D. M. Desrochers. 2003. *Standards for what? The
economic roots of K--16 reform.* Princeton, NJ: Educational Testing
Service.

Carr, J., and D. Harris. 2001. *Succeeding with standards: Linking
curriculum, assessment, and action planning.* Alexandria, VA:
Association for Supervision and Curriculum Development.

CASAS National Consortium. 2005. *Comprehensive Adult Student Assessment
System (CASAS).* <http://www.casas.org/home/index.cfm> (accessed
December 2007).

Celce-Murcia, M., ed. 2001. *Teaching English as a second language.*
Boston, MA: Heinle and Heinle.

Cochran-Smith, M., and S. L. Lytle. 1999. The teacher research movement:
A decade later. *Educational Researcher,* 28 (7): 15--25.

Cohen, D. K., and H. C. Hill. 2000. Instructional policy and classroom
performance: The mathematics reform in California. *Teachers College
Record,* 102 (2), 294--343.
<http://www.tcrecord.org/content.asp?contentid=10377> (accessed December
2007).

Comings, J., H. Beder, S. Reder, B. Bingman, and C. Smith. 2003.
*Establishing an evidence-based adult education system.* Cambridge MA:
National Center for the Study of Adult Literacy and Learning.
<http://www.ncsall.net/index.php?id=26> (accessed December 2007).

Consortium for Policy Research in Education (CPRE). 1993. *Developing
content standards: Creating a process for change.* Philadelphia, PA:
University of Pennsylvania, CPRE (including Rutgers, the State
University of New Jersey; University of Wisconsin--Madison; Harvard
University; Stanford University; University of Michigan).

Council of Chief State School Officers (CCSSO). 2004. *Criteria for
judging the quality of content standards.*
<http://www.ed.gov/policy/elsec/guid/standardsassessment/crjudge.html>
(accessed December 2007).

Council of Chief State School Officers (CCSSO). 2003. *State content
standards: A 50-state resource*. Washington, DC: CCSSO.
<http://www.ccsso.org/content/pdfs/StateContentStandards.pdf> (accessed
December 2007).

Council of Chief State School Officers (CCSSO). 1997. *Criteria for
alignment of expectations and assessments in mathematics and science
education.* Washington, DC: CCSSO.
[http://www.ccsso.org](http://www.ccsso.org/) (accessed December 2007).

Council of Chief State School Officers (CCSSO). n.d. *Alignment
analysis.* <http://www.ccsso.org/Projects/Alignment_Analysis> (accessed
December 2007).

Council of Chief State School Officers (CCSSO). n.d. *Survey of enacted
curriculum: Tools for aligning instruction, standards, and assessments.*
[http://www.ccsso.org/projects/\
Surveys_of_Enacted_Curriculum/](http://www.ccsso.org/projects/Surveys_of_Enacted_Curriculum/)
(accessed December 2007).

Darling-Hammond, L. 2003. Standards and assessments: Where we are and
what we need. *Teachers College Record,* February 16.
<http://www.tcrecord.org/Content.asp?ContentID=11109> (accessed December
2007).

Darling-Hammond, L. 1999. *Teacher quality and student achievement: A
review of state policy evidence.* Seattle, WA: University of Washington,
Center for the Study of Teaching and Policy.
<http://depts.washington.edu/ctpmail/PDFs/LDH_1999.pdf> (accessed
December 2007).

Darling-Hammond, L., and G. Sykes, eds. 1999. *Teaching as the learning
profession: Handbook of policy and practice.* San Francisco:
Jossey-Bass.

Doyle, D., and S. Pimentel. 1999. *Raising the standard: An eight-step
guide for schools and communities*. 2nd ed. Thousand Oaks, CA: Corwin
Press.

Dutro, E., and S. W. Valencia. 2004. *The relation between state and
district literacy standards: Issues of alignment, influence, and
utility.* Seattle, WA: University of Washington, Center for the Study of
Teaching and Policy.
<http://depts.washington.edu/ctpmail/PDFs/LitStandards-EDSV-01-04.pdf>
(accessed December 2007).

Elmore, R. F. 2002. *Bridging the gap between standards and achievement:
The imperative for professional development in education.* Washington,
DC: Albert Shanker Institute.
<http://www.shankerinstitute.org/Downloads/Bridging_Gap.pdf> (accessed
December 2007).

Equipped for the Future. n.d. *Special collection---Content standards.*
<http://eff.cls.utk.edu/resources/home.htm> (accessed December 2007).

Erikson, H. L. 2003. *Concept-based curriculum and instruction: Teaching
beyond the facts*. Thousand Oaks, CA: Corwin Press.

Garet, M. S., A. C. Porter, L. Desimone, B. F. Birman, and K. S. Yoon.
2001. What makes professional development effective? Results form a
national sample of teachers. *American Educational Research Journal,* 38
(4), 915--945. <http://aztla.asu.edu/ProfDev1.pdf> (accessed December
2007).

Jones, J. 1996. *The standards movement: Past and present*. Milwaukee,
WI: Parents Raising Educational Standards in Schools.

Kohn, A. 2000. *The case against standardized testing: Raising the
scores, ruining the schools*. Portsmouth, NH: Heinemann.

Kruidenier, J. 2002. *Research-based principles for adult basic
education reading instruction.* Portsmouth, NH: RMC Research
Corporation. The Partnership for Reading, National Institute for
Literacy.
<http://www.nifl.gov/partnershipforreading/publications/html/adult_ed>
(accessed December 2007).

Kutner, M., R. Sherman, J. Tibbetts, and L. Condelli. 1997. *Evaluating
professional development: A framework for adult education.* Washington,
DC: PRO-NET (Office of Vocational and Adult Education and the Pelavin
Research Center). <http://www.calpro-online.org/pubs/evalmon.pdf>
(accessed December 2007).

Loucks-Horsley, S., P.W. Hewson, N. Love, and K. E. Stiles. 1998.
*Designing professional development for teachers of science and
mathematics.* Thousand Oaks, CA: Corwin Press.

Maryland Adult Literacy Resource Center at UMBC. 2003. *Adult ESOL
content standards.* <http://umbc.edu/alrc/ESOLCS405.pdf> (accessed
December 2007).

Marzano, R. J., and J. S. Kendall. 1998. *Awash in a sea of standards*.
Aurora, CO: Mid-continent Research for Education and Learning.
[http://www.mcrel.org/topics/products/120](http://www.mcrel.org/topics/products/120/)/
(accessed December 2007).

Marzano, R. J., and J. S. Kendall. 1997. *The fall and rise of
standards-based education:* *A National Association of State Boards of
Education (NASBE) Issues in Brief.*
<http://www.mcrel.org/topics/Standards/products/125/> (accessed December
2007).

Massachusetts Department of Education. Adult and Community Learning
Services. 2001. *Massachusetts adult basic education curriculum
frameworks for mathematics and numeracy.*
<http://www.doe.mass.edu/acls/frameworks> (accessed December 2007).

Massell, D. 1998. *State strategies for building local capacity:
Addressing the needs of standards-based reform* (CPRE Policy Brief
RB-25). Philadelphia, PA: Graduate School of Education, University of
Pennsylvania. Consortium for Policy Research in Education (CPRE).
<http://www.cpre.org/images/stories/cpre_pdfs/rb25.pdf> (accessed
December 2007).

McNamara, C. n.d. *Basic guide to program evaluation*. St. Paul, MN:
Management Assistance Program for Nonprofits.
<http://www.mapnp.org/library/evaluatn/fnl_eval.htm> (accessed December
2007).

Merrifield, J. 1998. *Contested ground: Performance accountability in
adult basic education.* NCSALL report no. 1. Cambridge, MA: National
Center for the Study of Adult Literacy and Learning (NCSALL).
<http://www.ncsall.net/index.php?id=29> (accessed December 2007).

Merrow, J. 2001. Undermining standards. *Phi Delta Kappan,* May,
652--659.

Mid-continent Research for Education and Learning. n.d. *Content
knowledge: 4th edition standards & benchmarks.*
<http://www.mcrel.org/standards-benchmarks> (accessed December 2007).

Morgan, D. L., and R. A. Krueger. 1997. *The Focus Group Kit.* 6 vols.
Thousand Oaks, CA: Sage Publications, Inc.

National Center for the Study of Adult Learning and Literacy. 2003.
Curriculum development. *Focus on Basics,* 6 (C).
<http://www.ncsall.net/index.php?id=153> (accessed December 2007).

National Center for the Study of Adult Learning and Literacy. n.d.
*Teaching and training materials: Improving practice* (Guides for study
circles, mentoring, and inquiry).
<http://www.ncsall.net/index.php?id=25> (accessed December 2007).

National Center for the Study of Adult Learning and Literacy.
Publications, Reports, and *Focus on Basics*.
[http://www.ncsall.net](http://www.ncsall.net/) (accessed December
2007).

National Center on Education and the Economy. 1990. *America's choice:
High skills or low wages?* Washington, DC: National Center on Education
and the Economy.

National Center on Student Progress Monitoring.
[http://www.studentprogress.org](http://www.studentprogress.org/)
(accessed December 2007).

National Commission on Excellence in Education. 1983. *A nation at
risk*: *The imperative for educational reform* (Publication no. ED 224
839). Washington, DC: Government Printing Office.

National Council of Teachers of English (NCTE), and International
Reading Association (IRA). 1996. *Standards for the English language
arts.* Urbana, IL: NCTE.

National Council of Teachers of Mathematics (NCTM). 2000. *Principles
and standards for school mathematics.* Reston, VA: NCTM.

National Council of Teachers of Mathematics (NCTM). 1991. *Professional
standards for teaching mathematics*. Reston, VA: NCTM.

National Education Goals Panel. 1997. *Recommendations regarding the
implementation of standards.* <http://govinfo.library.unt.edu/negp>
(accessed December 2007).

National Education Goals Panel. 1991. *The National Education Goals
Panel report: Building a nation of learners*. Washington, DC: National
Education Goals Panel. <http://govinfo.library.unt.edu/negp> (accessed
December 2007).

National Education Goals Panel. n.d. *Guide to getting out your
message*. Washington, DC: National Education Goals Panel.
<http://govinfo.library.unt.edu/negp/Reports/getting.pdf> (accessed
December 2007).

National Institute for Literacy. *Adult Reading Components Study (ARCS):
Mini course in reading.*
<http://www.nifl.gov/readingprofiles/MC_Intro.htm> (accessed December
2007).

National Institute for Literacy. n.d. *Assessment strategies and reading
profiles.* <http://www.nifl.gov/readingprofiles> (accessed December
2007).

National Reading Panel. 2000. *Teaching children to read: An
evidenced-based assessment of the scientific research literature on
reading and its implications for reading instruction.* Washington, DC:
National Institute of Child Health and Human Development.
<http://www.nichd.nih.gov/publications/nrp/smallbook.cfm> (accessed
December 2007).

National Reporting System for Adult Education (NRS).
[http://www.nrsweb.org](http://www.nrsweb.org/) (accessed December
2007).

National Research Council. 1999. *Testing, teaching, and learning: A
guide for states and school districts.* Ed. R. F. Elmore and R. Rothman,
Committee on Title I Testing and Assessment, Board on Testing and
Assessment, Commission on Behavioral and Social Sciences and Education.
Washington, DC: National Academy Press.

New York State Education Department, and the State University New York.
1997. *Adult education resource guide and learning standards*. Albany,
NY: New York State Education Department.
<http://www.nald.ca/fulltext/hudson/adult_ed/adedres.pdf> (accessed
December 2007).

No Child Left Behind Act. 2001. (PL 107-110).
<http://www.ed.gov/nclb/overview/intro/execsumm.html> (summary);
<http://www.ed.gov/policy/elsec/leg/esea02/index.html> (full text)
(accessed December 2007).

North Central Regional Educational Laboratory (NCREL). 2000. *Critical
Issue: Integrating standards into the curriculum.*
<http://www.ncrel.org/sdrs/areas/issues/content/currclum/cu300.htm>
(accessed December 2007).

Ohio Department of Education. Office of Career-Technical and Adult
Education. 2004. *Sample learning activities: Connecting Ohio's revised
math, reading and writing standards to student performance.*
[http://www.ode.state.oh.us/GD/Templates/Pages/ODE/ODEDetail.aspx?page=3&TopicRelation\
ID=966&Content=13630](http://www.ode.state.oh.us/GD/Templates/Pages/ODE/ODEDetail.aspx?page=3&TopicRelationID=966&Content=13630)
(accessed December 2007).

Ohio Department of Education. Office of Career-Technical and Adult
Education. 2003. *Ohio ABLE standards and benchmarks and reporting
forms.*
[http://www.ode.state.oh.us/GD/Templates/Pages/ODE/ODEPrimary.aspx?page=2&Topic\
RelationID=155](http://www.ode.state.oh.us/GD/Templates/Pages/ODE/ODEPrimary.aspx?page=2&TopicRelationID=155)
(accessed December 2007).

Ohio Department of Education. Office of Career-Technical and Adult
Education. 2003. *Ohio ABLE standards implementation portfolio---Summer
2003.*
[http://www.ode.state.oh.us/GD/Templates/Pages/ODE/ODEDetail.aspx?page=3&TopicRelation\
ID=966&Content=13171](http://www.ode.state.oh.us/GD/Templates/Pages/ODE/ODEDetail.aspx?page=3&TopicRelationID=966&Content=13171)
(accessed December 2007).

Ohio Department of Education. Office of Career-Technical and Adult
Education. 2003. *Standards institute orientation* (Standards Institute
Presentations).
<http://www.ode.state.oh.us/GD/DocumentManagement/DocumentDownload.aspx?DocumentID=13206>
(Page 17) (accessed December 2007).

Ohio Department of Education. Office of Career-Technical and Adult
Education. n.d. *Adult basic and literacy education: ESOL standards
(2005).*
[http://www.ode.state.oh.us/GD/Templates/Pages/ODE/ODEDetail.aspx?page=3&TopicRelation\
ID=966&Content=13171](http://www.ode.state.oh.us/GD/Templates/Pages/ODE/ODEDetail.aspx?page=3&TopicRelationID=966&Content=13171)
(accessed December 2007).

Ohio Department of Education. Office of Career-Technical and Adult
Education. n.d. *Adult basic and literacy education: Program rubric for
implementation of standards-based education (SBE).*
<http://www.ode.state.oh.us/GD/DocumentManagement/DocumentDownload.aspx?DocumentID=13206>
(Pages 18--19) (accessed December 2007).

Ohio Department of Education. Office of Career-Technical and Adult
Education. n.d. *Ohio ABLE assessment alignments.*
[http://www.ode.state.oh.us/GD/Templates/Pages/ODE/ODEPrimary.aspx?page=2&Topic\
RelationID=155](http://www.ode.state.oh.us/GD/Templates/Pages/ODE/ODEPrimary.aspx?page=2&TopicRelationID=155)
(accessed December 2007).

Ohio Department of Education. Office of Career-Technical and Adult
Education. n.d. *Standards-based education program self assessment
form*. Columbus, OH: Ohio Department of Education.
[http://www.ode.state.oh.us/GD/Templates/Pages/ODE/ODEDetail.aspx?page=3&TopicRelation\
ID=882&Content=13](http://www.ode.state.oh.us/GD/Templates/Pages/ODE/ODEDetail.aspx?page=3&TopicRelationID=882&Content=13582)582
(accessed December 2007).

Ohio Department of Education. Office of Career-Technical and Adult
Education. n.d. *TABE 9&10 to Ohio Adult Basic and Literacy Education:
Standards and Benchmarks.*
<http://literacy.kent.edu/opas/2004/TABECorrelationtoABLEStandards.pdf>
(accessed December 2007).

Popham, W. J. 2001. Whittling wish-list standards down to reality.
*Education Digest,* 66 (7): 8. <http://pqasb.pqarchiver.com/eddigest>
(accessed December 2007).

Rebell, M. A. 2001. Standards-based reform: Where are the courts going?
In *Education* *reform success stories* (NES 2000 Annual Conference
Report)*,* 65--89. Amherst, MA: National Evaluation Systems.
<http://www.nesinc.com/PDFs/2001_07Rebell.pdf> (accessed December 2007).

Sandrock, P. 1997. *State standards: Connecting a national vision to
local implementation* (ACTFL Professional Issues Report: White Paper).
Alexandria, VA: American Council on the Teaching of Foreign Languages.
<http://www.actfl.org/files/public/winter1997STstan.pdf> (accessed
December 2007).

Sherman, R., J. Voight, J. Tibbets, D. Dobbins, A. Evans, and D.
Weidler. 2000. *Adult educator's guide to designing instructor
mentoring*. Washington, DC: Pelavin Research Institutes.

Smith, C., and J. Hofer. 2003. *The characteristics and concerns of
adult basic education teachers* (NCSALL Report No. 26). Cambridge, MA:
National Center for the Study of Adult Learning and Literacy.
<http://www.ncsall.net/index.php?id=29> (accessed December 2007).

StandardsWork. n.d. *The StandardsWork criteria for judging standards.*
<http://goalline.org/StandardsWork/Criteria.html> (accessed December
2007).

Stites, R. 2004. *An approach to assessment based on cognitive science.*
EFF Research to Practice Note 4. Washington, DC: National Institute for
Literacy.

Stites, R. 2004. *Establishing a common set of definitions.*
Presentation at the Adult Education Content Standards Consortia,
Washington, DC.

Stites, R. 1999. A user's guide to standards-based educational reform:
From theory to practice. *Focus on Basics,* 3 (C). Cambridge, MA:
National Center for the Study of Adult Literacy and Learning (NCSALL).
<http://www.ncsall.net/index.php?id=352> (accessed December 2007).

Teachers of English to Speakers of Other Languages (TESOL). 1997. *ESL
standards for pre-K--12 students*. Alexandria, VA: TESOL, Inc.
<http://www.tesol.org/s_tesol/seccss.asp?CID=95&DID=1565> (accessed
December 2007).

Texas Center for the Advancement of Literacy and Learning (TCALL) at
Texas A&M University. 2004. *Developing standards for Texas' adult
education programs.* College Station, TX: TCALL.

U.S. Department of Labor. Employment and Training Administration. 1992.
*SCANS: Secretary's commission on achieving necessary skills.*
Washington DC: U.S. Department of Labor. <http://wdr.doleta.gov/SCANS>
(accessed December 2007).

U.S. Department of Labor. Secretary's Commission on Achieving Necessary
Skills (SCANS). 1991. *What work requires of schools* (Publication no.
ED 332 054). Washington, DC: U.S. Government Printing Office.

Vaughn S., J. S. Schumm, and J. M. Sinagub. 1996. *Focus group
interviews in education and psychology.* Thousand Oaks, CA: Sage
Publications.

Webb, N. L. 1999. *Alignment of science and mathematics standards and
assessments in four states* (NISE Research Monograph No. 18).
Washington, DC: Council of Chief State School Officers; Madison, WI:
National Institute for Science Education, University of
Wisconsin--Madison.
<http://www.ccsso.org/content/pdfs/AlignmentPaper.pdf> (accessed
December 2007).

Webb, N. L. 1997. *Criteria for alignment of expectations and
assessments in mathematics and science education* (NISE Research
Monograph No. 6). Washington, DC: Council of Chief State School
Officers; Madison, WI: National Institute for Science Education,
University of Wisconsin--Madison.
[http://www.ccsso.org](http://www.ccsso.org/) (accessed December 2007).

West Virginia Department of Education. 2001. *Essential IGO verification
guidelines.* Charleston, WV: West Virginia Department of Education.
<http://wvabe.org/essential_igos.htm> (accessed December 2007).

Wiggins, G., and J. McTighe. 2001. *Understanding by design.* Upper
Saddle River, NJ: Prentice-Hall, Inc.

Workforce Investment Act, 1998. HR 1385, 105th Cong., Pub. L. No.
105-220.

[^1]: Adapted from S. Vaughn, J. S. Schumm, and J. M. Sinagub, *Focus
    Group Interviews in Education and Psychology* (Thousand Oaks, CA:
    Sage Publications, 1996).