U.S. DEPARTMENT OF EDUCATION

SM

Analysis of State K-3 Reading 

Standards and Assessments
Analysis of State K-3 Reading

Standards and Assessments

Final Report

By

E. Allen Schenck

Douglas R. Walker

Carrie R. Nagel

RMC Research Corporation

Arlington, Va.

Loretta C. Webb

The McKenzie Group

Washington, D.C.

Prepared for:

U.S. Department of Education

Office of Planning, Evaluation and Policy Development

Policy and Program Studies Service

2005
This report was prepared for the U.S. Department of Education under Contract No. ED-01-CO-0055/0011,

Task 7. The project monitor was Beth A. Franklin in the Policy and Program Studies Service. The views

expressed herein are those of the contractor. No official endorsement by the U.S. Department of Education

is intended or should be inferred.

U.S. Department of Education

Margaret Spellings

Secretary

Office of Planning, Evaluation and Policy Development

Tom Luce

Assistant Secretary

Policy and Program Studies Service

Alan L. Ginsburg

Director

Program and Analytic Studies Division

David Goodwin

Director

December 2005

This report is in the public domain, except for the photograph on the front cover, which is used with

permission and copyright, 2005, Getty Images. Authorization to produce this report in whole or in part is

granted. Although permission to reprint this publication is not necessary, the citation should be: U.S.

Department of Education, Office of Planning, Evaluation and Policy Development, Policy and Program

Studies Service, Analysis of State K-3 Reading Standards and Assessments, Washington, D.C., 2005.

To order copies of this report, write:

ED Pubs

Education Publications Center

U. S. Department of Education

P. O. Box 1398

Jessup, MD 20794-1398;

via fax, dial (301) 470-1244;

You may also call toll-free: 1-877-433-7827 (1-877-4-ED-PUBS). If 877 service is not yet available in your

area, call 1-800-872-5327 (1-800-USA-LEARN); Those who use a telecommunications device for the deaf

(TDD) or a teletypewriter (TTY), should call 1-800-437-0833.

To order online, point your Internet browser to: www.edpubs.org.

This report is also available on the Department’s Web site at:

http://www.ed.gov/about/offices/list/opepd/ppss/index.html.

On request, this publication is available in alternative formats, such as Braille, large print, audiotape, or

computer diskette. For more information, please contact the Department’s Alternate Format Center at (202)

260-9895 or (202) 205-8113.
iii

Contents

List of Figures............................................................................................................................iv

List of Tables..............................................................................................................................v

Acknowledgments ....................................................................................................................vii

Executive Summary...................................................................................................................ix

Introduction and Background......................................................................................................1

Methodology...............................................................................................................................3

Review of Standards .............................................................................................................3

Review of State Assessments ................................................................................................5

Findings......................................................................................................................................7

Representation of Essential Elements by State Reading Standards.........................................7

Level of Representation, Quality, and Organization of Standards........................................14

State Assessments as Reading First Outcome Measures ...................................................... 17

Similarities in How Well Standards and Assessments Represent the Essential Elements...... 20

Discussion and Conclusions...................................................................................................... 21

Degree and Quality of Standards’ Representation of the Essential Elements........................ 21

Statewide Reading Assessments’ Representation of the Essential Elements......................... 22

Relationship between Assessments’ and Standards’ Representation of the Essential 

Elements........................................................................................................................ 22

References ................................................................................................................................ 23

Appendices............................................................................................................................... 25

Appendix A National Reading Panel Description of the Essential Components of Reading

Instruction ..................................................................................................................... 27

Appendix B Stratified Random Sample of 20 States............................................................29

Appendix C Documentation Used in Review of State K-3 Reading Standards in 2003-04 ... 31

Appendix D Standards Review Instrument........................................................................... 35

Appendix E State Assessment Review Form Sample Page................................................... 51

Appendix F Results of Analyses of Ratings of State K-3 Reading Standards........................ 53
iv

Figures

Figure 1. Mean Number of Clear and Ambiguous Standards Representing Each Essential

Element in 2003-04 ...................................................................................................8

Figure 2. Percentage of States by Appropriateness to Grade Level of K-3 Reading Standards

Representing Each Essential Element in 2003-04.......................................................9

Figure 3. Percentage of States by Adequacy of Coverage of K-3 Reading

Standards Representing Each Essential Element in 2003-04 ....................................10

Figure 4. Percentage of States by Level of Detail of K-3 Reading Standards Representing

Each Essential Element in 2003-04..........................................................................12

Figure 5. Number of States Administering Statewide Reading Assessments in Grades

K-3 in 2003-04 ........................................................................................................17

Figure 6. Number of States Identifying Their Statewide Reading Assessments in Grades

K-3 as Reading First Outcome Measures in 2003-04 ...............................................18

Figure 7. Number of States with Statewide K-3 Reading Assessments Identified as

Reading First Outcome Measures in 2003-04...........................................................19
v

Tables

Table 1. Mean Number of State K-3 Reading Standards Clearly Representing the Essential

Elements by How Adequately the Standards Cover Each Element in 2003-04 .........14

Table 2. Mean Number of State K-3 Reading Standards Clearly Representing the Essential

Elements by Grade Appropriateness in 2003-04.......................................................15

Table 3. Mean Number of State K-3 Reading Standards Clearly Representing the Essential

Elements by the Appropriateness of Detail in 2003-04.............................................16

vii

Acknowledgments

We want to thank Stephanie Al-Otaiba, Michael Coyne, Meaghan Edmonds, Michael McKenna,

and Jean Osborn, the expert reviewers who performed the most challenging task of this study:

judging state reading standards’ representation of the five essential components of reading

instruction identified by the National Reading Panel. We also appreciate the contributions made

to this study by Loretta Webb of The McKenzie Group, particularly in organizing the reviewer

orientation and in analyzing the reviewers’ explanations of their ratings. We thank Judy Carr for

her assistance in developing the standards review instrument and in analyzing the organization of

state reading standards. Finally, we appreciate the guidance and support of our federal project

officer, Beth Franklin, at the U.S. Department of Education.

ix

Executive Summary

Background and Methodology

The National Reading Panel (NRP) issued a report in 2000 that responded to a congressional

mandate to help parents, teachers, and policymakers identify key skills and instructional methods

central to reading achievement. The panel identified five areas that they found to be critical to

effective reading instruction: phonemic awareness, phonics, fluency, vocabulary, and text

comprehension (see Appendix A in the complete final report for a description of these areas).

Using these findings as a foundation, the No Child Left Behind Act (NCLB) of 2001 established

the Reading First program under Title I, Part B, Subpart 1 of the Elementary and Secondary

Education Act (ESEA) to ensure that all children in America are reading at or above grade level

by the end of third grade. This initiative is designed to achieve this goal through the

establishment of high quality reading instruction in kindergarten through grade 3 that includes

the five essential components identified in the research. Congress also recognized that state

academic content standards and assessments play an important role in supporting instruction in

the classroom. Section 1205 of ESEA calls for an evaluation of whether state standards correlate

with and assessments measure these essential components of reading instruction.

This report addresses the relationship between state content standards and assessments and the

essential components of effective reading instruction. The extent to which the essential

components are addressed in the standards and assessments indicates the extent to which states

have integrated the essential components into their reading curriculum. This report describes

reviews of state assessments and standards, the purpose of which was twofold:

• to evaluate the degree to which state reading content standards for K-3 students reflected

expectations for learning in these five essential areas of effective reading instruction; and

• to determine the extent to which state assessments administered in the K-3 grade span

played a role in the measurement of Reading First outcomes in the five areas.

The methods used to address these two purposes differed in both emphasis and approach. This

study conducted an expert review in January 2004 of state reading content standards for grades

K-3 from a random sample of 20 states. Five consultants with expertise in reading instruction,

scientifically based reading research, staff development in reading, and familiarly with state

content standards reviewed the standards from the 20 selected states. Teams of two reviewers

determined how many of each state’s standards represent the five areas of reading instruction and

the degree to which this representation is clear, is appropriate for the intended grade level,

provides complete coverage of each area, and provides an appropriate level of detail to guide

instruction.

The analysis of state assessments was made simpler by existing data. State Reading First

applications included information on which states were using their existing statewide

assessments to measure the five essential components of effective reading instruction. Project

staff conducted a systematic review of approved Reading First applications for all states and the
x

District of Columbia to determine which states’ 2003-04 K-3 statewide assessments were

identified as measures of the five essential areas of reading instruction.

Key Findings

Comprehension and, to a lesser extent, vocabulary are better

represented by sampled states K-3 reading standards than are the

other three essential elements of reading instruction.

• Reading comprehension is the most represented of the essential elements in state K-3

reading content standards with an average of 57 standards per state, followed by

vocabulary (19), phonics (16), fluency (6), and phonemic awareness (6).

• Most standards representing each essential element were judged to be placed at the

appropriate grade by most of the states. A few states were found to have placed

standards representing phonemic awareness and phonics at too high of a grade level.

• Most states have standards that adequately cover comprehension and phonics, while just

over half of the states provide adequate coverage for vocabulary, phonemic awareness,

and fluency. Comprehension standards were judged to cover most or all of the

appropriate content in 90 percent of the states, followed by phonics (80 percent),

vocabulary (60 percent), phonemic awareness (60 percent), and fluency (55 percent).

• Most states (75 percent) provide an appropriate level of detail for comprehension

standards, followed by vocabulary (70 percent), phonics (60 percent), phonemic

awareness (50 percent), and fluency (35 percent). In most cases, when standards were

judged as not having an appropriate level of detail, it was because they were too broad.

• All of the 20 sampled states make comprehension clearly visible in their organization of

reading standards. Almost all (18) make some of the other elements visible. Half make

all five elements visible and they tend to do so at relatively high levels within their

organizational hierarchy.

States with larger numbers of K-3 reading standards organized to make the five

essential elements more visible were judged to represent these elements better.

• For each of the essential elements, states with larger numbers of standards have standards

that provide better coverage, are more likely to be at an appropriate grade level, and are

written more often at an appropriate level of detail.

• Both the number and quality of reading standards—within and across the five

elements—were directly related to the degree of element visibility within the

organization within the reading standards.
xi

With the possible exception of vocabulary and comprehension in

grade 3, statewide reading assessments in 2003-04 do not

significantly address expected student outcomes from reading

instruction in the five essential areas.

• Thirty states administer statewide reading assessments in grade 3, and very few do so at

grades below 3.

• Twenty of these states identify their grade 3 statewide reading assessments as measures

of Reading First outcomes, primarily for just vocabulary and comprehension.

• None of the states identify their statewide reading assessments as outcome measures in

the area of fluency, presumably because it requires individual assessment of children.

There is a slight relationship between how state standards and

assessments represent the five essential elements of reading

instruction.

• States that identified their statewide reading assessments as Reading First outcome

measures tended to have more reading standards that visibly represented the five essential

elements of effective reading instruction.
xii
1

Introduction and Background

Many children struggle when learning to read. This failure can have a tremendous long-term

impact on their self-confidence, motivation to learn, future performance in school, and success in

life. An extensive knowledge base now exists that pinpoints the skills children must learn in

order to read well and the components of reading instruction which are essential to that learning

(Armbruster, Lehr, and Osborn, 2003). This information can serve as the foundation for

educational policy decisions, curriculum development and instructional planning aimed at

helping children learn to read and overcome the problems that come with reading failure.

The National Reading Panel (NRP) issued a report in 2000 that responded to a congressional

mandate to help parents, teachers, and policymakers identify skills and methods central to

reading achievement (National Reading Panel [NRP], 2000). The NRP was charged with

reviewing research in reading instruction and identifying instructional methods that consistently

relate to reading success.

The NRP identified five areas as critical to effective reading instruction: phonemic awareness,

phonics, fluency, vocabulary, and text comprehension. They found that reading is a complex

system of deriving meaning from print. To be effective, instruction in reading must address all of

the five critical areas explicitly and systematically. The panel’s report (NRP, 2000) details the

nature and importance of each of these five essential reading components. Excerpts from this

report describing these components and the evidence of their importance are presented in

Appendix A.

Using these findings as a foundation, the No Child Left Behind Act (NCLB) of 2001 established

the Reading First program under Title I, Part B, Subpart 1 of the Elementary and Secondary

Education Act (ESEA) to ensure that all children in America read at or above grade level by the

end of third grade. This initiative is designed to achieve this goal by establishing high quality

reading instruction in kindergarten through grade 3 that includes the five essential components

identified in the research. Congress recognized the need to study the implementation of this

initiative, including instructional practices and materials, reading assessments used for screening,

diagnosis, and student progress, and professional development. The U.S. Department of

Education (ED) has funded two other evaluations related to Reading First. The Reading First

Implementation study is an evaluation of how the Reading First program is being implemented in

a nationally representative sample of Reading First schools. Results from this evaluation are

expected in 2006 and 2007. The Reading First Impact Study will focus primarily on impact

using a quasi-experimental design (regression discontinuity). The evaluation will collect data

from 250 Reading First and non-Reading First schools. Reports are expected in 2007 and 2008.

Congress also recognized that state academic content standards and assessments play an

important role in supporting instruction in the classroom. The impact that effective reading

instruction has on students’ learning to read will depend on the alignment of state standards and

assessment with that instruction (e.g., Cohen, 1987; Smith and O’Day, 1991; Webb, 1997).

Absent a central focus of state reading standards and related state assessments on student

expectations in these five areas, it is less likely that local curriculum and classroom instruction
2

will address the five essential elements. In Section 1205, the Reading First legislation calls for

an evaluation of whether state standards correlate with and assessments measure these essential

components of reading instruction. Accordingly, ED also funded a review of the relationship

between state standards and assessments in K-3 reading and these components.

This report presents the results of this review of state standards and assessments, the purpose of

which was twofold:

• to evaluate the degree to which state reading content standards for K-3 students reflect

expectations for learning in the five essential areas of effective reading instruction; and

• to determine the extent to which state assessments administered in the K-3 grade span

play a role in the measurement of Reading First outcomes in the five essential areas.

The methods employed to conduct these two reviews are described next. Then the results of the

reviews are presented. The final section contains a discussion of these results and conclusions.
3

Methodology

Review of Standards

A review of state reading standards was conducted to examine the standards’ relationship to the

five essential elements by addressing five questions:

• Which reading standards describe student knowledge and skills in each essential area?

• How clearly and explicitly does each standard represent the area?

• How appropriate are the standards representing each area for the grade(s) to which the

state has assigned them?

• Do the identified standards provide complete coverage of student knowledge and skills in

the area?

• Do the standards representing each area provide an appropriate level of detail with

sufficient specificity to promote alignment among curriculum, instruction, and

assessment and sufficient flexibility to provide curricular guidelines that could be

translated into instructional activities tailored to the needs of different students?

These questions address important aspects of how well each state’s reading standards represent

the five essential elements of reading instruction. They are based on a previous review of state

reading standards for the primary grades, conducted by the Center for the Improvement of Early

Reading Achievement (CIERA) (Wixson and Dutro, 1998). The CIERA study examined how

well state reading standards for grades K-3 represented five areas of important reading content

that are quite similar to the five essential components of reading instruction that are the basis for

this review. The four criteria employed by CIERA—complexity, level of detail, content

coverage, and appropriateness of content—also informed the construction of the above questions

addressed by this review.

A sample of 20 states was randomly selected to be geographically representative of the country

and to include states with relatively large numbers of students. The 50 states and the District of

Columbia were stratified by region, and a probability sample of 20 states was drawn with the

probability of being sampled proportionate to total enrollment.1

 The sampled states accounted

for almost 70 percent of the country’s total enrollment, and four to six states were sampled from

each of four geographical regions. A list of the sampled states with their enrollment and region,

as well as additional sampling details, can be found in Appendix B.

The content standards for reading in grades K-3 were obtained from the 20 sampled states during

January 2004. The documentation collected for the review process included descriptions of the

state K-3 reading standards, per se, and descriptions of benchmarks or grade-level expectations

that provided the most specific available delineation of expected student knowledge and skills in

 1

 Total enrollment was used as a proxy for the size of the Reading First grant. At the time that the sample was

drawn, the Reading First funding levels for the first year were not available for all states. The correlation

between total enrollment and Reading First funding is very high. Based on the 49 states for which both pieces

of data were available, the correlation was 0.91.
4

reading. Although states organize and describe their K-3 reading standards in a variety of ways,

it was possible in most cases to obtain documentation that dealt exclusively with reading

expectations for grades K-3. States vary with regard to the amount of material they include in

their standards documents and the “grain size” of the material presented. What is a standard in

one state might be a grade level expectation in another. Many states use “large” standards that

cut across grade levels, but one state has different standards for each grade level. No state

explicitly uses the five essential components of reading instruction as the organizer for standards

and grade level expectations in grades K-3. While aspects of varying components are present to

varying degrees in the respective sets of standards (as will be discussed further below), they are

somewhat “scattered” throughout the documents. A listing of these documents and the Internet

addresses where these documents were found is presented in Appendix C.

A data collection instrument was designed to obtain information about each state’s reading

standards that answered the five review questions stated above (see Appendix D). The

instrument asks for the identification of state standards that represent each of the five essential

elements of reading instruction described by the National Reading Panel. It also asks how

clearly each standard represents an element, how appropriate the standards are for the grade level

at which they are assigned, how well they cover the elements, and how appropriate their level of

detail is for guiding instruction. Comments are requested on the ratings given and on the

strengths and weaknesses of the entire set of K-3 reading standards.

Five consultants with expertise in reading instruction, scientifically based reading research, and

staff development in reading, and who are familiar with the findings of the National Reading

Panel and state content standards, were recruited to review the standards from the 20 selected

states. The reviewers attended a one-day training session where the data collection instrument

was explained, and expectations for the conduct of the review were presented. They were

introduced to the standards review project and their role was explained. The data collection

instrument was shared and explained, and expectations for the conduct of the review were

presented. Most of the day was spent practicing and discussing issues emerging from the use of

the instrument with documents describing the K-3 reading standards of a state not in the sample

of 20.

Two experts reviewed each state’s standards. Assignments were made so that each expert was

paired with each of the other four experts for two states. Thus, each expert reviewed the

standards of eight states. Assignment was arbitrary; however, reviewers were not assigned the

standards of any states with which they had any professional affiliation or contact. Reviewers

worked independently. However, they were allowed, but not required, to discuss their findings

for any state with the assigned co-reviewer. They were not required to attain consensus on each

state’s ratings.

The pairing of each expert with all others reduced the need for calibrating the ratings of

individual reviewers that might have been unusually high or low in their ratings. Consensus was

not required because differences in points of view were important to capture. In spite of very

little reported collaboration between co-reviewers, discrepancies of more than one rating scale

value occurred less than 10 percent of the time. These differences were judged small enough to

use the average value of the two reviewers for each state in the analysis.
5

Review of State Assessments

The Reading First program requires states to provide evidence that their proposed assessments

for screening, diagnosis, and measuring student progress address phonemic awareness, phonics,

fluency, vocabulary, and text comprehension (U.S. Department of Education, 2002). In addition,

states are required to identify valid and reliable assessments of reading achievement that are used

to evaluate the outcomes of the Reading First program. This report determines the extent to

which states are using their own statewide assessments of reading to measure Reading First

outcomes for the essential components. Using assessments already in place would be more

efficient than requiring schools participating in the Reading First program to administer

additional assessments. More importantly, any state assessment that provides valid and reliable

measurement of outcomes in one or more of the five essential instructional areas in the K-3 grade

span supports the delivery of instruction in these areas throughout the entire state—not just for

schools participating in the Reading First program.

A review of all state Reading First applications approved by ED prior to Oct. 1, 2004, was

conducted to answer the following questions:

• How many states administer reading assessments to virtually all students in one or more

grades in the K-3 span?

• How many states identify their K-3 statewide reading assessments as outcome measures

for Reading First programs?

• For which of the five essential areas of reading instruction are K-3 statewide reading

assessments identified?

This information was summarized on the State Assessment Review Form (see Appendix E)

using the following procedure:

First, the reviewer determined what, if any, reading assessments were administered by the state

to all students in any one of grades K-3. Next, the reviewer determined whether or not the state

assessment was identified in the application as a Reading First outcome measure and, if so,

whether or not the application indicated for which of the five essential components. If the

components were identified, the reviewer indicated whether or not the state assessment was the

only outcome measure for each component.

A statewide assessment was included in this review if it was clear that the assessment was in

place when the Reading First application was approved and that it could be used for measuring

Reading First outcomes. If a state only required school districts to administer assessments

selected from an approved list, or if a state only required school districts to administer a

particular assessment because they were implementing a special initiative, these assessments

were considered to be statewide if virtually all districts (>90 percent) chose the same assessment

or participated in the program or initiative.

7

Findings

This section presents the results of the reviews of state reading content standards and

assessments. These results address the twofold purpose of this study—to evaluate how well state

standards reflect the five essential components of reading instruction, and to determine the role

of state assessments in measuring Reading First outcomes. Results of analyses of the

relationship between how well state standards reflect these components and the use of state

assessments are also provided.

Representation of Essential Elements by State Reading Standards

Reading comprehension is the most represented of the essential elements.

Presumably, the number of standards clearly representing each of the essential elements suggests

the relative emphasis the state reading standards are placing on each element. An average of 104

K-3 reading standards per state were identified as representing the five essential elements (see

Table E-1 in Appendix E).2 Almost 80 percent of these standards were judged to do so in a clear

and explicit manner. There is a very large range in the number of relevant standards identified

across the 20 states—from 19 to 227.

An example of a standard that a reviewer judged to clearly represent phonics is:

Use letters sounds, word patterns and parts of simple compound

words to decode unfamiliar words when reading.

An example of a standard not clearly representing phonics is:

Identify letters, words and sentences.

The average number of standards representing each essential element also varies considerably

across the essential elements (see Figure 1). The highest average number of the standards

identified was in the area of comprehension (57) and this area has the highest percentage of clear

standards (84 percent). The second most frequently represented element is vocabulary; the

average number of standards is 19, and phonics comes in third with an average of 16. The

smallest average number of standards was identified in the areas of phonemic awareness and

fluency (6), and these areas also have the smallest percentages of clear standards (60 percent and

66 percent, respectively).

 2 Reviewers were not asked to identify standards that did not represent the five elements since some states do not

clearly delineate which standards represent reading. Identifying all reading standards that did not represent any

of the five elements would have been beyond the scope of this review.
8

Figure 1

Mean Number of Clear and Ambiguous K-3 Standards

Representing Each Essential Element, 2003-04

0

10

20

30

40

50

60

Phonemic Awareness

PhonicsFluency

Vocabulary

Comprehension

Essential Elements

Mean Number of Standards

Ambiguous

Clear

The reviewers found that half of the states did not make a clear distinction between standards

representing phonemic awareness and phonics. For example, one state presents a standard

entitled, “Phonemic Awareness, Word Recognition and Fluency” and lists more specific

expectations for kindergarten students, including

• Identify and complete rhyming words and patterns.

• Distinguish the number of syllables in words by using rhythmic clapping, snapping or

counting.

• Distinguish and name all upper- and lowercase letters.

• Recognize, say and write the common sounds of letters.

• Hear and say the separate phonemes in words, such as identifying the initial consonant

sound in a word, and blend phonemes to say words.

• Read one-syllable and often-heard words by sight.

According to the reviewers, the first two expectations represent phonological awareness, the next

two represent phonics, the next represents phonemic awareness, and the last fluency.

Most standards were judged to be placed at the appropriate grade.

The grade appropriateness of state standards is relatively high for all five elements (see Table F2 in Appendix F and Figure 2). Virtually all sampled states have at least most of their

comprehension, vocabulary, and fluency standards at an appropriate grade level. The degree of

grade appropriateness for standards representing phonics and phonemic awareness, however, is

not as high; 25 and 33 percent of the states, respectively, were judged to have half or less of their
9

standards assigned to an appropriate grade level. For example, two vocabulary standards from a

state illustrating appropriate placement at second and third grade are:

Grade 2—Use knowledge of base words to interpret meaning of

unfamiliar words

Grade 3—Decode words using knowledge of base words, root

words, and common prefixes and suffixes

Figure 2

Percentage of States by Appropriateness to Grade Level of K-3

Reading Standards Representing Each Essential Element in 2003-04

0

20

40

60

80

100

Phonemic Awareness

Phonics

Fluency

Vocabulary

Comprehension

Essential Elements

Percentage of States

Half or Less

Appropriate

All or Most

Appropriate

The reviewers’ main comment about standards in phonemic awareness and phonics was that the

standards were not sufficiently challenging. This comment was found in the review of 13 states.

For example, one state placed the following standard at the second grade:

The student demonstrates the ability to segment words by

phonemes according to beginning, middle, and ending sounds.

According to a reviewer, this standard should have been placed at kindergarten.
10

A reviewer comment regarding phonics provides another example:

Most standards are one or two grades too late. For example,

Grade 2: PS 1 states “decode unknown words using basic

elements of phonetic analysis (e.g., common letter-sound

relationships).” Students should have mastered this standard in

kindergarten/first grade. Second grade students should be using

more advanced phonic strategies (spelling patterns, letter

combinations, etc.)

Very few comments were made that phonemic awareness and phonics were mentioned in

standards in inappropriately early grades.

Most sampled states have standards that adequately cover comprehension and phonics.

The third review item asked for the reviewer’s judgment of how well all of a state’s standards,

identified as representing an essential element, cover the student knowledge and skills taught in

that area of instruction. Again, standards representing reading comprehension do fairly well.

Almost two-thirds of the sampled states were judged to have standards that adequately cover the

entire area of comprehension, and 90 percent of the states have standards that provide coverage

of all or most of that area (see Table E-3 in Appendix E and Figure 3).

Figure 3

Percentage of States by Adequacy of Coverage of K-3 Reading

Standards Representing Each Essential Element, 2003-04

0

20

40

60

80

100

Phonemic Awareness

PhonicsFluency

Vocabulary

Comprehension

Essential Elements

Percentage of States

Half or Less Covered

All or Most Covered

Phonics appears to be well-covered by most states, too. Eighty percent of the sampled states

were judged to have standards that provide coverage for all or most of the phonics area. Fluency
11

is the most poorly covered element, although the results for this area are only slightly lower than

vocabulary and phonemic awareness. The reading standards for two states do not address

phonemic awareness at all.

The reviewers found a variety of missing content for phonemic awareness (e.g., phoneme

isolation, identity, segmentation, blending, manipulation, etc.) The primary coverage issue for

fluency appears to be not including all of the subcomponents, namely rate, accuracy, and

expression (prosody). Comments about missing content in vocabulary center on failing to

include direct instruction or strategies for determining the meaning of new words. A large

number of comments point out that vocabulary is not covered at all in certain grades, especially

kindergarten and grade 1. Also, there are several references to a tendency for states to place what

the reviewers believe is phonics content under vocabulary.

The number of states providing an appropriate level of detail was greatest for

comprehension standards, followed by vocabulary, phonics, phonemic awareness, and

fluency.

The fourth review item asked how well a state’s standards, which were identified as representing

an essential element, provide a level of detail appropriate for guiding instruction in that area.

The reviewers found 75 percent of the states to have reading comprehension standards that have

an appropriate level of detail, sufficiently specific to promote alignment among curriculum,

instruction, and assessment, and sufficiently flexible to guide instructional activities tailored to

the needs of different students. The other states’ comprehension standards are either somewhat

too broad or somewhat too specific (see Table E-4 in Appendix E and Figure 4).

Most (70 percent) sampled states’ vocabulary standards were also judged to have an appropriate

level of detail. The rest were judged somewhat or much too broad. Reading standards

representing fluency received the lowest judgments of detail level; 65 percent of the states were

rated as having fluency standards that were somewhat or much too broad. Phonemic awareness

and phonics fall between vocabulary and fluency.

Here is an example from one state related to phonemic awareness that was rated as too broad:

Demonstrate phonemic awareness by blending or segmenting

phonemes in a one-syllable word

Here is an example from one state related to phonemic awareness standards rated as an

appropriate level of detail:

[S]egment one-syllable spoken words into individual phonemes,

including three and four phoneme words, clearly producing

beginning, medial, and final sounds

It appears that when state reading standards are not at an appropriate level of detail, with the

exception of the area of comprehension, they err on the side of being too broad.
12

Figure 4

Percentage of States by Level of Detail of K-3 Reading

Standards Representing Each Essential Element in 2003-04

0

20

40

60

80

100

Phonemic Awareness

Phonics

Fluency

Vocabulary

Comprehension

Essential Elements

Percentage of States

Too Broad/Specific

Appropriate Detail

The reviewers noted that the standards representing phonemic awareness and phonics most often

compress important sub-elements together in a single statement or description of a standard, for

example:

The student uses basic elements of phonetic analysis to decode

unknown words (e.g., one-syllable words with 3-4 phonemes).

They also cited the grouping together of standards about phonemic awareness and phonics as

contributing to the lack of specificity.

The reviewers frequently criticized fluency standards for lack of specificity. For example, a

state’s standard for grade 1 makes no attempt to delineate fluency into automaticity and

expression:

Student will read aloud independently with fluency and

comprehension any text that is appropriately designed for

emergent readers.

Also, many comments were made about the absence of specific criteria for the rate of reading,

for example:
13

Student reads primarily in large, meaningful phrases with only a

few slowdowns for problem solving of words or reading to confirm

accuracy or meaning.

All sampled states make comprehension clearly visible in their organization of reading

standards.

In addition to the above analyses of reviewer ratings and comments regarding how well state

standards represent the five essential elements, this study examined the visibility of these

elements in the organization of each state’s reading standards for K-3. There are two ways in

which the organization can make these elements more visible: the use of language in labels and

content which is consistent with these elements, and using this language closer to the top of the

organizational hierarchy. Presumably, higher-level descriptions are more visible to teachers and

curriculum specialists and are more likely to guide instruction.

The results of this examination varied substantially across the essential elements. Text

comprehension is visible in the content standards of all sampled states. For example, in one state

English language arts is divided into Reading, Writing, Conventions, and Listening and

Speaking. Standards are organized under three areas of focus: (a) word analysis, fluency, and

vocabulary; (b) reading comprehension; and (c) literary response and analysis. All essential

elements except comprehension are under the first area of focus. Comprehension standards are

very visible under the second area.

Standards focusing on phonemic awareness, phonics, fluency, and vocabulary are not always

visible. Half of the sample of 20 states use categories of organization or have statements of

student expectations at the most specific levels that clearly represent all five elements (e.g.,

phonemic awareness, word recognition, fluency, vocabulary acquisition, and reading

process—concepts of print and comprehension). Another eight states use language that

represents one or two elements in addition to comprehension (e.g., reading process and

comprehension, analysis of texts, and phonics and word study or analysis). The other two states

make only comprehension clearly visible (e.g., applying a variety of comprehension strategies,

understanding literature, and understanding information texts).

Most of the states with all elements clearly visible do so with organizing language at the top of

the hierarchy; only three do so at an intermediate or most specific level. On the other hand, most

of the states with partial representation of the elements do so at the most specific level of

organization. Thus, there is a relationship between the completeness with which states represent

the essential elements in their organization of reading standards and the level where that

representation occurs.

In summary, comprehension and, to a lesser extent, vocabulary are better represented by

sampled state K-3 reading standards than are the other three essential elements of

reading instruction.

Clearly, the number and quality of standards, as judged by the expert reviewers, favors the area

of comprehension of text. Not only do states have a substantially larger number of standards in

this area, but these standards are, on average, clearer, more likely to be assigned to an
14

appropriate grade level, more comprehensive in coverage, at a more appropriate level of detail,

and more visible. Conversely, the area of phonemic awareness faired the worst. Looking at the

average counts and ratings (see Appendix E), this element is the lowest or second lowest on all

criteria.

Level of Representation, Quality, and Organization of Standards

It is difficult to present information about the number, coverage, detail, and grade

appropriateness of state reading standards without entertaining the idea that there might be some

relationships among these characteristics—perhaps for some of the essential elements, or even

all of them. Analyses were carried out to study these relationships and, more specifically, to

answer the following questions:

• Is the number of state standards clearly representing an essential element related to the

coverage of that element?

• Is the number of state standards clearly representing an essential element related to their

grade appropriateness?

• Is the number of state standards clearly representing an essential element related to their

level of detail?

• Do standards with more appropriate level of detail provide better coverage?

• How does organizational visibility relate to number and coverage, grade appropriateness,

and level of detail?

States with larger numbers of standards have standards that provide more adequate

coverage.

Clearly, states with reading standards that provide adequate coverage have more standards—at

least twice as many—clearly representing each of the five essential elements (see Table 1).

Table 1

Mean Number of State K-3 Reading Standards Clearly Representing the Essential

Elements by How Adequately the Standards Cover Each Element in 2003-04

Standards Provide

Adequate Coverage *

Standards Provide

Inadequate Coverage

Essential Element of

Reading Instruction

Number

of States

Mean Number of

Clear Standards

Number

of States

Mean Number of

Clear Standards

Phonemic Awareness 12 5.1 8 1.7

Phonics 16 13.8 4 2.8

Fluency 11 5.9 9 2.1

Vocabulary 12 18.2 8 9.8

Comprehension 18 50.6 2 20.2

* Adequate coverage means that the standards cover all or most of the element’s content.
15

States with larger numbers of standards have standards that are more likely to be at an

appropriate grade level.

For reading standards representing phonemic awareness and phonics, states that were rated as

assigning most of their standards to an appropriate grade level had many more standards clearly

representing these two areas (see Table 2). Although there may be no reason to expect larger

numbers of standards to be associated with their grade appropriateness, this finding—combined

with the other associations in this section—suggests that states with better standards in general

have created relatively larger numbers of standards as well. Because all but one state had their

standards in the other three areas rated as mostly appropriate for the assigned grade level,

comparisons with states whose standards were not rated mostly appropriate is impossible or

difficult.

Table 2

Mean Number of State K-3 Reading Standards Clearly Representing the Essential

Elements by Grade Appropriateness in 2003-04

Standards Are Mostly

Appropriate

Standards Are Not Mostly

Appropriate

Essential Element of

Reading Instruction

Number

of States

Mean Number of

Clear Standards

Number

of States

Mean Number of

Clear Standards

Phonemic Awareness 12 5.3 6 1.8

Phonics 15 14.2 5 3.6

Fluency 19 4.4 1 1.0

Vocabulary 19 15.5 1 1.0

Comprehension 20 47.5 0 --

States with larger numbers of standards have standards that are written more often at an

appropriate level of detail.

The average number of standards representing each essential element for states with standards

having appropriate detail and for states with standards that are either too broad or too specific is

presented in Table 3.
16

Table 3

Mean Number of State K-3 Reading Standards Clearly Representing the Essential

Elements by the Appropriateness of Detail in 2003-04

Standards Have

Appropriate Detail

Standards Are Too Broad or

Too Specific *

Essential Element of

Reading Instruction

Number

of States

Mean Number of

Clear Standards

Number

of States

Mean Number of

Clear Standards

Phonemic Awareness 9 5.7 9 2.7

Phonics 13 13.0 7 9.0

Fluency 7 6.1 13 3.2

Vocabulary 14 17.2 6 9.2

Comprehension 18 49.8 2 27.5

* Standards were always too broad except for the comprehension element.

Again, for all five of the essential elements, states with standards written at an appropriate level

of detail have more standards that clearly represent each element than states with standards

judged to be either too broad or too specific.

There is a modest relationship between level of detail and adequacy of coverage.

The relationship between the standards’ level of detail and adequacy of coverage is of interest

because some might argue that it is easier to provide greater coverage with standards that are

fairly broad in their level of detail. Yet, when states were grouped on these two characteristics

above, states with broad standards and states with inadequate coverage had fewer standards,

suggesting that the same states might tend to have broad standards and standards providing

inadequate coverage. Correlation coefficients calculated between coverage and level of detail

ranged between 0.24 (for phonemic awareness) and 0.45 (for fluency), indicating a small, but

positive, relationship between these two characteristics of state reading standards. States with

standards that have an appropriate level of detail tend to be the same states whose standards

provide adequate coverage of the essential elements.

There is a positive relationship between the visibility of the essential elements within the

organization of reading standards and how well the standards represent these elements.

Both the number and quality of reading standards—within and across the five elements—were

found to be directly related to the degree of element visibility. For example, the average number

of standards clearly representing all five elements for the 10 states with all elements visible is

98.6. For the eight states with some elements visible, the average number of standards is 72.0,

and for the two states with only comprehension visible, the average is 24.5. This is not simply

due to the “better visibility” states having more standards in certain areas such as phonemic

awareness, phonics, and fluency. This pattern in the average number of standards is the same for

each essential element. For example, the average number of standards clearly representing

comprehension of text for states with all, some, or only comprehension elements visible is 55.1,

43.4, and 20.2, respectively.
17

The percentage of states providing coverage for most or all of each element is similarly related to

the degree of element visibility. That is, states that organize their standards so that the elements

are visible were more likely to be judged to have standards that provide good coverage of the

elements. Coverage is also directly related to the level at which the elements are visible.

Coverage is best for states that make the elements visible in the top of their hierarchies and worst

for states with elements visible at the bottom, most specific level.

State Assessments as Reading First Outcome Measures

Thirty of the 51 states (59 percent) administer statewide reading assessments in at least

one grade in the K-3 grade span.

These 30 states all administer statewide reading assessments in the third grade, whereas only 13

(26 percent) do so in second grade, seven (14 percent) in first grade, and six (12 percent) do so in

kindergarten (see Figure 5).

Figure 5

Number of States Administering Statewide

Reading Assessments in Grades K-3, 2003-04

6 7

13

30

0

10

20

30

40

50

K123

Grade

Number of States

The few states with statewide reading assessments in K-2 include two states (Texas and Virginia)

with reading assessments that are not, strictly speaking, required of all students, but, in fact, are

administered to virtually all students in these grades.

Twenty-one states (41 percent) identify statewide reading assessments as measures of

Reading First outcomes in at least one of the K-3 grades.

Twenty (39 percent) identify their third-grade assessments as Reading First outcome measures.

Very few states have reading assessments in K-2, and fewer still also identify these assessments
18

for Reading First—six (12 percent) in second grade, three (6 percent) in first grade, and four (8

percent) in kindergarten (see Figure 6).

Figure 6

Number of States Identifying Their Statewide

Reading Assessments in Grades K-3 as Reading

First Outcome Measures, 2003-04

4 3

6

20

0

10

20

30

40

50

K123

Grade

Number of States

Looking only at the 30 states with statewide reading assessments in at least one of the K-3

grades, most (21 or 70 percent) identify their state assessments as Reading First outcome

measures in at least one grade. Of the 30 states with statewide reading assessments in third

grade, 20 (67 percent) identify them for Reading First. Six of 13 (46 percent) do so in second

grade, three of seven (43 percent) in first grade, and four of six (67 percent) in kindergarten (see

Figure 7).
19

Figure 7

Number of States with Statewide K-3 Reading

Assessments Identified as Reading First

Outcome Measures, 2003-04

4 3

6

20

2 4

7

10

0

10

20

30

K123

Grade

Number of StateNot RF

Outcome

Measures

RF Outcome

Measures

Most states identifying their statewide assessments as Reading First outcome measures

specify the targeted reading components, and they tend to be comprehension and

vocabulary.

Of the 21 states that use their statewide reading assessments as Reading First outcome measures,

most (15 or 71 percent) specify the components of reading instruction their assessments are

expected to measure. At specific grades, 14 of 20 (70 percent) specify the components for third

grade, three of six (50 percent) do so for second grade, one of three (33 percent) for first grade,

and two of four (50 percent) for kindergarten. Of the six states that do not specify the measured

reading components, two indicate that their statewide reading assessments are being used in

conjunction with other assessments (e.g., the DIBELS or the Woodcock-Johnson) required by

the state for all Reading First districts.

For kindergarten, the two states identifying reading components indicate their statewide

assessments are intended to measure all components except fluency. In one of these states, the

statewide assessment is the only Reading First outcome measure identified for these elements; in

the other state, additional assessments are required as Reading First outcome measures. For

grade 1, the one state identifying reading components indicates the statewide assessment

measures all components except fluency. That state also requires districts to employ other

assessments as Reading First outcome measures for these components.

For grade 2, phonics, vocabulary, and comprehension are the components measured by the

statewide reading assessments of the three states. One state identifies its statewide assessment as

the only Reading First outcome measure of vocabulary and comprehension, while there are

additional outcome measures for phonics. In the other two states, additional outcome measures

are required for Reading First districts.
20

For grade 3, all 14 states identify their statewide reading assessment as measuring

comprehension; 11 also identify vocabulary. Phonics is identified by only three states. Nine of

these 14 states identify the statewide reading assessment as the only Reading First outcome

measure for a component. Five states require other assessments for Reading First districts. Two

states use both strategies for different components.

In summary, 30 states administer statewide reading assessments in grade 3, and few do so at

grades below 3. Twenty of these states identify their grade 3 statewide reading assessments as

measures of Reading First outcomes, primarily for just vocabulary and comprehension. None of

the states identify their statewide reading assessments as outcome measures in the area of

fluency, presumably because it requires individual assessment of children.

Similarities in How Well Standards and Assessments Represent the

Essential Elements

This study looked for similarities in how well state K-3 reading standards represent the five

essential elements and whether the state identified its reading assessments as measuring Reading

First outcomes. Few relationships were found in the 20 sampled states. There is a moderate

difference in the number of reading standards clearly representing the essential elements. States

identifying their reading assessments as Reading First outcome measures average 92.6 standards,

while states not identifying their assessments and states with no K-3 assessments average 72.1

and 76.1 standards, respectively. Also, the degree of visibility of the five essential elements in

the organization of state reading standards was found to relate somewhat to the identification of

state reading assessments as Reading First outcome measures. Five of the eight states (62.5

percent) that identify their statewide reading assessments as Reading First outcome measures

make all elements visible, compared to 50 percent of the states that do not identify their

assessments for Reading First or have no K-3 reading assessments.
21

Discussion and Conclusions

Degree and Quality of Standards’ Representation of the Essential

Elements

State reading standards for K-3 represent reading comprehension better than any of the other

essential elements of reading instruction in the 20 sampled states. Comprehension of text is

represented by the largest number of standards and the highest percentage of clear standards,

followed by vocabulary, phonics, fluency, and phonemic awareness. Some standards

representing phonemic awareness or phonics were judged unclear due to the confusion and

mixing together of standards representing these two elements.

The appropriateness of the grade level assignment of standards was fairly high overall. The

lowest percentage of states judged to have assigned most or all standards to an appropriate grade

level was 67 percent for phonemic awareness. The highest percentage was for comprehension,

followed by vocabulary, fluency, phonics, and phonemic awareness. A typical reviewer

comment was that the grade placement of phonemic awareness and phonics standards was too

high, suggesting that expectations in these two areas could be increased, especially for students

in kindergarten and grade 1.

The coverage of content in each essential element followed a similar pattern. Comprehension

standards were judged to cover most or all of the appropriate content in 90 percent of the states.

Comprehension was followed by phonics, vocabulary, phonemic awareness, and fluency.

Fluency standards appeared to include some but not all of three subcomponents: rate, accuracy,

and expression (prosody). The phonemic awareness standards were also faulted for failing to

include all subcomponents but not any in particular. Some states left out phoneme identity, some

left out phoneme segmentation, etc. The content missing for vocabulary was most frequently

direct instruction and strategies for figuring out the meaning of new words.

State reading standards were not judged as high on appropriate level of detail, erring primarily on

the side of being too broad. The highest percentage of states with an appropriate level of detail

was 80 percent for comprehension of text, followed by vocabulary, phonics, phonemic

awareness, and fluency. In most cases, when standards were judged as not having an appropriate

level of detail, it was because they were too broad. Fluency standards appeared to be judged too

broad because, as in the case of content coverage, they did not include specifics about all three

subcomponents: rate, accuracy, and expression.

Overall, standards representing comprehension were most numerous, clear, at an appropriate

grade level, comprehensive in coverage, and written at an appropriate level of detail.

Vocabulary usually came next. This outcome is consistent with states’ having a great deal of

experience developing standards in comprehension and vocabulary, and less experience with the

other three elements only recently emphasized by the National Reading Panel reports.

There were several interesting relationships observed among the reviewers’ judgments about the

number and quality of reading standards. States with larger numbers of standards have standards
22

that provide better coverage, are more likely to be at an appropriate grade level, and are written

more often at an appropriate level of detail. A relationship between coverage and level of detail

was also found. Apparently, reviewers required a reasonably small “grain size” to be able to see

whether there was adequate coverage.

Finally, certain aspects of the organization of reading standards were related to how well the

standards represent the essential elements of effective reading instruction. Both the number and

quality of reading standards were directly related to the number of essential elements that were

clearly visible in the organizational hierarchy. The more elements that were clearly visible, the

more standards there were representing each of the elements and the better the coverage that was

provided for each element. Coverage was also found to relate to the organization level at which

the elements were made visible. Visibility at the highest level related to better coverage of the

elements.

Statewide Reading Assessments’ Representation of the Essential

Elements

The review of the 50 state and the District of Columbia Reading First applications suggests that

statewide reading assessments in place during the 2003-04 school year do not play a significant

role in promoting reading instruction in the five essential areas, with the possible exception of

vocabulary development and text comprehension in grade 3. Only 30 states administer statewide

reading assessments in grade 3, and very few do so at grades below 3. Even in grade 3, only

two-thirds of the states with statewide reading assessments (20 out of 30) identify their

assessments as measures of Reading First outcomes, and primarily for only vocabulary and

comprehension. As states develop statewide reading assessments for grade 3 students to comply

with NCLB, they may develop them to be compatible with the Reading First program

requirements.

Most states identifying their statewide assessments as Reading First outcome measures also

identify the essential elements that are measured. None of the states identifies its statewide

reading assessments as Reading First outcome measures in the area of fluency. They target

phonemic awareness and phonics as well as early vocabulary development and comprehension

skills. The measurement of (oral) fluency presents, and will probably continue to present,

challenges to large-scale assessment programs, because it requires individual assessment of

children which is costly and time-consuming.

Relationships between Standards’ and Assessments’ Representation

of Essential Elements

Modest differences in the number and visibility of reading standards were found between states

that identified their statewide reading assessments as Reading First outcome measures and those

that did not. It is possible that states identifying their assessments for Reading First outcome

measures have paid more attention to whether their assessments and standards are aligned with

the essential elements of effective reading instruction.
23

References

Armbruster, B. B., Lehr, F., and Osborn, J. (2003). Put reading first: The research building

blocks for teaching children to read. (2nd ed.) Washington, D.C.: Partnership for Reading.

Cohen, S. A. (1987). Instructional alignment: Searching for a magic bullet. Educational

Researcher, 16(8), 16-20.

Council of Chief State School Officers. (1996). Criteria for standards. Washington, D.C.:

Author.

National Center for Education Statistics. Common core of data, 2001-02: Build a table. U.S.

Department of Education. Available: http://nces.ed.gov/ccd/bat/.

National Center for Education Statistics. (2003). Digest of Education Statistics, 2002.

Washington, D.C.: U.S. Department of Education.

National Reading Panel (2000). Teaching children to read: An evidence-based assessment of the

scientific research literature on reading and its implications for reading instruction.

Washington, D.C.: National Institute of Child Health and Human Development.

Smith, M. S., and O’Day, J. (1991). Systemic school reform. In S. H. Fuhrman and B. Malen

(eds.), The politics of curriculum and testing. Bristol, Pa.: Falmer.

U.S. Department of Education. (2002). Guidance for the Reading First program. Washington,

D.C.: Office of Elementary and Secondary Education, U.S. Department of Education.

Webb, N. L. (1997). Determining alignment of expectations and assessments in mathematics and

science education. Madison, Wis.: National Center for Improving Science Education,

University of Wisconsin-Madison.

Wixson, K. K. and Dutro, E. (1998). Standards for primary-grade reading: An analysis of state

frameworks. Ann Arbor, Mich.: Center for the Improvement of Early Reading Achievement,

University of Michigan.

25

Appendices

Appendix A. National Reading Panel Description of the Essential Components of

Reading Instruction

Appendix B. Stratified Random Sample of 20 States

Appendix C. Documentation Used in Review of State K-3 Reading Standards

Appendix D. Standards Review Instrument

Appendix E. State Assessment Review Form

Appendix F. Results of Analyses of Ratings of State K-3 Reading Standards

27

Appendix A. National Reading Panel Description of the Essential

Components of Reading Instruction

The following excerpts are taken from the summary report of the National Reading Panel (2000).

They provide a brief description of each of the five essential components of reading instruction

and the evidence for their impact on learning to read.

Phonemic Awareness

Phonemes are the smallest units composing spoken language. For example, the

words “go” and “she” each consist of two sounds or phonemes. Phonemes are

different from letters that represent phonemes in the spellings of words.

Instruction in phonemic awareness (PA) involves teaching children to focus on

and manipulate phonemes in spoken syllables and words. (p. 7)

Overall, the findings showed that teaching children to manipulate phonemes in

words was highly effective under a variety of teaching conditions with a variety

of learners across a range of grade and age levels and that teaching phonemic

awareness to children significantly improves their reading more than instruction

that lacks any attention to PA. (p. 7)

Phonics

The primary focus of phonics instruction is to help beginning readers understand

how letters are linked to sounds (phonemes) to form letter-sound correspondences

and spelling patterns and to help them learn how to apply this knowledge in their

reading. (p. 8)

The meta-analysis revealed that systematic phonics instruction produces

significant benefits for students in kindergarten through sixth grade and for

children having difficulty learning to read. The ability to read and spell words was

enhanced in kindergartners who received systematic beginning phonics

instruction. First-graders who were taught phonics systematically were better able

to decode and spell, and they showed significant improvement in their ability to

comprehend text. Older children receiving phonics instruction were better able to

decode and spell words and to read text orally, but their comprehension of text

was not significantly improved. (p. 9)

Fluency

Fluent readers are able to read orally with speed, accuracy, and proper expression.

Fluency is one of several critical factors necessary for reading comprehension.

(p. 11)

... [T]he Panel concluded that guided repeated oral reading procedures that

included guidance from teachers, peers, or parents had a significant and positive

impact on word recognition, fluency, and comprehension across a range of grade

levels. (p. 12)
28

Vocabulary

There are two types of vocabulary—oral and print. A reader who encounters a

strange word in print can decode the word to speech. If it is in the reader’s oral

vocabulary, the reader will be able to understand it. If the word is not in the

reader’s oral vocabulary, the reader will have to determine the meaning by other

means, if possible. Consequently, the larger the reader’s vocabulary (either oral or

print), the easier it is to make sense of the text. (p. 13)

The findings on vocabulary yielded several specific implications for teaching

reading. First, vocabulary should be taught both directly and indirectly. Repetition

and multiple exposures to vocabulary items are important. Learning in rich

contexts, incidental learning, and use of computer technology all enhance the

acquisition of vocabulary. (p. 14)

Text Comprehension

... [C]omprehension is an active process that requires an intentional and

thoughtful interaction between the reader and the text.... Thus, readers derive

meaning from text when they engage in intentional, problem solving thinking

processes. (p. 14)

... [T]he evidence suggests that teaching a combination of reading comprehension

techniques is the most effective. When students use them appropriately, they

assist in recall, question answering, question generation, and summarization of

texts. When used in combination, these techniques can improve results in

standardized comprehension tests. (p. 15)
29

Appendix B. Stratified Random Sample of 20 States

State Enrollment Region

Pennsylvania 1,821,627 Northeast

New York 2,872,132 Northeast

New Jersey 1,341,656 Northeast

Massachusetts 973,140 Northeast

Virginia 1,163,091 Southeast

South Carolina 691,078 Southeast

North Carolina 1,315,363 Southeast

Louisiana 731,328 Southeast

Florida 2,500,478 Southeast

Ohio 1,830,985 Central

Missouri 909,792 Central

Minnesota 851,384 Central

Michigan 1,730,668 Central

Illinois 2,071,391 Central

Wyoming 88,128 West

Washington 1,009,200 West

Texas 4,163,447 West

Nevada 356,814 West

New Mexico 320,260 West

California 6,248,610 West

Total 32,990,572

Percent of U.S. Enrollment 69.2%

U.S. Enrollment 47,687,871

Sampling Method: Each of the 50 states and the District of Columbia was assigned to one of four

geographic regions (using the first definition in the Digest of Education Statistics, 2002). The total student

enrollment for each state was taken from the Build A Table resource of the Common Core of Data, 2001-

02. The states were stratified by region and a probability sample of 20 states was drawn with the

probability of being sampled proportionate to total enrollment. This procedure facilitated drawing states

that were geographically representative of the country and serving relatively large numbers of students.

Appendix C. Documentation Used in Review of State K-3 Reading

Standards in 2003-04

The documents used to review the K-3 reading content standards for each of the 20 sampled

states are listed below. The Internet address where these standards documents were found is also

provided. (These addresses were last accessed successfully on or about Jan. 31, 2004.)

California

English-Language Arts Content Standards for California Public Schools: Kindergarten

Through Grade 12

http://www.cde.ca.gov/cdepress/standards-pdfs/english-language-arts.pdf

Florida

Language Arts Standards: Pre-K-2 and 3-5

Grade Level Expectations for the Sunshine State Standards: K-2

Grade Level Expectations for the Sunshine State Standards: 3-5

Grade Level Expectations for the Sunshine State Standards: K, 1, 2, and 3

http://www.firn.edu/doe/curric/prek12/frame2.htm

(Provides access to all of the above.)

Illinois

English Language Arts: State Goals: 1-5

http://www.isbe.state.il.us/ils/english/english.html

English Language Arts Performance Descriptors

http://www.isbe.net/ils/pdfs/English_PDs_1-5.pdf

Louisiana

Content Standards

http://www.doe.state.la.us/lde/uploads/2909.pdf

English Language Arts (ELA) Grade Level Expectations

http://www.doe.state.la.us/lde/uploads/3906.pdf

Massachusetts

Massachusetts English Language Arts Curriculum Framework, June 2001

http://www.doe.mass.edu/frameworks/ela/0601.pdf
32

Michigan

Michigan Curriculum Framework (English Language Arts Section)

http://michigan.gov/documents/MichiganCurriculumFramework_8172_7.pdf

Draft Grade Level Content Expectations—Reading K-5

(Not posted on Internet. Obtained from state.)

Minnesota

Minnesota Academic Standards: Language Arts K-12, May 19, 2003

http://education.state.mn.us/content/009200.pdf

Missouri

The Show-Me Knowledge Standards: Communication Arts

http://dese.mo.gov/standards/comarts.html

Framework for Curriculum Development in Communication Arts K-12

http://dese.mo.gov/divimprove/curriculum/webframeworks/01CA.PDF

Communication Arts Grade-Level Expectations—Aug. 15, 2003

http://dese.mo.gov/divimprove/curriculum/GLE/Comm_Arts_GradeLevel_Expectations_8.15.03.pdf

Nevada

Nevada English Language Arts Content Standards for Kindergarten and Grades 1, 2, 3, 4, 5, 6,

7, 8, and 12

http://www.nde.state.nv.us/sca/standards/standardsfiles/ela/elacont.pdf

New Jersey

New Jersey Core Curriculum Content Standards for Language Arts Literacy

http://www.nj.gov/njded/cccs/02/lal.pdf

New Mexico

New Mexico Curriculum Framework—Language Arts

http://164.64.166.11/cilt/downloads/standards/stand_la.pdf

New York

Learning Standards for English Language Arts. March, 1996

http://www.emsc.nysed.gov/ciai/ela/pub/elalearn.pdf

English Language Arts Resource Guide: Core Curriculum

http://www.emsc.nysed.gov/ciai/ela/pub/ccela.pdf

Early Literacy Guidance: Prekindergarten—Grade 3

http://www.emsc.nysed.gov/ciai/ela/early.pdf
33

North Carolina

English Language Arts Curriculum (Selected Sections/Links)

http://www.ncpublicschools.org/curriculum/languagearts/

(click on “Curriculum Approved: 1999” link)

Ohio

Academic Content Standards: K-12 English Language Arts

http://www.ode.state.oh.us/academic_content_standards/pdf/ENGLISH.pdf

Pennsylvania

Academic Standards for Reading, Writing, Speaking and Listening

http://www.pde.state.pa.us/k12/lib/k12/Reading.pdf

Standards and Eligible Content

http://www.pde.state.pa.us/a_and_t/cwp/view.asp?A=108&Q=98808#blueprint

South Carolina

South Carolina English Language Arts Curriculum Standards 2002

http://www.sde.state.sc.us/offices/cso/standards/ela/documents/standards.pdf

Texas

Texas Administrative Code (TAC), Title 19, Part II

Chapter 110. Texas Essential Knowledge and Skills for English Language Arts and Reading

http://www.tea.state.tx.us/rules/tac/chapter110/

Virginia

English Standards of Learning for Virginia Public Schools, Nov. 20, 2002

http://www.pen.k12.va.us/VDOE/Superintendent/Sols/EnglishSOLFinal02.pdf

Washington

Essential Academic Learning Requirements—Reading

http://www.k12.wa.us/curriculumInstruct/reading/ealrs.aspx

Grade Level Expectations Reports

http://www.k12.wa.us/curriculuminstruct/reading/pubdocs/ReadingEALR-GLE.pdf

Wyoming

Wyoming Language Arts Content and Performance Standards—July 7, 2003

http://www.k12.wy.us/eqa/nca/pubs/standards/lang.pdf
34

.
34

Appendix D. Standards Review Instrument

How Well Do the Standards Represent Phonemic Awareness?

hat describe student knowledge and skills in the area of Phonemic Awareness. Write, copy, or reference each standard in one of

describes how clearly and explicitly the standard represents this area of reading instruction. If referencing, please use a code, such

n the state’s documentation, that accurately and uniquely identifies the standard in the state’s documentation. Also, indicate the

y placing a K, 1, 2, or 3 in parentheses next to it.

citly represent

Standards that require some interpretation or“reading between the lines” to make theconnection to Phonemic Awareness

Standards that are only vaguely or remotelyrelated to Phonemic Awareness

r features of the state’s organization of its standards that might be inconsistent with instruction in Phonemic

ading research reported by the National Reading Panel. Give or reference examples.
35

presenting Phonemic Awareness describe student knowledge and skills appropriate for the grades to which the standards have

Indicate which column is the best response by checking, circling, bolding, underlining, etc.)

Most are appropriate About half are appropriate Few are appropriate None are appropriate

m 3. If not, please explain here how these standards are not grade-appropriate, e.g., inappropriate sequencing,

ify which standards and which areas of Phonemic Awareness are affected.

rovide complete coverage of the student knowledge and skills in the area of Phonemic Awareness? (Indicate which column is

Most are covered About half are covered Few are covered None are covered

covered, go to item 4. If not, please explain here what student knowledge and skills are missing. If coverage

cribe how.
36

presenting Phonemic Awareness provide an appropriate level of detail? An appropriate level of detail would provide sufficient

ment among curriculum, instruction, and assessment and sufficient flexibility to provide curricular guidelines that could be

ctivities tailored to the needs of different students? (Indicate which column is the best response.)

mewhat too broad/general Appropriate level of detail Somewhat too specific Much too specific

ropriate level of detail, go to item 5. If not, please describe and give examples of how these standards provide

xplain how instruction in the area of Phonemic Awareness would be affected.

s you would like to make that would help us understand how well this state’s standards represent the reading instructional area of
37

How Well Do the Standards Represent Phonics?

hat describe student knowledge and skills in the area of Phonics. Write, copy, or reference each standard in one of the columns

w clearly and explicitly the standard represents this area of reading instruction. If referencing, please use a code, such as an outline

documentation, that accurately and uniquely identifies the standard in the state’s documentation. Also, indicate the grade level of

1, 2, or 3 in parentheses next to it.

citly represent

Standards that require some interpretation or“reading between the lines” to make theconnection to Phonics

Standards that are only vaguely or remotelyrelated to Phonics

r features of the state’s organization of its standards that might be inconsistent with instruction in Phonics that is

ted by the National Reading Panel. Give or reference examples.
38

presenting Phonics describe student knowledge and skills appropriate for the grades to which the standards have been assigned

h column is the best response by checking, circling, bolding, underlining, etc.)

Most are appropriate About half are appropriate Few are appropriate None are appropriate

m 3. If not, please explain here how these standards are not grade-appropriate, e.g., inappropriate sequencing,

ify which standards and which areas of Phonics are affected.

rovide complete coverage of the student knowledge and skills in the area of Phonics? (Indicate which column is the best

Most are covered About half are covered Few are covered None are covered

covered, go to item 4. If not, please explain here what student knowledge and skills are missing. If coverage

cribe how.
39

presenting Phonics provide an appropriate level of detail? An appropriate level of detail would provide sufficient specificity to

urriculum, instruction, and assessment and sufficient flexibility to provide curricular guidelines that could be translated into

d to the needs of different students? (Indicate which column is the best response.)

mewhat too broad/general Appropriate level of detail Somewhat too specific Much too specific

ropriate level of detail, go to item 5. If not, please describe and give examples of how these standards provide

xplain how instruction in the area of Phonics would be affected.

s you would like to make that would help us understand how well this state’s standards represent the reading instructional area of
40

How Well Do the Standards Represent Fluency?

hat describe student knowledge and skills in the area of Fluency. Write, copy, or reference each standard in one of the columns

w clearly and explicitly the standard represents this area of reading instruction. If referencing, please use a code, such as an outline

documentation, that accurately and uniquely identifies the standard in the state’s documentation. Also, indicate the grade level of

1, 2, or 3 in parentheses next to it.

citly represent

Standards that require some interpretation or“reading between the lines” to make theconnection to Fluency

Standards that are only vaguely or remotelyrelated to Fluency

r features of the state’s organization of its standards that might be inconsistent with instruction in Fluency that is

ted by the National Reading Panel. Give or reference examples.
41

presenting Fluency describe student knowledge and skills appropriate for the grades to which the standards have been assigned

h column is the best response by checking, circling, bolding, underlining, etc.)

Most are appropriate About half are appropriate Few are appropriate None are appropriate

m 3. If not, please explain here how these standards are not grade-appropriate, e.g., inappropriate sequencing,

ify which standards and which areas of Fluency are affected.

rovide complete coverage of the student knowledge and skills in the area of Fluency? (Indicate which column is the best

Most are covered About half are covered Few are covered None are covered

covered, go to item 4. If not, please explain here what student knowledge and skills are missing. If coverage

cribe how.
42

presenting Fluency provide an appropriate level of detail? An appropriate level of detail would provide sufficient specificity to

urriculum, instruction, and assessment and sufficient flexibility to provide curricular guidelines that could be translated into

d to the needs of different students? (Indicate which column is the best response.)

mewhat too broad/general Appropriate level of detail Somewhat too specific Much too specific

ropriate level of detail, go to item 5. If not, please describe and give examples of how these standards provide

xplain how instruction in the area of Fluency would be affected.

s you would like to make that would help us understand how well this state’s standards represent the reading instructional area of
43

How Well Do the Standards Represent Vocabulary?

hat describe student knowledge and skills in the area of Vocabulary. Write, copy, or reference each standard in one of the

ribes how clearly and explicitly the standard represents this area of reading instruction. If referencing, please use a code, such as

he state’s documentation, that accurately and uniquely identifies the standard in the state’s documentation. Also, indicate the

y placing a K, 1, 2, or 3 in parentheses next to it.

citly represent

Standards that require some interpretation or“reading between the lines” to make theconnection to Vocabulary

Standards that are only vaguely or remotelyrelated to Vocabulary

r features of the state’s organization of its standards that might be inconsistent with instruction in Vocabulary

h reported by the National Reading Panel. Give or reference examples.
44

presenting Vocabulary describe student knowledge and skills appropriate for the grades to which the standards have been

ate which column is the best response by checking, circling, bolding, underlining, etc.)

Most are appropriate About half are appropriate Few are appropriate None are appropriate

m 3. If not, please explain here how these standards are not grade-appropriate, e.g., inappropriate sequencing,

ify which standards and which areas of Vocabulary are affected.

rovide complete coverage of the student knowledge and skills in the area of Vocabulary? (Indicate which column is the best

Most are covered About half are covered Few are covered None are covered

covered, go to item 4. If not, please explain here what student knowledge and skills are missing. If coverage

cribe how.
45

presenting Vocabulary provide an appropriate level of detail? An appropriate level of detail would provide sufficient specificity

 curriculum, instruction, and assessment and sufficient flexibility to provide curricular guidelines that could be translated into

d to the needs of different students? (Indicate which column is the best response.)

mewhat too broad/general Appropriate level of detail Somewhat too specific Much too specific

ropriate level of detail, go to item 5. If not, please describe and give examples of how these standards provide

xplain how instruction in the area of Vocabulary would be affected.

s you would like to make that would help us understand how well this state’s standards represent the reading instructional area of
46

How Well Do the Standards Represent Comprehension?

hat describe student knowledge and skills in the area of Comprehension. Write, copy, or reference each standard in one of the

ribes how clearly and explicitly the standard represents this area of reading instruction. If referencing, please use a code, such as

he state’s documentation, that accurately and uniquely identifies the standard in the state’s documentation. Also, indicate the

y placing a K, 1, 2, or 3 in parentheses next to it.

citly represent

Standards that require some interpretation or“reading between the lines” to make theconnection to Comprehension

Standards that are only vaguely or remotelyrelated to Comprehension

r features of the state’s organization of its standards that might be inconsistent with instruction in

on reading research reported by the National Reading Panel. Give or reference examples.
47

presenting Comprehension describe student knowledge and skills appropriate for the grades to which the standards have been

ate which column is the best response by checking, circling, bolding, underlining, etc.)

Most are appropriate About half are appropriate Few are appropriate None are appropriate

m 3. If not, please explain here how these standards are not grade-appropriate, e.g., inappropriate sequencing,

ify which standards and which areas of Comprehension are affected.

rovide complete coverage of the student knowledge and skills in the area of Comprehension? (Indicate which column is the best

Most are covered About half are covered Few are covered None are covered

covered, go to item 4. If not, please explain here what student knowledge and skills are missing. If coverage

cribe how.
48

presenting Comprehension provide an appropriate level of detail? An appropriate level of detail would provide sufficient

ment among curriculum, instruction, and assessment and sufficient flexibility to provide curricular guidelines that could be

ctivities tailored to the needs of different students? (Indicate which column is the best response.)

mewhat too broad/general Appropriate level of detail Somewhat too specific Much too specific

ropriate level of detail, go to item 5. If not, please describe and give examples of how these standards provide

xplain how instruction in the area of Comprehension would be affected.

s you would like to make that would help us understand how well this state’s standards represent the reading instructional area of
49

General Questions

n which the state’s reading content standards provide support for the type of reading instruction called for in the five essential

mprove the level of this support?

h any of the state’s reading content standards are inconsistent with the five essential areas of reading instruction identified based

by the National Reading Panel. What could be done to reduce these inconsistencies?
50

Appendix E. State Assessment Review Form

Sample Page

atewide

eading

essments

Used for

RF

Outcome?

Component(s)

Specified?

Phonemic

Awareness Phonics Fluency Vocabulary Compre-hension

Date RFApplicationApproved

e of

ssment Yes/No Yes/No Only/Plus Only/Plus Only/Plus Only/Plus Only/Plus 

52

ndix F. Results of Analyses of Ratings of State K-3 Reading Standards

Table F-1

 Standards Representing Student Knowledge and Skills in Each Essential Element in 2003-04

Number of Standards Clearly

and Explicitly Representing

Element

Number of Standards

Ambiguously Representing

Element

Total Number of Standards

Representing Element

Range Mean % Range Mean % Range Mean %

0-11 3.8 60.3 0-8 2.5 39.7 0-14 6.3 100.0

0-40 11.6 73.9 0-14 4.1 26.1 1-40 15.7 100.0

0-14 4.2 65.6 0-8 2.2 34.4 2-16 6.4 100.01-35 14.8 79.6 0-13 3.8 20.4 3-40 18.6 100.0

9-123 47.5 83.5 0-25 9.4 16.5 9-130 56.9 100.0

12-216 81.9 78.9 4-40 21.9 21.1 19-227 103.8 100.0

Table F-2

Distribution of States by Appropriateness of Grade Level to Which Standards

Representing Each Essential Element Are Assigned in 2003-04

All Are

Appropriate

Most Are

Appropriate

About Half Are

Appropriate

Few Are

Appropriate

None Are

Appropriate

Mean

Appropriateness*

N % N % N % N % N % M Label

6 33.3 6 33.3 4 22.2 1 11.1 0 0.0 2.3 Most/Half

8 40.0 7 35.0 4 20.0 1 5.0 0 0.0 2.2 Most

13 65.0 6 30.0 1 5.0 0 0.0 0 0.0 1.7 All/Most15 75.0 4 20.0 1 5.0 0 0.0 0 0.0 1.7 All/Most16 80.0 4 20.0 0 0.0 0 0.0 0 0.0 1.5 All/Most

ed 1 (for All) through 5 (for None) before calculating the mean. The labels were assigned to the resulting averages

1-1.25=All, 1.25-1.75=All/Most, 1.75-2.25=Most, 2.25-2.75=Most/Half, 2.75-3.25=Half, etc.

ds representing phonemic awareness.
53

Table F-3

ates by Adequacy of Coverage of Standards Representing Each Essential Element in 2003-04

All Are

Covered

Most Are

Covered

About Half Are

Covered

Few Are

Covered

None Are

Covered Mean Coverage*

N % N % N % N % N % M Label

7 35.0 5 25.0 2 10.0 4 20.0 2 10.0 2.7 Most/Half4 20.0 12 60.0 1 5.0 3 15.0 0 0.0 2.4 Most/Half

6 30.0 5 25.0 8 40.0 1 5.0 0 0.0 2.4 Most/Half4 20.0 8 40.0 6 30.0 2 10.0 0 0.0 2.5 Most/Half3 65.0 5 25.0 1 5.0 1 5.0 0 0.0 1.6 All/Most

ed 1 (for All) through 5 (for None) before calculating the mean. The labels were assigned to the resulting averages

1-1.25=All, 1.25-1.75=All/Most, 1.75-2.25=Most, 2.25-2.75=Most/Half, 2.75-3.25=Half, etc.

Table F-4

by Appropriateness of Level of Detail of Standards Representing Each Essential Element in 2003-04

Much Too

road/General

Somewhat Too

Broad/General

Appropriate

Level of Detail

Somewhat Too

Specific

Much Too

Specific

Mean Level of

Detail*

N % N % N % N % N % M Label

0 0.0 9 50.0 9 50.0 0 0.0 0 0.0 2.2

Somewhat

Broad/

Appropriate

2 10.0 6 30.0 12 60.0 0 0.0 0 0.0 2.2

Somewhat

Broad/

Appropriate

4 20.0 9 45.0 7 35.0 0 0.0 0 0.0 1.9

Somewhat

Broad

1 5.0 5 25.0 14 70.0 0 0.0 0 0.0 2.5

Somewhat

Broad/

Appropriate

0 0.0 2 10.0 15 75.0 3 15.0 0 0.0 3.0 Appropriate

ed 1 (for Much Too Broad) through 5 (for Much Too Specific) before calculating the mean. The labels were assigned

d on the following rules: 1-1.25=Much Too Broad, 1.25-1.75=Much Too Broad/Somewhat Broad, 1.75-

2.75=Somewhat Broad/Appropriate, 2.75-3.25=Appropriate, etc.

ds representing phonemic awareness.
Our mission is to ensure equal access to education

and to promote educational excellence throughout the nation.

www.ed.gov