Vol. 4 No. 1 August 2000

How Does NAEP Select Schools and Students?

The National Assessment of Educational Progress

(NAEP), the nation’s report card, is the only nationally

representative and continuing assessment of what

America’s students know and can do in various subject

areas. Since 1969, assessments have been conducted

periodically in reading, mathematics, science, writing,

history, geography, and other fields. Only information

related to academic achievement is collected under this

program. NAEP guarantees the privacy of individual

students and their families.

Each year, the National Center for Education Statistics

(NCES), an agency within the U.S. Department of Education, asks thousands of schools and students to participate in NAEP. Their participation in NAEP allows

NCES to provide accurate information for the nation as

a whole on how our students are performing in a variety

of academic subjects.

In addition, NCES uses NAEP data to compare the performance of students in individual states against the national average, and against students in other states. No

other assessment can do this. NCES also uses NAEP

data to make comparisons of student performance over

time, both nationally and at the state level. These comparisons can be made for all student subgroups—males

and females, for example, and for blacks, whites, Hispanics, Asians, and American Indians, as long as the

subgroups are well represented in the sample.

Although NCES releases several major NAEP reports

every year, most people are not familiar with NAEP.

The national press usually refers to NAEP reports as “a

national study” or “a federal report,” coming from “the

federal government” or “the U.S. Department of Education,” without identifying NAEP as the source. Because

people do not know what NAEP is or what it does, they

can be skeptical about why they should participate in

NAEP. In fact, NAEP is a unique source of information

on education in America, information that policymakers

and the general public need if they are to make informed

decisions about education in America.

How did I or my school get

selected for a NAEP assessment?

NCES uses a multistage sampling method to select the

schools and students who will participate in a given

NAEP assessment (see figure 1). This method allows

NCES to give an accurate picture of student performance, while keeping the burden on students and schools

to a minimum. For example, a NAEP national assessment in reading will require the participation of about

8,000 fourth-grade students (out of a population of approximately 3.5 million). Yet this assessment will provide data that are used to estimate the reading performance of all 3.5 million students, and also for subgroups—Hispanic fourth-graders, for example, who

constitute only 11 percent of the fourth-grade population.

National assessments

For its national assessments, NCES begins by dividing

the country into about 1,000 geographical sampling

units, varying in population from several million to
2

45,000 (figure 1). The larger a sampling unit is, the

greater its chance of being selected. In fact, the 22 largest sampling units—the nation’s largest metropolitan

areas—are always selected. The remainder are arranged

in 72 categories, defined by such factors as geographical

location, minority population, education level, and income level. By selecting one sampling unit from each

category, NCES will obtain a representative student

sample for the nation.

Within a sampling unit, NCES arranges schools in categories similar to the categories used in the selection of

sampling units themselves. NCES deliberately oversamples private schools and schools with high minority

populations. Oversampling allows NCES to obtain accurate information on the performance of minority and

private-school students.1

State assessments

NAEP national and state assessments are done separately. States participate in NAEP state assessments on a

voluntary basis. An increasing number of states, after

deciding to participate in a NAEP state assessment, are

requiring schools to participate in the assessment.

In a NAEP state assessment the state is not divided into

geographical sampling units. The entire state is treated

as a sampling unit, and the schools are arranged in categories in the same manner as in a national assessment.

At the state level, NCES does not oversample schools

with a large number of minority students, although a

state may augment its sample to obtain greater representation of subpopulations if it wishes. In the past,

NCES collected data at the state level for private-school

students, but in most states the sample for these students

was too small to be useful.

After consulting with organizations of private schools,

NCES will test a new approach to collecting privateschool data nationally in 2000.2

Selection of students

Once a school has been selected for either a state or national assessment, students within the school are classified by grade (4th, 8th, or 12th) and then selected at random.3

 The number of students assessed in a school for a

single assessment will usually range from about 30 to

150. The number of students assessed in a school is directly dependent on the size of the school and the type

of assessment. The larger the school, the greater the

number of students assessed.

Students are selected from the entire eligible student

population. For example, if there are three 4th-grade

classrooms in a school, students are likely to be selected

from all three classrooms. This avoids the bias that

might result from taking students from only one of several classrooms. Bias could arise from any number of

factors—the use of ability tracking by a school, for example, or the efforts of an exceptional teacher.

Schools are frequently selected for more than one assessment. For example, in 2000, NAEP will be conducting assessments in mathematics, science, and reading. Because each assessment has different materials,

instructions, and time periods, each assessment is administered in a separate session, even if several assessments are being administered in the same school on the

same day.

Why should I participate in NAEP?

High participation rates help ensure the quality of

NAEP. If schools or students selected to participate in

NAEP decline to do so in large quantities, this affects

the validity of NAEP data. Overall, over 80 percent of

schools, and 85–95 percent of students, selected to participate in NAEP do participate. NCES cannot report

data from a participating state if less than 70 percent of

the initially selected schools agree to participate, and

this has happened on occasion.

When a school declines to participate, the possibility

exists that the school was somehow different from those

schools that agreed to participate, and thus that the actual sample is not representative of the population as a

whole. For example, low-performing schools might be

more likely to decline than middle- and high-performing

schools. Because there is no survey information available on a nonparticipating school, NCES cannot determine whether or not the school was in fact different. The

greater the number of nonparticipating schools, the

greater the possibility that the sample is not representative. Nonparticipation of individual students has a

smaller but similar effect.

Can schools or students volunteer to

participate in NAEP?

NCES cannot accept either schools or students as voluntary participants in NAEP. The sampling techniques

that NCES uses in NAEP allow NCES to produce detailed results on student performance, while using only a

small sample, and usually requiring no more than an

hour and a half of a student’s time for administration of

the assessment. The validity of these results depends on

the ability of NCES to know the probability of selection

at every stage of the process, from geographical sam-
Geographical Sampling Units

In a national assessment, the United State is divided

geographically into approximately 1,000 geographical

sampling units. Ninety-four are selected for each assessment.

In a state assessment, states are not subdivided

geographically.

Schools

In national assessments, NCES oversamples schools

with large minority populations, in order to ensure adequate samples of minority students. Private schools are

also oversampled, for the same reason. In calculating

overall results, NCES adjusts for oversampling.

In state assessments, NCES does not oversample minority students. In 2000, NCES will not include private

schools in state assessments.

Students

Within each school, students are grouped by grade or age and selected at random

from the eligible student population. The larger the school, the greater the number of

students that will be selected. The usual range is from a low of 30 to a high of 150.

Figure 1.NAEP Sampling Procedures

3
3

pling unit to school to student. Certain types of schools

or students may be more likely to volunteer for NAEP,

and these differences could be related to performance.

For example, higher performing schools or individuals

might be more likely to volunteer. Inclusion of such

schools or students could result in a sample that was not

representative of the whole population. For this reason,

NCES cannot accept schools or students not in the sample that volunteer to participate in NAEP.

Conclusion

For the past 30 years, NAEP has provided policymakers,

educators, and the general public with valuable information on the academic performance of students in

American schools. This is achieved at minimal burden

for schools and students through the sampling and assessment techniques developed and employed by NCES.

Participating in NAEP does require a commitment of

time and effort by schools and students. This willingness

to participate makes it possible for NCES to provide the

nation and the states with unique data on student performance. If all schools and students selected to participate in NAEP do participate, the accuracy and timeliness of the NAEP data are enhanced. High participation

rates in NAEP allow Americans to make informed decisions about education and educational policy using the

best possible data.

Endnotes

1 In calculating the average performance of all students, NCES adjusts for the fact that certain groups were oversampled.

2

 In the past, NCES published results for private school students as a

whole, and separately for Catholic and “other” private schools. In the

national assessments conducted in 2000, with the oversampling of

private schools, NCES will publish data on Catholic, Lutheran, Conservative Christian, “other religious,” “other nonsectarian,” and

schools that are members of the National Association of Independent

Schools.

3

 Most NAEP assessments assess students by grade, but some assess

them by age (9, 13, or 17).

For More Information

The NAEP Guide, 1997−1998 Edition. Single copies are

available free from ED Pubs, P.O. Box 1398, Jessup,

MD 20794–1398. Copies may also be obtained over the

World Wide Web

(http://nces.ed.gov/nationsreportcard/guide/97990.html).

The Focus on NAEP series briefly summarizes information about the ongoing development and implementation

of the National Assessment of Educational Progress

(NAEP). This series is a product of the National Center

for Education Statistics (NCES), Gary Phillips, Acting

Commissioner, and Peggy Carr, Associate Commissioner for Education Assessment. This issue was written

by Alan Vanneman, of the Education Statistics Services

Institute, and Sheida White, of NCES. For more information, contact Sheida White at 202−502−7473.

To order other NAEP publications, call toll free 1–877–

4ED–Pubs (1–877–433–7827), TTY/TDD 1–877–576–

7734;

E–mail: edpubs@inet.ed.gov;

Internet: http://www.ed.gov/pubs/edpubs.html

The NCES World Wide Web Home Page is

http://nces.ed.gov/

The NAEP World Wide Web Home Page is

http://nces.ed.gov/nationsreportcard/