Appendix G

NAEP 2015 Sample Design Memo
\\westat.com\dfs\NAEPLIB\2015\Memos\School Sampling\Public Grades 4 & 8-A\2015-m01v01a.docx 

Date: February 26, 2014 Memo: 2015- 

1.1A/1.1B/1.1G/1.1D/1.1E 

To: William Ward, NCES

Nicole Beaulieu, ETS

Ed Kulick, ETS

David Freund, ETS

Amy Dresher, ETS

Connie Smith, Pearson

Dianne Walsh

Lauren Byrne

Debby Vivari

Rob Dymowski

Jennifer Kali

David Ferraro

Erin Wiley

Dwight Brock

Amy Lin

Chris Averett 

Kavemuii Murangi 

David Hubble

Sarah Shore 

William Wall

 

From: Lloyd Hicks and John Burke

Reviewer: Keith Rust 

Subject: Sample Design for 2015 NAEP - Draft

I. Introduction 

For 2015, the NAEP assessment involves the following components:

A. National assessments in reading, mathematics, and science at grades 4, 8, and 12; 

B. State-by-state and Trial Urban District Assessment (TUDA) assessments in reading and

mathematics for public schools at grades 4 and 8;

C. A state-by-state assessment in science for public schools at grades 4 and 8; 

D. No state-by-state assessments in any states at grade 12;
Memorandum: 2015 - 1.1A/1.1B/1.1G/1.1D/1.1E February 26, 2014

Page 2 of 23

E. Technology-based assessment (TBA) start-ups in reading, mathematics, and science at 

grades 4, 8, and 12;

F. Pilot assessments of science interactive computer tasks (ICTs) and hybrid hands-on 

tasks (HOTs) at grades 4, 8, and 12; 

G. A special national study of Knowledge and Skills Appropriate (KaSA) items in 

mathematics, at grades 4 and 8, including Puerto Rico.

Below is a summary list of the features of the 2015 sample design.

1. The alpha samples for grades 4 and 8 public, the gamma sample for grade 12 public, 

and the delta samples for private schools at grades 4, 8, and 12 will be used for the 

operational assessments in reading, mathematics, science, and meaning vocabulary. 

These samples will also be used for the national KaSA special studies at grades 4 and 8. 

2. There will be two sets of relatively small PSU-based samples for the TBA start-up 

assessments and for the pilot assessments of science ICTs/HOTs at grades 4, 8, and 12. 

The public school samples for these assessments are known as the Beta samples and the 

corresponding private samples are known as the Epsilon samples.

3. As in recent NAEP studies, Trial Urban District Assessment (TUDA) samples will form 

part of the corresponding state samples, and the state samples will form part of the 

national sample. There are twenty-one Trial Urban District Assessment (TUDA) 

participants. Twenty of the twenty-one participated also participated in 2013. Milwaukee 

WI is no longer participating, but Duvall County FL has been added in.

4. All states but Alaska, California, Colorado, Pennsylvania, and Vermont have signed on 

to do science at grades 4 and 8. BIE schools will not be doing science either. 

5. Most schools in states doing science will receive a 1:1:1 RMS spiral (1/3 reading, 1/3 

math, and 1/3 science). Most schools in states not doing science will receive a 10:10:1 

RMS spiral (10/21 reading, 10/21 math, and 1/21 science) to ensure that these states 

are sufficiently represented in the national science sample.

6. All TUDAs but Albuquerque, NM, Chicago, IL, and New York City, NY – those that 

constitute a relatively large proportion of their state population – will receive the 10:10:1 

RMS spiral. Albuquerque, Chicago, and New York will receive the 1:1:1 RMS spiral

(note that they are in states that have signed on for science). 

7. A small subset of schools from each jurisdiction, regardless of whether they are doing 

science, will receive a spiral that includes meaning vocabulary and KaSA booklets at a 

rate that will support national level reporting. This spiral, the NL spiral, will consist of 

6/21 reading, 6/21 math, 6/21 science, 2/21 meaning vocabulary, and 1/21 KaSA 

booklets. The subsample of schools selected for the NL spiral in each state is 

proportional to the representation of each state in the nation.
Memorandum: 2015 - 1.1A/1.1B/1.1G/1.1D/1.1E February 26, 2014

Page 3 of 23

8. There will be no samples in territories, other than for Puerto Rico at grades 4 and 8 for 

the KaSA study.

9. Most BIE schools will receive a 10:10:1 RMS spiral and a small subset will receive the 

NL spiral. 

10. All BIE schools and students will be included in the operational samples at grades 4 and 

8. This is because, after a hiatus in 2013, the National Indian Education Study (NIES) is 

resuming. Having all BIE students in sample is designed to provide detailed national 

results for American Indian and Alaskan Native (AIAN) students in reading and

mathematics, as part of the National Indian Education Study (NIES). Because science 

will be spiraled with math and reading, the science sample will be spread across almost 

all BIE schools at grades 4 and 8.

11. As in 2013, the Department of Defense Schools are expected to be reported as a single 

jurisdiction (DoDEA), instead of the two components of domestic (DDESS) and 

overseas (DoDDS). DoDEA schools will receive either the 1:1:1 RMS spiral or the NL 

spiral.

12. At grade 12, there will be no state-level samples. 

13. All private schools will receive a NL spiral.

14. Oversampling of private schools at grades 4 and 8 will be done at the same level as 

2013. Response rates permitting, this will allow separate reporting for reading, 

mathematics, and science for Catholic and non-Catholic schools, but no further. As in 

2013 there will be no oversampling of private schools at grade 12.

15. The sample sizes of assessed students for these various components are shown in Table 

1 (which also shows the approximate numbers of participating schools). 
Memorandum: 2015 - 1.1A/1.1B/1.1G/1.1D/1.1E February 26, 2014

Page 4 of 23

Table 1. Target sample sizes of assessed students, and expected number of participating

schools, for 2015 NAEP

 

Spiral Jurisdictions Students 

Spiral Indic. Total 

States (incl. 

DC, BIE, 

DoDEA) 

Urban 

districts 

Public school 

students 

Private 

school 

students 

Grade 4 

nat'l/state reading RS, RM, NL 53 21 133,000 3,000 136,000

nat'l/state math RS, RM, NL 53 21 133,000 3,000 136,000

nat'l/state science RS, RM, NL 47 105,000 3,000 108,000

reading - vocabulary NL 7,200 800 8,000

math KaSA NL 3,600 400 4,000

Puerto Rico KaSA PR 1 5,000 0 5,000

Total - alpha 4 386,800 386,800 

Total- delta 1 10,200 10,200 

Typical max. no. students/school 90 105

Average assessed students/school 51 25 50

Total schools - alpha, delta 7,550 410 7,960 

Reading - TBA Start-up TB 10,800 1,200 12,000

Mathematics - TBA Start-up TB 10,800 1,200 12,000

Science - TBA Start-up TB 7,200 800 8,000

Science ICT Pilot TB 3,600 400 4,000

Science HOT Pilot TB 2,700 300 3,000

Total - beta 1 35,100 35,100 

Total - epsilon 1 3,900 3,900 

Typical max. no. students/school 50 50

Average assessed students/school 41 25 39

Total schools - beta, epsilon 855 155 1,010

Total number of students grade 4 421,900 14,100 436,000 

Total number of schools grade 4 8,405 565 8,970 
Memorandum: 2015 - 1.1A/1.1B/1.1G/1.1D/1.1E February 26, 2014

Page 5 of 23

Table 1. Target sample sizes of assessed students, and expected number of participating 

schools, for 2015 NAEP (Continued)

 

Spiral Jurisdictions Students 

Spiral Indic. Total 

States (incl. 

DC, BIE, 

DoDEA) 

Urban 

districts 

Public school 

students 

Private 

school 

students 

Grade 8 

nat'l/state reading RS, RM, NL 53 21 133,000 3,000 136,000

nat'l/state math RS, RM, NL 53 21 133,000 3,000 136,000

nat'l/state science RS, RM, NL 47 105,000 3,000 108,000

reading - vocabulary NL 7,200 800 8,000

math KaSA NL 3,600 400 4,000

Puerto Rico KaSA PR 1 5,000 0 5,000

Total - alpha 4 386,800 386,800 

Total- delta 1 10,200 10,200 

Typical max. no. students/school 90 105

Average assessed students/school 64 26 62

Total schools - alpha, delta 6,020 400 6,420 

Reading - TBA Start-up TB 15,300 1,700 17,000

Mathematics - TBA Start-up TB 10,800 1,200 12,000

Science - TBA Start-up TB 7,200 800 8,000

Science ICT Pilot TB 3,600 400 4,000

Science HOT Pilot TB 2,700 300 3,000

Total – beta 1 39,600 39,600 

Total – epsilon 1 4,400 4,400 

Typical max. no. students/school 50 50

Average assessed students/school 44 25 41

Total schools - beta, epsilon 900 175 1,075 

Total number of students grade 8 426,400 14,600 441,000 

Total number of schools grade 8 6,920 575 7,495 
Memorandum: 2015 - 1.1A/1.1B/1.1G/1.1D/1.1E February 26, 2014

Page 6 of 23

Table 1. Target sample sizes of assessed students, and expected number of participating 

schools, for 2015 NAEP (Continued)

 

Spiral Jurisdictions Students 

Spiral Indic. Total 

States (incl. 

DC, BIE, 

DoDEA) 

Urban 

districts 

Public school 

students 

Private 

school 

students 

Grade 12 

nat'l/state reading NL 11,700 1,300 13,000

nat'l/state math NL 12,600 1,400 14,000

nat'l/state science NL 9,900 1,100 11,000

reading - vocabulary NL 5,400 600 6,000

Total – gamma 1 39,600 39,600 

Total- delta 1 4,400 4,400 

Typical max. no. students/school 100 100

Average assessed students/school 76 80 77

Total schools - gamma, delta 520 55 575 

Reading - TBA Start-up TB 13,500 1,500 15,000

Mathematics - TBA Start-up TB 10,800 1,200 12,000

Science - TBA Start-up TB 8,100 900 9,000

Science ICT Pilot TB 3,600 400 4,000

Science HOT Pilot TB 2,700 300 3,000

Total – beta 1 38,700 38,700 

Total – epsilon 1 4,300 4,300 

Typical max. no. students/school 50 50

Average assessed students/school 42 43 42

Total schools - beta, epsilon 920 100 1,020 

Total number of students grade 12 78,300 8,700 87,000 

Total number of schools grade 12 1,440 155 1,595 

 

GRAND TOTAL STUDENTS 926,600 37,400 964,000

GRAND TOTAL SCHOOLS 17,510 1,990 19,500

II. Assessment Types 

The assessment spiral types are shown in Table 2. Five different spirals will be used at each of the 

grades 4 and 8, and two at grade 12. Session IDs contain six digits, traditionally. The first two digits 

identify the assessment “type” (subjects and type of spiral in a general way). Grade is contained in 

the second pair of digits, and the session sequential number (within schools) in the last two digits.

For example, session RS0401 denotes the first grade 4 reading, mathematics, and science operational 

assessment in a given school.
Memorandum: 2015 - 1.1A/1.1B/1.1G/1.1D/1.1E February 26, 2014

Page 7 of 23

Table 2. NAEP 2015 assessment types and IDs

ID Type Subjects Grades Schools Comments 

RS Operational 

P&P

Reading, math, science 

(1:1:1) 4, 8 Public

Most public schools in states 

doing science, plus TUDAs in 

some of those states

RM Operational 

P&P

Reading, math, science 

(10:10:1) 4, 8 Public

Most public schools in states not 

doing science, plus TUDAs in 

those states and most others

NL Operational 

P&P

Reading, math, science, 

Meaning Vocab, KaSA 

(6:6:6:2:1, grades 4 & 8; 

13:14:11:6:0, grade 12)

4, 8, 12 Public, 

private

All private schools; all public 

schools at Grade 12; some 

public schools in each 

jurisdiction (except Puerto Rico 

and DoD overseas) at grades 4 

and 8

PR Operational 

P&P KaSA Mathematics 4, 8 Public Puerto Rico only

TB TBA start-ups 

& pilots

Reading, math, science, 

science ICT, science HOT 4, 8, 12 Public, private

Booklets will not be used in any 

other jurisdiction.

III. Sample Types and Sizes 

In similar fashion to past years (but somewhat different), we will identify five different types of 

school samples: Alpha, Beta, Gamma, Delta, and Epsilon. These distinguish sets of schools that will 

be conducting distinct portions of the assessment. 

1. Alpha Samples at Grades 4 and 8 

These are public school samples for grades 4 and 8. They will be used for the operational state-bystate assessments in reading, math, and science and contribute to the national samples for these 

subjects as well. They will also be used for the national meaning vocabulary assessment and for the 

national KaSA special studies. There will be alpha samples for each state, DC, DoDEA, BIE, and 

Puerto Rico. The alpha samples will not be used for any TBA start-up assessments or pilot 

assessments. 

The details of the target student sample sizes for the alpha samples are as follows:

A. At each grade, the target student sample size for a state depends on whether or not it signed 

on for science. For each state that signed on for science, the target student sample size is 
Memorandum: 2015 - 1.1A/1.1B/1.1G/1.1D/1.1E February 26, 2014

Page 8 of 23

7,600: roughly 2,200 assessed each for math, reading, and science. In these states the RS 

session type will be used. For the four states not signed on for science and BIE, the target 

student sample sizes are 5,300: 2,200 (assessed) each for math and reading and 220 for science. 

In these jurisdictions the RM session type will be used.

B. A national-level subsample will be applied to the sample schools at each grade with a target 

student sample size of 86,600. Schools that are selected for this subsample will be given the 

NL spiral.

C. There will be samples for twenty-one TUDA districts. For two of the five largest (New York 

City and Chicago) where we require a sizeable science sample so as to ensure an adequate state 

sample, the student target samples sizes are three-quarters the size of a science state (5,670). 

For the remaining three large TUDA districts (Los Angeles, Miami-Dade, and Houston) 

where only a smaller science sample is required, the student target sample sizes are threequarters the size of a non-science state sample (4,000). For Albuquerque, a small TUDA 

where a sizeable science sample is required, the student target sample size is half the size of a 

science state (3,780), whereas the remaining small TUDA districts where only a smaller science 

sample is required, the student target sample sizes are one-half the size of a non-science state 

sample (2,650). 

D. Note, that above, there is a conflict between sample size requirements at the state level, and 

the TUDA district level. This will be resolved as in previous years: the districts will have the 

target samples indicated in B, and reflected in Table 4. For the states that contain one or more 

of these districts, the target sample size indicated in A (and shown in Table 4) will be used to 

determine a school sampling rate for the state, which will be applied to the balance of the state 

outside the TUDA district(s). Thus the target student sample sizes, shown in Table 4, for 

states that contain a TUDA district, are only ‘design targets’, and are smaller than the final 

total sample size for the state, but larger than the sample for the balance of the state, exclusive 

of its TUDA districts. In the case of the District of Columbia, the state sample size 

requirement is that all schools and students be included. This renders moot any requirements 

for the DC TUDA sample, which by default consists of all schools operated by the DCPS 

district (but excludes charter schools in DC, even though those are all included in the state 

sample, as these are not operated by DCPS).

E. In Puerto Rico, the target sample size is 5,750 per grade (grades 4 and 8), with the goal of 

assessing 5,000 students. Only KaSA mathematics will be assessed in Puerto Rico (PR session 

type).

As in past state-by-state assessments, schools with fewer than 20 students in the grade in question 

will be sampled at a moderately lower rate than other schools (at least half, and often higher, 

depending upon the size of the school). This is in implicit recognition of the greater cost and burden 

associated with surveying these schools.

As mentioned above, the NAEP 2015 design includes an oversample of high proportion American 

Indian schools in certain states (as part of the NIES design). These schools will be sampled at higher 

rates than the other schools. The NIES oversample will take place in Arizona, Minnesota, North 
Memorandum: 2015 - 1.1A/1.1B/1.1G/1.1D/1.1E February 26, 2014

Page 9 of 23

Carolina, Oregon, Utah, and Washington. Schools with relatively large percentages of American 

Indian students will be separately stratified, as explained below, and oversampled by factors ranging 

from 2 to 6 based on state and grade. Table 3 below shows the thresholds used to define the NIES 

oversampling strata along with their corresponding oversampling factors.

Table 3. Percent American Indian thresholds and oversampling factors for the NIES school 

oversample by state and grade

State 

Grade 4 Grade 8 

Percent American 

Indian thresholds 

Oversampling 

factor 

Percent American 

Indian thresholds 

Oversampling 

factor 

Arizona 50 3 50 2

Utah 5 5 5 5

Minnesota 10 5 10 4

North Carolina 15 6 10 6

Oregon 10 6 15 6

Washington 10 6 15 6

Table 4 shows the target student sample sizes, and the approximate counts of schools to be selected 

in the alpha samples, along with the school and student frame counts, by state and TUDA districts 

for grades 4 and 8. The table also identifies the jurisdictions where we take all schools and where we 

take all students.
Memorandum: 2015 - 1.1A/1.1B/1.1G/1.1D/1.1E February 26, 2014

Page 10 of 23

Table 4. Grade 4 and 8 school and student frame counts, expected school sample sizes, and initial target student sample sizes for 

the 2013 state-by-state and TUDA district assessments (Alpha samples)

 Grade 4 Grade 8 

Jurisdiction 

Schools in 

frame 

Schools in 

sample 

Students 

in frame 

Overall 

target 

student 

sample 

size 

 

Schools in 

frame 

Schools in 

sample 

Students in 

frame 

Overall 

target 

student 

sample size 

Alabama 719 110 57,293 7,600 462 96 58,020 7,600 

Alaska 361 158 9,870 5,300 276 118 9,865 5,300

Arizona 1,184 111 83,596 7,600 784 104 82,121 7,600

Arkansas 488 113 36,667 7,600 300 99 35,481 7,600

Bureau of Indian Education 136 136 3,330 3,330 ** 112 112 2,904 2,904 **

California 5,907 103 470,328 7,600 2,876 96 454,580 7,600

Colorado 1,041 97 65,699 5,300 554 93 62,665 5,300

Connecticut 575 114 40,276 7,600 315 96 41,529 7,600

Delaware 118 94 10,099 7,600 61 61 9,885 7,600 *

District of Columbia 122 122 5,113 5,113 ** 73 73 4,539 4,539 **

DoDEA Schools 108 108 7,514 7,514 ** 63 63 5,596 5,596 **

Florida 2,183 96 204,096 7,600 1,173 94 201,815 7,600

Georgia 1,248 92 130,599 7,600 552 87 129,337 7,600

Hawaii 203 112 14,902 7,600 83 62 13,132 7,600

Idaho 373 127 22,095 7,600 212 101 21,610 7,600

Illinois 2,310 121 152,571 7,600 1,610 110 153,611 7,600

Indiana 1,059 110 78,537 7,600 496 90 80,280 7,600

Iowa 647 138 36,039 7,600 374 108 35,713 7,600

Kansas 704 145 36,223 7,600 395 115 35,689 7,600

Kentucky 734 113 51,728 7,600 407 99 50,978 7,600

Louisiana 754 115 54,823 7,600 479 102 52,027 7,600

Maine 326 169 13,609 7,600 199 116 13,687 7,600

Maryland 893 110 64,759 7,600 379 93 60,562 7,600

Massachusetts 955 111 70,874 7,600 477 90 71,996 7,600

Michigan 1,731 119 113,008 7,600 1,071 102 116,734 7,600

Minnesota 934 117 62,923 7,600 702 114 62,522 7,600

Mississippi 422 106 38,029 7,600 286 95 37,697 7,600

Missouri 1,160 130 68,565 7,600 691 110 68,104 7,600
Memorandum: 2015 - 1.1A/1.1B/1.1G/1.1D/1.1E February 26, 2014

Page 11 of 23

Table 4. Grade 4 and 8 school and student frame counts, expected school sample sizes, and initial target student sample sizes for 

the 2013 state-by-state and TUDA district assessments (Alpha samples) (Continued) 

 Grade 4 Grade 8 

Jurisdiction 

Schools in 

frame 

Schools in 

sample 

Students 

in frame 

Overall 

target 

student 

sample 

size 

 

Schools in 

frame 

Schools in 

sample 

Students in 

frame 

Overall 

target 

student 

sample size 

Montana 392 214 11,146 7,600 273 157 10,789 7,600 

Nebraska 536 167 22,939 7,600 303 126 21,793 7,600

Nevada 385 97 33,934 7,600 161 84 33,313 7,600

New Hampshire 268 141 13,975 7,600 141 90 14,348 7,600

New Jersey 1,360 114 99,003 7,600 750 97 98,145 7,600

New Mexico 440 130 25,977 7,600 225 102 25,261 7,600

New York 2,397 104 199,215 7,600 1,443 94 198,874 7,600

North Carolina 1,422 103 116,094 7,600 703 91 116,743 7,600

North Dakota 258 258 7,760 7,760 ** 183 183 7,403 7,403 **

Ohio 1,778 115 129,775 7,600 1,118 100 132,975 7,600

Oklahoma 872 136 50,101 7,600 584 120 47,828 7,600

Oregon 750 131 42,558 7,600 422 108 42,917 7,600

Pennsylvania 1,649 91 129,728 5,300 900 87 133,900 5,300

Puerto Rico 988 171 33,865 5,750 401 125 32,888 5,750

Rhode Island 166 121 10,751 7,600 59 59 10,248 7,600 *

South Carolina 633 101 55,665 7,600 307 89 55,154 7,600

South Dakota 314 197 10,084 7,600 245 160 9,476 7,600

Tennessee 989 109 76,003 7,600 581 97 73,679 7,600

Texas 4,373 99 386,022 7,600 2,224 95 372,168 7,600

Utah 613 102 48,529 7,600 244 91 45,931 7,600

Vermont 218 218 6,247 6,247 ** 122 122 6,138 6,138 **

Virginia 1,115 100 95,536 7,600 384 86 94,826 7,600

Washington 1,231 116 79,005 7,600 610 99 78,960 7,600

West Virginia 423 157 20,728 7,600 196 98 20,744 7,600

Wisconsin 1,090 136 61,306 7,600 642 109 61,502 7,600

Wyoming 185 185 7,185 7,185 ** 87 87 6,902 6,902 **
Memorandum: 2015 - 1.1A/1.1B/1.1G/1.1D/1.1E February 26, 2014

Page 12 of 23

Table 4. Grade 4 and 8 school and student frame counts, expected school sample sizes, and initial target student sample sizes for 

the 2013 state-by-state and TUDA district assessments (Alpha samples) (Continued) 

 Grade 4 Grade 8 

Jurisdiction 

Schools in 

frame 

Schools in 

sample 

Students 

in frame 

Overall 

target 

student 

sample 

size 

 

Schools in 

frame 

Schools in 

sample 

Students in 

frame 

Overall 

target 

student 

sample size 

Albuquerque 95 52 7,426 3,780 40 32 6,721 3,780 

Atlanta 55 42 4,238 2,650 24 24 3,509 3,509 **

Austin 79 44 6,988 2,650 24 24 5,542 2,650 *

Baltimore City 127 56 6,385 2,650 102 53 5,654 2,650

Boston 72 52 4,165 2,650 41 41 3,867 2,650 *

Charlotte 103 43 11,447 2,650 45 32 10,717 2,650

Chicago 501 102 29,331 5,670 482 100 28,794 5,670

Cleveland 71 71 2,970 2,970 ** 66 66 2,755 2,755 **

Dallas 147 43 13,031 2,650 42 42 10,799 2,650 *

Detroit 66 48 4,213 2,650 50 50 3,224 3,224 **

Duval County, FL 115 44 9,999 2,650 48 31 9,098 2,650

Fresno 71 44 5,854 2,650 23 23 5,086 2,650 *

Hillsborough County, FL 171 44 15,515 2,650 85 43 15,159 2,650

Houston 176 64 17,149 4,000 64 46 13,400 4,000

Jefferson County, KY 100 44 7,589 2,650 46 29 7,345 2,650

Los Angeles 556 66 50,563 4,000 191 64 45,215 4,000

Miami 271 67 26,278 4,000 162 65 27,130 4,000

New York City 768 74 72,429 5,670 527 71 68,327 5,670

Philadelphia 162 48 11,438 2,650 126 46 9,573 2,650

San Diego 145 47 10,378 2,650 62 36 9,231 2,650

District of Columbia PS 80 80 3,291 3,291 ** 32 32 2,289 2,289 **

Total 52,240 7,831 3,776,296 443,343 28,770 6,215 3,725,584 441,811

Counts for states do not reflect the oversampling for their constituent TUDA districts.

Target student sample sizes reflect sample sizes prior to attrition due to exclusion, ineligibility, and nonresponse.

* identifies jurisdictions where all schools (but not all students) for the given grade are included in the NAEP sample.

** identifies jurisdictions where all students for the given grade are included in the NAEP sample.
Memorandum: 2015 - 1.1A/1.1B/1.1G/1.1D/1.1E February 26, 2014

Page 13 of 23

Table 5 consolidates the target student (and resulting school) sample size numbers, to show the total 

target sample sizes in each state, combining the TUDA targets with those for the balance of the 

state.
Memorandum: 2015 - 1.1A/1.1B/1.1G/1.1D/1.1E February 26, 2014

Page 14 of 23

Table 5. Total sample sizes, combining state and TUDA samples

 Grade 4 Grade 8 

Jurisdiction 

Schools 

in frame 

Schools 

in sample 

Students 

in frame 

Overall 

target 

student 

sample 

size 

 

Schools in 

frame 

Schools in 

sample 

Students in 

frame 

Overall 

target 

student 

sample size 

Alabama 719 109 57,293 7,600 462 96 58,020 7,600 

Alaska 361 158 9,870 5,300 276 118 9,865 5,300

Arizona 1,184 111 83,596 7,600 784 104 82,121 7,600

Arkansas 488 113 36,667 7,600 300 99 35,481 7,600

Bureau Of Indian Education 136 136 3,330 3,330 ** 112 112 2,904 2,904 **

California 5,907 277 470,328 15,818 2,876 229 454,580 15,902

Colorado 1,041 97 65,699 5,300 554 93 62,665 5,300

Connecticut 575 114 40,276 7,600 315 96 41,529 7,600

Delaware 118 94 10,099 7,600 61 61 9,885 7,600 *

District Of Columbia 122 122 5,113 5,113 ** 73 73 4,539 4,539 **

DoDEA Schools 108 108 7,514 7,514 ** 63 63 5,596 5,596 **

Florida 2,183 227 204,096 14,969 1,173 209 201,815 14,962

Georgia 1,248 130 130,599 10,004 552 108 129,337 10,903

Hawaii 203 112 14,902 7,600 83 62 13,132 7,600

Idaho 373 127 22,095 7,600 212 101 21,610 7,600

Illinois 2,310 197 152,571 11,805 1,610 185 153,611 11,842

Indiana 1,059 110 78,537 7,600 496 90 80,280 7,600

Iowa 647 138 36,039 7,600 374 108 35,713 7,600

Kansas 704 145 36,223 7,600 395 115 35,689 7,600

Kentucky 734 142 51,728 9,137 407 113 50,978 9,157

Louisiana 754 115 54,823 7,600 479 102 52,027 7,600

Maine 326 169 13,609 7,600 199 116 13,687 7,600

Maryland 893 150 64,759 9,501 379 133 60,562 9,541

Massachusetts 955 155 70,874 9,803 477 125 71,996 9,842

Michigan 1,731 162 113,008 9,966 1,071 149 116,734 10,614

Minnesota 934 117 62,923 7,600 702 114 62,522 7,600

Mississippi 422 106 38,029 7,600 286 95 37,697 7,600

Missouri 1,160 130 68,565 7,600 691 110 68,104 7,600

Montana 392 213 11,146 7,600 273 157 10,789 7,600
Memorandum: 2015 - 1.1A/1.1B/1.1G/1.1D/1.1E February 26, 2014

Page 15 of 23

Table 5. Total sample sizes, combining state and TUDA samples (Continued) 

 Grade 4 Grade 8 

Jurisdiction 

Schools 

in frame 

Schools 

in sample 

Students 

in frame 

Overall 

target 

student 

sample 

size 

 

Schools in 

frame 

Schools in 

sample 

Students in 

frame 

Overall 

target 

student 

sample size 

Nebraska 536 167 22,939 7,600 303 126 21,793 7,600 

Nevada 385 97 33,934 7,600 161 84 33,313 7,600 

New Hampshire 268 141 13,975 7,600 141 90 14,348 7,600 

New Jersey 1,360 114 99,003 7,600 750 97 98,145 7,600 

New Mexico 440 151 25,977 9,188 225 111 25,261 9,347 

New York 2,397 142 199,215 10,507 1,443 132 198,874 10,659 

North Carolina 1,422 137 116,094 9,500 703 116 116,743 9,551 

North Dakota 258 258 7,760 7,760 ** 183 183 7,403 7,403 **

Ohio 1,778 182 129,775 10,396 1,118 162 132,975 10,198 

Oklahoma 872 136 50,101 7,600 584 120 47,828 7,600 

Oregon 750 131 42,558 7,600 422 108 42,917 7,600 

Pennsylvania 1,649 131 129,728 7,482 900 126 133,900 7,571 

Puerto Rico 988 171 33,865 5,750 401 125 32,888 5,750 

Rhode Island 166 121 10,751 7,600 * 59 59 10,248 7,600 *

South Carolina 633 101 55,665 7,600 307 89 55,154 7,600 

South Dakota 314 197 10,084 7,600 245 160 9,476 7,600 

Tennessee 989 109 76,003 7,600 581 96 73,679 7,600 

Texas 4,373 250 386,022 16,167 2,224 202 372,168 16,291 

Utah 613 102 48,529 7,600 244 91 45,931 7,600 

Vermont 218 218 6,247 6,247 ** 122 122 6,138 6,138 **

Virginia 1,115 100 95,536 7,600 384 86 94,826 7,600 

Washington 1,231 116 79,005 7,600 610 99 78,960 7,600 

West Virginia 423 157 20,728 7,600 196 98 20,744 7,600 

Wisconsin 1,090 136 61,306 7,600 642 109 61,502 7,600 

Wyoming 185 185 7,185 7,185 ** 87 87 6,902 6,902 **

Total 52,240 7,831 3,776,296 443,343 28,770 6,215 3,725,584 441,811

Sample sizes for each state do reflect the samples in the TUDA districts within the state.

* identifies jurisdictions where all schools (but not all students) for the given grade are included in the NAEP sample.

** identifies jurisdictions where all students for the given grade are included in the NAEP sample. 
Memorandum: 2015 - 1.1A/1.1B/1.1G/1.1D/1.1E February 26, 2014

Page 16 of 23

Stratification 

Each state and grade will be stratified separately, but using a common approach in all cases. TUDA 

districts will be separated from their state, and each part stratified separately. The first level of 

stratification will be based on urban-centered type of location. This variable has 12 levels (some of 

which may not be present in a given state or TUDA district), and these will be collapsed so that each 

of the resulting location categories contains at least nine percent of the student population. Within 

each of the resulting location categories, schools will be assigned a minority enrollment status. This 

is based on the two race/ethnic groups that are the second and third most prevalent within the 

location category. If these groups are both low in percentage terms, no minority classification will be 

used. Otherwise three (or occasionally four) equal-sized groups (generally high, medium, and low 

minority) will be formed based on the distribution across schools of the two minority groups.

Within the resulting location and minority group classes (of which there are likely to be from three 

to fifteen, depending upon the jurisdiction), schools will be sorted by a measure derived from school 

level results from the most recent available state achievement tests at the relevant grade. In general, 

mathematics test results will be used, but where these are not available, reading results will be used. 

In the few states that do not have math or reading tests at grades 4 and 8 (or where we are unable to 

match the results to the NAEP school frame), instead of achievement data, schools will be sorted 

using a measure of socio-economic status. This is the median household income of the 5-digit ZIP 

Code area where the school is located, based on the 2012 ACS (5-year) data.

Once the schools are sorted by location class, minority enrollment class, and achievement data (or 

household income), a systematic sample of schools will be selected using a random start. Schools 

will be sampled with probability proportional to size. The exact details of this process are described 

in the individual sampling specification memos.

2. Beta Sample 

The beta sample comprises the national public school samples at grades 4, 8, and 12. This sample

will be used to conduct the TBA start-ups in reading, mathematics, and science and the pilot 

assessments in science interactive computer tasks (ICTs) and hybrid hands-on tasks (HOTs) at all 

three grades. Each of these samples will be nationally representative, with the exception that schools 

that are included in the grades 4 and 8 alpha samples with certainty will not be represented. This is 

because there is to be no overlap with the grade 4 and 8 alpha school samples. 
Memorandum: 2015 - 1.1A/1.1B/1.1G/1.1D/1.1E February 26, 2014

Page 17 of 23

The first stage of sampling for the beta sample is the selection of a sample of geographic primary 

sampling units (PSUs). These will be selected using the same design as for the 2014 assessments, but 

minimizing the overlap with PSUs in that assessment. A total 67 PSUs will be selected, representing 

the U.S. (but not including Puerto Rico, or DoDEA schools that are located outside the 50 states 

and D.C.). This PSU component is needed because of the operational complexities of administering 

the computer-based assessment. A select group of staff will be trained to administer those

assessments. The school stratification of the beta sample within PSUs will be by type of location and 

median household income. Schools with more than 15 percent black or Hispanic students will be 

sampled at twice the rate of other schools, so as to increase the student sample sizes for these two 

groups.

The number of students targeted per school will be 50 (either 25 start-up and 25 ICT/HOT or 50 

start-up). 

3. Gamma Sample 

This is the public school sample at grade 12, used for the reading, mathematics, science, and 

meaning vocabulary assessments. It will consist of a single sample of schools covering only nationallevel estimates. The number of students targeted per school will be 95 (28 reading, 30 for math, 24 

science, and 13 meaning vocabulary).

As in past assessments, modest oversampling of Black and Hispanic students will be undertaken in 

this sample. This will be carried out at the school level. Each school with more than 15 percent 

Black and Hispanic students will be given twice the selection probability of other schools of 

comparable size. This means that while about 53 percent of the student population (including over 

90 percent of the Black and Hispanic students) are in the oversampled schools, about 70 percent of 

the sample students will come from these schools.

Stratification 

The Gamma sample will have an implicit stratification, using a hierarchy of stratifiers and a 

serpentine sort. The highest level of the hierarchy is Census division (9 implicit strata). The next 

stratifier in the hierarchy is type of location, which has twelve categories. Many of the type of 
Memorandum: 2015 - 1.1A/1.1B/1.1G/1.1D/1.1E February 26, 2014

Page 18 of 23

location strata nested within Census divisions will be collapsed with neighboring type of location 

cells (this will occur if the expected school sample size within the cell is less than 4.0). These 

geographic strata will be subdivided using a dichotomous high minority status category for 

oversampling purposes. Schools with more than 10 Black or Hispanic students and greater than 15 

percent Black or Hispanic students will be considered high minority and placed in an oversampling 

stratum. All other schools will be considered low minority and placed in a regular sampling stratum. 

If the expected sample size for a high or low minority stratum is less than 8.0, it will be left as is. If 

the expected sample size is greater than 8.0, then the stratum will be subdivided into up to four 

substrata (two for expected sample size up to 12.0, three for expected sample size up to 16.0, and 

four for expected sample size greater than 16.0). For the regular sampling strata, the subdivision will 

be by state or groups of contiguous states. For the oversampling strata, the subdivision will be by 

percentage Black and Hispanic. Within these substrata, the schools are to be sorted by school type 

(public, BIE, DoDEA) and median household income from the 2012 5-year ACS (using a serpentine 

sort within the school type substrata).

4. Delta Samples 

These are the private school samples at grades 4, 8, and 12 for conducting the operational 

assessments in reading, mathematics, science, and meaning vocabulary, as well as the KaSA special 

studies at grades 4 and 8 that are incorporated into the NL spiral. The number of students targeted 

per schools will be 105 (30 reading, 30 math, 30 science, 10 meaning vocabulary, and 5 KaSA) at 

grades 4 and 8, and 95 (28 reading, 30 for math, 24 science, and 13 meaning vocabulary) at grade 12.

The sample sizes are large enough to report results by Catholic and non-Catholic at grades 4 and 8, 

but at grade 12 only for private schools as a whole. Approximately half the sample at each grade will 

be from Catholic schools. This is very similar to the design used in 2013, at each grade.

Stratification 

The private schools are to be explicitly stratified by private school type (Catholic/Other). Within 

each private school type, stratification will be by Census region (4 categories), type of location (12

categories), and enrollment size. In general, where there are few or no schools in a given stratum, 

categories will be collapsed together, always preserving the private school type. 
Memorandum: 2015 - 1.1A/1.1B/1.1G/1.1D/1.1E February 26, 2014

Page 19 of 23

5. Epsilon Sample 

This sample is analogous to the beta sample, but for private schools, at grades 4, 8, and 12. The 

same PSUs will be used as for the beta sample. There will be no oversampling of private schools for 

this sample. The epsilon sample schools will not overlap with the delta sample schools.

IV. New Schools 

To compensate for the fact that files used to create the NAEP school sampling frames are at least 

two years out of date at the time of frame construction, we will supplement the Alpha, Gamma, and 

Delta sample with new school samples at each grade. 

The new school samples will be drawn using a two-stage design. At the first stage, a minimum of ten 

school districts (in states with at least ten districts) will be selected from each state for public 

schools, and ten Catholic dioceses will be selected nationally for the private schools. The sampled 

districts and dioceses will be asked to review lists of their respective schools and identify new 

schools. Frames of new schools will be constructed from these updates, and new schools will be 

drawn with probability proportional to size using the same sample rates as their corresponding 

original school samples.

The school sample sizes in the above tables do not reflect new school samples.

V. Substitute Samples 

Substitute samples will be selected for each of the Beta, Gamma, Delta, and Epsilon samples. The 

substitute school for each original will be the next “available” school on the sorted sampling frame, 

with the following exceptions:

A. Schools selected for any NAEP samples will not be used as substitutes. 

B. Private schools whose school affiliation is unknown will not be used as substitutes. Also, 

unknown affiliated private schools in the original samples will not get substitutes.

C. A school can be a substitute for one and only one sample. (If a school is selected as a 

substitute school for grade 12, for example, it cannot be used as a substitute for either grade 4 

or grade 8.)
Memorandum: 2015 - 1.1A/1.1B/1.1G/1.1D/1.1E February 26, 2014

Page 20 of 23

D. A public school substitute will always be in the same state as its original school. 

VI. Contingency Samples 

The districts that are taking part in the TUDA program are volunteers. Thus it is possible that at 

some point over the next few months, a given district might choose to opt out of the TUDA 

program for 2015. However, it is not acceptable for all schools in such a district to decline NAEP, 

as then the state estimates will be adversely affected. Thus to deal with this possibility, in each 

TUDA district, subsamples of the alpha sample schools will be identified as contingency samples. In 

the event that the district withdraws from the TUDA program prior the selection of the student 

sample, all alpha sampled schools from that district will be dropped from the sample, with the 

exception of those selected in the contingency sample. The contingency sample will provide a 

proportional representation of the district, within the aggregate state sample. Student sampling in 

those schools will then proceed in the same way as for the other schools within the same state.
Memorandum: 2015 - 1.1A/1.1B/1.1G/1.1D/1.1E February 26, 2014

Page 21 of 23

VII. Student Sampling 

Students within the sampled schools will be selected with equal probability. The student sampling 

parameters vary by sample type (Alpha, Beta, Gamma, Delta, and Epsilon) and grade, as described 

below.

Alpha Sample, Grades 4 and 8 Schools (Except Puerto Rico) 

A. The sample size for each school will depend upon which spiral the school was assigned, either 

RS, RM or NL.

B. In schools assigned the RS spiral, all students will be selected, up to 100. If the school has 

more than 100 students, 90 will be selected. In some schools, the school may be assigned 

more than one ‘hit’ in sampling. In these schools we will select a sample of size 90 times the 

number of hits, taking all students if this target is greater than or equal to 90/100 of the total

enrollment. 

C. In schools assigned the RM spiral, all students, up to 70, will be selected. If the school has 

more than 70 students, 63 will be selected. In some schools, the school may be assigned more 

than one ‘hit’ in sampling. In these schools we will select a sample of size 63 times the number 

of hits, taking all students if this target is greater than or equal to 63/70 of the total 

enrollment.

D. In schools assigned the NL spiral, all students will be selected, up to 100. If the school has 

more than 100 students, 90 will be selected. 

E. The student sampling procedures for DC at grades 4 and 8 have yet to be determined. 

 

Alpha Sample, Puerto Rico Grades 4 and 8 

A. All students, up to 55, will be selected.

B. If the school has more than 55 students, a systematic sample of 50 students will be selected 

with no oversampling.

C. All students will be assigned to assessment type PR.

 

 

 

 
Memorandum: 2015 - 1.1A/1.1B/1.1G/1.1D/1.1E February 26, 2014

Page 22 of 23

Delta Samples, Grades 4 and 8 

 

A. All students, up to 115, will be selected.

B. If the school has more than 115 students, a systematic sample of 105 students will be selected.

C. All students will be assigned to assessment type NL. 

Beta and Epsilon Samples, Grade 8 

A. In each school, a sample for computer-based testing will be selected as follows: All students 

up to 50 will be selected. If there are more than 50 students enrolled, a sample of 50 students 

will be selected.

B. All students will be assigned to the TB session.

Gamma and Delta Samples, Grade 12 

A. All students, up to 105, will be selected.

B. If the school has more than 105 students, a systematic sample of 95 students will be selected.

C. All students will be assigned to assessment type NL. 

VIII. Weighting Requirements 

Weighting activities for the 2015 NAEP assessments are not covered under the current NAEP 

contracts. However, based on past experience, below is a description of the likely weighting 

requirements.

The Operational Samples 

These samples will have a single set of weights for each subject (reading, math, science, and meaning 

vocabulary at grades 4, 8, and 12) applied to reflect probabilities of selection, school and student 

nonresponse, any trimming, and the random assignment to the particular subject. There will be a 

separate replication schemes by grade and public/private. Such weights will also be derived for the 

Puerto Rico KaSA assessment and the national KaSA special studies at grades 4 and 8.
Memorandum: 2015 - 1.1A/1.1B/1.1G/1.1D/1.1E February 26, 2014

Page 23 of 23

The Pilot Test Samples 

Fully adjusted weights will not be provided for the students in the TBA start-up and science ICT 

and HOT pilot test studies. However, preliminary weights will be available for these samples. These 

will reflect the school and student selection probabilities, but with no adjustments for nonresponse.

The NIES Samples 

The NIES survey samples consist of two grade-specific samples, comprising students selected for 

each of the grade 4 and 8 operational samples. We will create one set of weights for each gradespecific sample. The NIES weights are designed for any aggregation of the NIES data, not involving 

NAEP achievement data. NIES analyses involving NAEP achievement data should use the 

appropriate NAEP operational weights.