HIGHLIGHTS

Mathematics 2003 and 2005

Performance in Puerto Rico

National Assessment of Educational Progress

U.S. Department of Education

NCES 2007-459
For over three decades, NAEP assessments have

been conducted periodically in reading, mathematics, science, writing, history, geography, and other

subjects. By making objective information available

on student performance at the national, state, and

local levels, NAEP is an integral part of our nation’s

evaluation of the condition and progress of education. Only information related to academic achievement and relevant variables is collected. The privacy

of individual students is protected, and the identities

of participating schools are not released. NAEP is

a congressionally mandated project of the National

Center for Education Statistics (NCES) within

the Institute of Education Sciences of the U.S.

Department of Education. The Commissioner of

Education Statistics is responsible for carrying

out the NAEP project. The National Assessment

Governing Board oversees and sets policy for NAEP.

The Nation's Report CardTM informs the public about the academic achievement of elementary and secondary

students in the United States and its jurisdictions, including Puerto Rico. Report cards communicate the

findings of the National Assessment of Educational Progress (NAEP), a continuing and nationally representative

measure of achievement in various subjects over time. The Nation's Report CardTM compares performance

among states, urban districts, public and private schools, and student demographic groups.

What is the Nation's Report CardTM?

Contents

M A R C H 2 0 0 7

1 Executive Summary

2 Understanding the Results

4 2003 Results

9 2005 Results

14 Technical Notes
THE NATION’S REPORT CARD ● 1

2005 Findings

• Overall, fourth- and eighth-grade students in

Puerto Rico scored lower, on average, than public

school students in the nation.

Figure B.

Average NAEP mathematics scores for public school students

in Puerto Rico and the nation in 2005

2003 Findings

• On average, fourth- and eighth-grade students in

Puerto Rico scored lower than public school

students in the nation.

Figure A.

Average NAEP mathematics scores for public school students

in Puerto Rico and the nation in 2003

About this report

Throughout this report, results for Puerto Rico are compared to results for public school students in the nation because in Puerto Rico only

public school students participated in the 2003 and 2005 NAEP mathematics assessments. The national sample does not include Puerto Rico

at this time, although the intent is to include Puerto Rico as part of the national sample in future NAEP administrations.

Executive Summary

The NAEP mathematics assessment was administered to public school students in Puerto Rico for the first

time in 2003. Although NAEP had previously administered some of the assessment in Spanish to students who

required accommodations, this was the first time an entire NAEP administration was in a language other than

English. The NAEP mathematics assessment was administered again to public school students in both fourthand eighth-grades in Puerto Rico in 2005. This report presents the results of the NAEP mathematics assessment

for Puerto Rico for 2003 and 2005. Because of modifications to the 2005 Puerto Rico administration, results

from 2003 should not be compared to results from 2005. Although parallel changes were not made in the

nation in 2005, within year comparisons between Puerto Rico and the nation are valid.

* Significantly different (p < .05) from students in Puerto Rico.

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center

for Education Statistics, National Assessment of Educational Progress (NAEP), 2003

Mathematics Assessment.

* Significantly different (p < .05) from students in Puerto Rico.

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center

for Education Statistics, National Assessment of Educational Progress (NAEP), 2005

Mathematics Assessment.

■ Puerto Rico ■ Nation ■ Puerto Rico ■ Nation

• At fourth-grade, 9 percent of students in Puerto

Rico and 76 percent of students in the nation

scored at or above Basic. At eighth-grade, 4 percent of students in Puerto Rico and 67 percent of

students in the nation scored at or above Basic.

• Fourth-grade female students in Puerto Rico

scored significantly higher than male students in

the geometry and spatial sense content area.

• Twelve percent of students in Puerto Rico and

79 percent of students in the nation scored at or

above Basic in grade 4. Six percent of students

in Puerto Rico and 68 percent of students in the

nation scored at or above Basic in grade 8.

• Eighth-grade female students in Puerto Rico

scored significantly higher than male students in

the data analysis and probability content area.
2

Understanding the Results

The Mathematics Framework

The content of the mathematics assessment is based

on a framework which describes in detail how mathematics should be assessed by NAEP. The framework

was developed by the National Assessment Governing

Board in a comprehensive process involving a broad

spectrum of interested parties, including teachers,

curriculum specialists, subject-matter specialists,

school administrators, parents, and members of the

general public. The NAEP mathematics framework

specifies the content to be assessed at each grade level

and the percentage of questions to be assessed in each

of five content areas (exhibit 1). The frameworks are

available at http://www.nagb.org/pubs/pubs.html.

Revisions are periodically made to the framework to

reflect changing curricular emphases and objectives.

Whenever possible, NAEP maintains a connection

to previous frameworks when revisions are made. In

2005, names of some of the content areas changed but

the percentage of questions to be assessed in each of

five content areas remained the same for grade 4. At

grade 8, there was a 5 percent decreased emphasis on

number properties and operations and a corresponding

5 percent increased emphasis on algebra compared

to 2003.

In addition to specifying content, the 2005 framework

called for an assessment that measures different levels

of mathematical complexity to assure that NAEP

assesses a variety of ways of knowing and doing

mathematics. The level of complexity of a question is

determined by the cognitive demands that it places on

students. For example, a question with a high level of

complexity at grade 4 might ask students to explain

and justify their solutions to a problem.

Administering the NAEP Mathematics

Assessment in Puerto Rico

Title I of the Elementary and Secondary Education Act

of 1965, as amended, requires all jurisdictions receiving

Title I funds to participate in the National Assessment

of Educational Progress in fourth- and eighth-grade

reading and mathematics every other year beginning

in 2003. The U.S. Department of Education decided

that Puerto Rico should not participate in the NAEP

reading assessment, because that assessment measures

a student’s ability to read in English, and Spanish is

the language of instruction in Puerto Rico.

The NAEP mathematics assessment was translated

into Spanish to allow Puerto Rico to participate on a

trial basis. First in 2003 and then again in 2005, the

NAEP mathematics assessment was administered to

fourth- and eighth-grade public school students in

Puerto Rico. The content of both the 2003 and 2005

NAEP assessments was the same as the content

administered to all other jurisdictions for those years.

In future NAEP administrations, the intent is to include

Puerto Rico as part of the national sample. The Puerto

Rico results for 2003 and 2005 were not combined

with those of other states or jurisdictions because of

concerns that translated items used in Puerto Rico were

not functioning like the English-only items used in the

nation. Further analysis of the 2003 and 2005 assessments concluded that the Puerto Rico results were

accurate in terms of the NAEP scale, and comparisons

between Puerto Rico and the nation could be made.

However, the Puerto Rico 2003 results cannot be compared to those for 2005 because changes were made to

the 2005 assessment to improve the translation. Details

are provided in the forthcoming Technical Report of

the NAEP Mathematics Assessment in Puerto Rico:

Focus on Statistical Issues. This report will be available

at http://nationsreportcard.gov/puertorico_2005/.

GRADE 8

Exhibit 1.

Distribution of NAEP mathematics items, by grade and

content area in 2005

GRADE 4

__
The Governing Board established its achievement

levels in 1990 based upon the NAEP mathematics

content framework and a standard-setting process.

A cross section of educators and interested citizens

from across the nation were asked to judge what

students should know and be able to do relative to

the content set out in the NAEP mathematics framework. As provided by law, NCES, upon review of

congressionally mandated evaluations of NAEP,

has determined that achievement levels are to be

used on a trial basis and should be interpreted with

caution. NAEP achievement levels have been widely

used by national and state officials.

Interpreting Results

NAEP uses widely accepted statistical standards in

analyzing sample data. In the tables and charts of this

report, the symbol (*) is used to indicate scores or

percentages that have been determined to be statistically significant from one another at the .05 level, with

appropriate adjustments for multiple comparisons.

Differences between scale scores or percentages are

calculated using unrounded numbers. In some

instances, the result of the subtraction differs from

what would be obtained by subtracting the rounded

values shown in the accompanying figure or table.

Exhibit 2.

NAEP achievement levels

The three NAEP achievement levels from lowest to highest are Basic,

Proficient, and Advanced.

Basic: This level denotes partial mastery of prerequisite knowledge

and skills that are fundamental for proficient work at each grade.

Proficient: This level represents solid academic performance for each

grade assessed. Students reaching this level have demonstrated competency over challenging subject matter, including subject-matter

knowledge, application of such knowledge to real-world situations,

and analytical skills appropriate to the subject matter.

Advanced: This level signifies superior performance.

Cut scores

Cut scores represent the minimum score required for performance

at each NAEP achievement level. The mathematics cut scores on the

0–500 NAEP scale that define the lower boundaries of each of the

achievement levels are:

Grade 4 Grade 8

Basic 214 262

Proficient 249 299

Advanced 282 333

Reporting the Results

This report is one of a series of three on the administration and results of the 2003 and 2005 NAEP

mathematics assessments in Puerto Rico. The forthcoming Technical Report of the NAEP Mathematics

Assessment in Puerto Rico: Focus on Statistical Issues

provides details on the technical quality of the NAEP

assessments administered in Puerto Rico. A second

report, Mathematics 2005 Performance in Puerto

Rico: Focus on the Content Areas, provides results

by content area and includes a discussion of student

performance on a sample of items. This, the third

report, presents results for Puerto Rico and the

nation in two ways: as average scores on the NAEP

mathematics scale and as percentages of students

attaining different NAEP mathematics achievement

levels. Published reports are available at

http://nationsreportcard.gov/puertorico_2005/.

Scale Scores. NAEP mathematics results are reported

for both grades on the same 0–500 scale. The NAEP

mathematics assessment is a composite combining

separate scales for each of the five mathematics

content areas. Results for each content area are also

reported on a 0–500 scale. Average scale scores are

computed for groups, not for individual students. The

average scores are based on analyses of the percentages of students who answered each item successfully;

NAEP does not produce individual student scores.

While the score ranges at each grade in mathematics

are identical, the scales were derived independently at

each grade. Therefore, average scale scores across

grades cannot be compared. For example, equal scale

scores on the grade 4 and grade 8 scales do not imply

equal levels of mathematics achievement. In this

report, average scores are presented for public school

students in Puerto Rico and the nation. Scale scores

of students at selected percentiles are used to illustrate

the relative performance of students in both Puerto

Rico and the nation.

Achievement Levels. In addition to scale scores, results

are presented in terms of mathematics achievement

levels as adopted by the Governing Board. Achievement

levels are intended to measure how well students'

actual achievement matches the achievement expected

of them. At each grade level, the achievement-level

cut scores are placed on the mathematics scales and

percentages of students performing at or above three

achievement levels are reported (exhibit 2). The

three achievement levels are: Basic, Proficient, and

Advanced. Percentages below Basic are also reported.

THE NATION’S REPORT CARD ● 3

Understanding the Results

__
4

Overall performance differences between

Puerto Rico and the nation

Overall performance on the NAEP mathematics

assessment is summarized on a 0–500 scale. Figure

1 presents the average scale scores of students from

Puerto Rico, students from low-income families in

the nation, and all students from the nation.

On average, in 2003, students at grade 4 in Puerto

Rico scored lower on the mathematics assessment

than students in the nation. The average scale score

for students in Puerto Rico was 179, compared

with 234 for the nation. Puerto Rican students’

scores were 42 points lower than those of students

from low-income families and 55 points lower than

students in the nation.

2003 Results

NAEP collects data on students’ eligibility for free or reduced-priced school lunch as an indicator of family economic

status. Eligibility for free or reduced-price lunch is determined by a student’s family income in relation to the federally

established poverty level. See http://www.fns.usda. gov/cnd/lunch/ for more information. For reporting purposes, the term

low-income is used when referring to students who are eligible for free or reduced-price lunch under the National School

Lunch Program. One-hundred percent of the students at grades 4 and 8 in Puerto Rico were eligible for free or reducedprice lunch in 2003. This percentage differed significantly from the nation, where 44 percent of fourth-graders and 36 percent of eighth-graders were from low-income families, as determined by their eligibility for free or reduced-price lunch.

Figure 1.

Average fourth-grade NAEP mathematics scores for public

school students in Puerto Rico and the nation in 2003

* Significantly different (p < .05) from students in Puerto Rico.

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center

for Education Statistics, National Assessment of Educational Progress (NAEP), 2003

Mathematics Assessment.

A note on family income
To get a better picture of the Puerto Rico NAEP

results, scores on the mathematics assessment for

students scoring at selected percentiles are presented

in figure 2. A percentile indicates the percentage of

students whose scores fall at or below a particular

score. Examining the performance of students at

selected percentiles can indicate whether lower-,

middle-, or higher-scoring students diverge from

the picture for students overall. The 10th and 25th

percentiles represent lower-scoring students, the

50th percentile represents middle-scoring students,

and the 75th and 90th percentiles represent higherscoring students.

Figure 3.

Percentage of fourth-grade public school students in Puerto

Rico and the nation, by NAEP mathematics achievement

level in 2003

* Significantly different (p < .05) from students in Puerto Rico.

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center

for Education Statistics, National Assessment of Educational Progress (NAEP), 2003

Mathematics Assessment.

# Rounds to zero for proficient and advanced levels.

* Significantly different (p < .05) from students in Puerto Rico.

NOTE: Detail may not sum to totals because of rounding.

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center

for Education Statistics, National Assessment of Educational Progress (NAEP), 2003

Mathematics Assessment.

4th Grade 2003

THE NATION’S REPORT CARD ● 5

For the 2003 NAEP mathematics assessment, 9 percent of fourth-graders in Puerto Rico performed at or

above Basic (figure 3). In the nation, 76 percent of

fourth-graders performed at or above Basic.

Figure 2.

Fourth-grade NAEP mathematics scores at selected

percentiles for public school students in Puerto Rico and

the nation in 2003

◆ Puerto Rico

■ Nation

In 2003, similar differences between fourth-graders

in Puerto Rico and the nation were found at various

percentiles spanning the entire distribution (figure 2).

Scale scores were consistently higher for fourthgraders in the nation than for those in Puerto Rico.

For example, fourth-graders at the 90th percentile

in Puerto Rico had a score of 212 compared to a

score of 270 in the nation. In the nation, 75 percent

of students had scores above 215 (i.e., the score for

the nation at the 25th percentile).
6

4th Grade 2003

Gender differences in Puerto Rico in

geometry and spatial sense

Puerto Rican male and female students’ scores on

the mathematics assessment overall were not significantly different (table 1). However, there was a significant difference between male and female students’

scores in the geometry and spatial sense content

area. Female students scored 5 points higher than

male students. In the nation, male students scored

higher than female students in mathematics overall

and in each content area except geometry and spatial

sense, where there was no significant difference.

Table 1.

Average fourth-grade NAEP mathematics scores for public

school students in Puerto Rico and the nation in 2003,

by gender

Puerto Rico Nation

Content area Male Female Male Female

Overall 179 179 235* 233

Number sense, properties,

and operations 171 170 233* 230

Measurement 186 184 236* 231

Geometry and spatial sense 192* 198 233 233

Data analysis, statistics,

and probability 176 180 237* 236

Algebra and functions 181 179 241* 238

* Significantly different (p < .05) from female students.

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center

for Education Statistics, National Assessment of Educational Progress (NAEP), 2003

Mathematics Assessment.

Content area differences between Puerto

Rico and the nation

On average, fourth-grade students in Puerto Rico

scored lower than national public school students

in each mathematics content area. At grade 4, differences between the performance of Puerto Rico

and the nation were between 58 and 61 points for

three of the five mathematics content areas. For

the content areas of geometry and spatial sense,

and measurement, the differences were 38 and

48 points, respectively (figure 4).

Figure 4.

Average fourth-grade NAEP mathematics scores for public

school students in Puerto Rico and the nation in 2003, by

content area

* Significantly different (p < .05) from students in Puerto Rico.

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center

for Education Statistics, National Assessment of Educational Progress (NAEP), 2003

Mathematics Assessment.

■ Puerto Rico ■ Nation

Geometry and spatial sense

The geometry and spatial sense content area focuses

on the calculation of lengths, areas, and volumes of

common shapes, as well as the concepts of symmetry

and transformation.

Students in grade 4 are expected to be familiar with

simple plane figures such as lines, circles, triangles,

and rectangles, as well as solid figures such as

cubes, spheres, and cylinders. They are also expected

to be able to recognize examples of parallel and

perpendicular lines.

Grade 4 sample question
Eighth-graders at the 90th percentile in Puerto Rico

had a score of 251 (below Basic) compared to a score

of 321 (Proficient) for their peers in the nation (figure 6).

Overall performance differences

between Puerto Rico and the nation

The average scale score for eighth-grade

students in Puerto Rico was 212, compared

with 276 for the nation. As shown in figure

5, eighth-graders in Puerto Rico scored

46 points lower than students from lowincome families in the nation and 64 points

lower than students in the nation.

Figure 7.

Percentage of eighth-grade public school students in Puerto

Rico and the nation, by NAEP mathematics achievement level

in 2003

# Rounds to zero for proficient and advanced levels.

* Significantly different (p < .05) from students in Puerto Rico.

NOTE: Detail may not sum to totals because of rounding.

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center

for Education Statistics, National Assessment of Educational Progress (NAEP), 2003

Mathematics Assessment.

THE NATION’S REPORT CARD ● 7

* Significantly different (p < .05) from students in Puerto Rico.

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center

for Education Statistics, National Assessment of Educational Progress (NAEP), 2003

Mathematics Assessment.

◆ Puerto Rico

■ Nation

Figure 6.

Eighth-grade NAEP mathematics scores at selected percentiles

for public school students in Puerto Rico and the nation in 2003

* Significantly different (p < .05) from students in Puerto Rico.

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center

for Education Statistics, National Assessment of Educational Progress (NAEP), 2003

Mathematics Assessment.

Figure 5.

Average eighth-grade NAEP mathematics scores for public

school students in Puerto Rico and the nation in 2003

2003 Results

The results from the 2003 NAEP Puerto Rico assessment show that 4 percent of eighth-graders performed

at or above Basic in 2003 (figure 7). The percentage

of eighth-graders in the nation who performed at or

above Basic was substantially higher (67 percent).
Average scores in Puerto Rico below Basic

in all content areas

Students from Puerto Rico scored lower than students from the nation in all five NAEP mathematics

content areas. The difference between students in

Puerto Rico and the nation was between 51 and

67 points for all content areas except measurement,

where the difference was 80 points (figure 8).

No gender differences in grade 8 in

Puerto Rico

Average scores for male and female students in

grade 8 in Puerto Rico and the nation are shown

in table 2. There were no significant differences

between Puerto Rican male and female students’

scores overall or in any of the five mathematics

content areas. In the nation, male students scored

higher than female students in mathematics overall

and in each content area except geometry and

spatial sense, and algebra and functions, where

there were no significant differences.

8

Table 2.

Average eighth-grade NAEP mathematics scores for public

school students in Puerto Rico and the nation in 2003,

by gender

Figure 8.

Average eighth-grade NAEP mathematics scores for public

school students in Puerto Rico and the nation in 2003,

by content area

* Significantly different (p < .05) from students in Puerto Rico.

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center

for Education Statistics, National Assessment of Educational Progress (NAEP), 2003

Mathematics Assessment.

■ Puerto Rico ■ Nation

8th Grade 2003

The measurement content area focuses on the use

of numbers and measures to describe and compare

mathematical and real-world objects.

At grade 8, the emphasis is on the use of square units

for measuring areas and surface area, cubic units

for measuring volume, degrees for measuring angles,

and constructed units such as miles per hour.

Measurement Grade 8 sample question

6

6

4 4

Puerto Rico Nation

Content area Male Female Male Female

Overall 212 213 277* 275

Number sense, properties,

and operations 209 214 277* 274

Measurement 197 191 276* 271

Geometry and spatial sense 222 224 274 273

Data analysis, statistics,

and probability 210 213 280* 278

Algebra and functions 216 217 279 278

* Significantly different (p < .05) from female students.

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center

for Education Statistics, National Assessment of Educational Progress (NAEP), 2003

Mathematics Assessment.
THE NATION’S REPORT CARD ● 9

2005 Results

Overall performance differences between

Puerto Rico and the nation

On average, fourth-graders in Puerto Rico scored

lower than students in the nation on the NAEP

mathematics assessment. The average scale score

for fourth-grade students in Puerto Rico was 183

compared to 237 in the nation. On average, Puerto

Rican students scored 42 points lower than students

from low-income families in the nation and 54

points lower than students in the nation.

Figure 9.

Average fourth-grade NAEP mathematics scores for public

school students in Puerto Rico and the nation in 2005

* Significantly different (p < .05) from students in Puerto Rico.

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center

for Education Statistics, National Assessment of Educational Progress (NAEP), 2005

Mathematics Assessment.

In 2003, the percentage of fourth-graders identified as students with disabilities in Puerto Rico did not differ from

the nation. At eighth-grade, 8 percent of students in Puerto Rico were identified as students with disabilities.

This percentage differed from the nation, where 11 percent of students were identified as students with disabilities.

In 2005, the percentage of fourth-graders identified as students with disabilities in Puerto Rico differed from the

nation. In Puerto Rico, 15 percent were identified as students with disabilities, compared to 12 percent in the nation.

The percentage of eighth-graders identified as students with disabilities in Puerto Rico did not differ from the nation.

A note on students with disabilities
10

Figure 10.

Fourth-grade NAEP mathematics scores at selected

percentiles for public school students in Puerto Rico and

the nation in 2005

* Significantly different (p < .05) from students in Puerto Rico.

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center

for Education Statistics, National Assessment of Educational Progress (NAEP), 2005

Mathematics Assessment.

◆ Puerto Rico

■ Nation

Figure 11.

Percentage of fourth-grade public school students in Puerto

Rico and the nation, by NAEP mathematics achievement

level in 2005

# Rounds to zero for proficient and advanced levels.

* Significantly different (p < .05) from students in Puerto Rico.

NOTE: Detail may not sum to totals because of rounding.

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center

for Education Statistics, National Assessment of Educational Progress (NAEP), 2005

Mathematics Assessment.

Lower-, middle-, and higher-performing students

in Puerto Rico scored lower than students in the

nation at corresponding percentiles (figure 10).

Fourth-graders at the 90th percentile in Puerto Rico

had a score of 216 compared to a score of 272 for

students at the 90th percentile in the nation. Lower

performing students (those at the 25th percentile)

had scores of 166 and 219 in Puerto Rico and the

nation, respectively.

4th Grade 2005

As shown in figure 11, the results from the 2005

NAEP Puerto Rico assessment indicate that

12 percent of fourth-graders performed at or above

Basic in mathematics. The percentage of fourthgraders in the nation who performed at or above

Basic was significantly higher (79 percent).

More information on the 2003 and

2005 NAEP mathematics assessments

in Puerto Rico

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Details on the technical quality of the NAEP

assessments are presented in the forthcoming

Technical Report of the NAEP Mathematics

Assessment in Puerto Rico: Focus on

Statistical Issues.

Details of student performance by content area

can be found in Mathematics 2005 Performance

in Puerto Rico: Focus on the Content Areas

available at

http://nationsreportcard.gov/puertorico_2005/.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

__
The number properties and operations content area

focuses on ways to represent, calculate, and estimate

numbers.

At grade 4, students are expected to have an understanding of numbers, operations, estimation, and

applications to real-world situations.

Score differences between students in Puerto

Rico and the nation in all content areas

On average, fourth-grade students in Puerto Rico

scored lower than students in the nation overall

and in each mathematics content area (figure 12).

Differences between the performance of students in

Puerto Rico and the nation were between 52 and

60 points for all content areas except geometry,

where the difference was 39 points.

Gender differences in grade 4 in Puerto

Rico in geometry

On average, Puerto Rican male and female students’

scores were not significantly different on the overall

mathematics assessment (table 3). However, there

was a significant difference in geometry scores,

where female students scored 4 points higher than

male students. In the nation, male students scored

higher than female students overall and in each

content area except geometry, where there was no

significant difference. Figure 12.

Average fourth-grade NAEP mathematics scores for public

school students in Puerto Rico and the nation in 2005,

by content area

* Significantly different (p < .05) from students in Puerto Rico.

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center

for Education Statistics, National Assessment of Educational Progress (NAEP), 2005

Mathematics Assessment.

4th Grade 2005

Table 3.

Average fourth-grade NAEP mathematics scores for public

school students in Puerto Rico and the nation in 2005,

by gender

■ Puerto Rico ■ Nation

THE NATION’S REPORT CARD ● 11

Puerto Rico Nation

Content area Male Female Male Female

Overall 184 183 238* 236

Number properties and

operations 178 173 237* 234

Measurement 181 182 238* 234

Geometry 195* 199 235 236

Data analysis and

probability 188 188 241* 240

Algebra 190 192 244* 242

* Significantly different (p < .05) from female students

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center

for Education Statistics, National Assessment of Educational Progress (NAEP), 2005

Mathematics Assessment.

Number properties and operations Grade 4 sample question
12

As shown in figure 15, the results from the 2005 NAEP

Puerto Rico assessment indicate that 6 percent of

eighth-graders performed at or above Basic. The majority

of students in Puerto Rico scored below Basic (94 percent). In comparison, 32 percent of students in the

nation scored below Basic.

* Significantly different (p < .05) from students in Puerto Rico.

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center

for Education Statistics, National Assessment of Educational Progress (NAEP), 2005

Mathematics Assessment.

◆ Puerto Rico

■ Nation

Figure 15.

Percentage of eighth-grade public school students in Puerto

Rico and the nation, by NAEP mathematics achievement

level in 2005

# Rounds to zero for proficient and advanced levels.

* Significantly different (p < .05) from students in Puerto Rico.

NOTE: Detail may not sum to totals because of rounding.

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center

for Education Statistics, National Assessment of Educational Progress (NAEP), 2005

Mathematics Assessment.

2005 Results

Average scores below Basic in grade 8

in Puerto Rico

On average, eighth-grade students in Puerto

Rico scored lower on the NAEP mathematics

assessment than eighth-grade students in the

nation. The average scale score was 218 and 278

for students in Puerto Rico and the nation, respectively. As shown in figure 13, students in Puerto

Rico scored 43 points lower than students from

low-income families in the nation and 59 points

lower than students in the nation.

Figure 13.

Average eighth-grade NAEP mathematics scores for public

school students in Puerto Rico and the nation in 2005

* Significantly different (p < .05) from students in Puerto Rico.

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center

for Education Statistics, National Assessment of Educational Progress (NAEP), 2005

Mathematics Assessment.

Eighth-graders at the 90th percentile in Puerto Rico

had a score of 255 (below Basic) compared to a

score of 323 (Proficient) for their peers in the nation

(figure 14).

Figure 14.

Eighth-grade NAEP mathematics scores at selected percentiles

for public school students in Puerto Rico and the nation in 2005
Gender differences in Puerto Rico in data

analysis and probability area

Average scores for male and female eighth-graders

in Puerto Rico and the nation are shown in table 4.

In the nation, male students scored higher than

female students in mathematics overall and in the

number properties and operations, and measurement content areas. On average, Puerto Rican male

and female students’ scores were not significantly

different on the assessment overall and for four of

the five content areas. However, female students

scored 8 points higher than male students in the

content area of data analysis and probability.

Content area differences in grade 8 between

Puerto Rico and the nation

On average, eighth-grade students in Puerto Rico

scored lower than students in the nation in each of

the five mathematics content areas. Differences

between the performance of Puerto Rico and the

nation were between 49 and 59 points for each content area except measurement, where the difference

was 76 points (figure 16).

THE NATION’S REPORT CARD ● 13

Table 4.

Average eighth-grade NAEP mathematics scores for public

school students in Puerto Rico and the nation in 2005,

by gender

8th Grade 2005

Figure 16.

Average eighth-grade NAEP mathematics scores for public

school students in Puerto Rico and the nation in 2005,

by content area

■ Puerto Rico ■ Nation

* Significantly different (p < .05) from students in Puerto Rico.

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center

for Education Statistics, National Assessment of Educational Progress (NAEP), 2005

Mathematics Assessment.

Puerto Rico Nation

Content area Male Female Male Female

Overall 217 220 278* 277

Number properties and

operations 217 218 278* 274

Measurement 200 196 277* 272

Geometry 224 227 275 275

Data analysis and

probability 220* 228 280 280

Algebra 220 224 280 281

* Significantly different (p < .05) from female students.

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center

for Education Statistics, National Assessment of Educational Progress (NAEP), 2005

Mathematics Assessment.

The data analysis and probability content area focuses

on students’ skills in four areas: data representation,

characteristics of data sets, experiments and samples,

and probability.

At grade 8, students are expected to have knowledge

of experiments and samples, recognize possible

sources of bias in sampling, and identify random versus

nonrandom sampling.

Data analysis and probability Grade 8 sample question
14

Puerto Rico Nation

Number of Weighted Number of Weighted

Grade and year students percent students percent

Grade 4: 2003 3,000 94 184,300 94

Grade 8: 2003 2,800 92 147,600 91

Grade 4: 2005 2,800 95 163,000 94

Grade 8: 2005 2,800 93 152,800 91

NOTE: The numbers of students are rounded to the nearest hundred.

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center

for Education Statistics, National Assessment of Educational Progress (NAEP), 2003 and

2005 Mathematics Assessments.

Table 7.

Number and percentage of public school students

participating in NAEP mathematics assessment in Puerto

Rico and the nation, by grade and year

NAEP sampling procedures in Puerto Rico

The schools and students participating in the NAEP

assessment are chosen to be nationally representative. Sampling was conducted in two stages. In the

first stage, schools were selected from stratified

frames within each jurisdiction. In the second stage,

students were selected from within schools. Sampling procedures in Puerto Rico did not differ from

the procedures in other jurisdictions because the

intent is to include Puerto Rico as part of the national

sample in future NAEP administrations.

For the trial NAEP administrations in 2003 and 2005,

approximately 100 schools and 3,000 students per

grade were sampled in public schools in Puerto Rico.

Private schools did not participate in the trial NAEP

administrations in Puerto Rico. Table 5 presents the

sample sizes and target populations for the 2003

and 2005 fourth- and eighth-grade mathematics

assessment for public school students in Puerto Rico

and the nation. Information is only presented for

public school students in the nation, although private

school students did participate.

School and student participation rates

To reduce the possibility of biased estimates, NCES

and the Governing Board established participation

rate standards that all jurisdictions, including Puerto

Rico, are required to meet. Results are reported for

jurisdictions that have weighted participation rates

of at least 85 percent. In the 2003 and 2005 Puerto

Rico assessments, this participation rate standard

was met at both grades 4 and 8. Table 6 provides

school participation rates and Table 7 provides

student participation rates.

Technical Notes

Puerto Rico Nation

Student Target Student Target

Grade and year sample size population sample size population

Grade 4: 2003 3,000 48,000 191,400 3,603,000

Grade 8: 2003 2,800 45,000 153,500 3,575,000

Grade 4: 2005 2,800 42,000 168,900 3,745,000

Grade 8: 2005 2,800 40,000 159,200 3,662,000

NOTE: Student sample sizes are rounded to the nearest hundred, and target populations are

rounded to the nearest thousand.

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center

for Education Statistics, National Assessment of Educational Progress (NAEP), 2003 and

2005 Mathematics Assessments.

Table 5.

Student sample size and target populations for NAEP

mathematics assessment for public school students in

Puerto Rico and the nation, by grade and year

Puerto Rico Nation

Number of Weighted Number of Weighted

Grade and year schools percent schools percent

Grade 4: 2003 110 100 6,910 99.8

Grade 8: 2003 100 100 5,530 99.6

Grade 4: 2005 110 100 8,700 99.6

Grade 8: 2005 110 100 6,460 99.5

NOTE: The numbers of schools are rounded to the nearest tenth.

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center

for Education Statistics, National Assessment of Educational Progress (NAEP), 2003 and

2005 Mathematics Assessments.

Table 6.

Number and percentage of public schools participating in

NAEP mathematics assessment in Puerto Rico and the

nation, by grade and year
Interpreting statistical significance

Comparisons between groups are based on statistical tests that consider both the size of the differences and the standard errors of the two statistics

being compared. Standard errors are margins of

error, and estimates based on smaller samples are

likely to have larger margins of error than estimates

based on larger samples. The size of the standard

errors may also be influenced by other factors,

such as how representative the students are of the

population as a whole.

When an estimate, such as an average score, has a

large standard error, a difference between that score

and another that seems large may not be statistically

significant. Differences of the same magnitude may

or may not be statistically significant, depending

on the size of the standard error of the statistics.

For example, a 1-point difference between male

and female students may be statistically significant

in the nation, while a 3-point difference between

these same groups may not be significant in Puerto

Rico because of the smaller sample size on which

the estimate is based. Standard errors for the NAEP

scores and percentages presented in this report are

available at http://nces.ed.gov/nationsreportcard/nde.

Any difference between scores or percentages that

is identified as higher, lower, larger, or smaller in

this report has been determined to be statistically

significant at the .05 level with appropriate adjustments for multiple comparisons.

Accommodations

In 2003, the NAEP mathematics assessment began

permitting the use of accommodations, such as extra

testing time or individual rather than group administration, for students who required them. In the

nation, accommodations were provided to students

with disabilities and to English language learners.

However, in Puerto Rico, the NAEP mathematics

assessment was translated into Spanish because

Spanish is the language of instruction in Puerto Rico.

Therefore, accommodations were only provided to

students with disabilities. Table 8 presents the

number and percentage of students with disabilities

who received accommodations at grades 4 and 8 in

Puerto Rico and the nation in 2003 and 2005.

THE NATION’S REPORT CARD ● 15

Technical Notes

Puerto Rico Nation

Number of Percent of Number of Percent of

students students students students

accommodated accommodated accommodated accommodated

(unweighted) (weighted) (unweighted) (weighted)

Grade 4: 2003 100 4 13,800 7

Grade 8: 2003 <50 2 9,800 6

Grade 4: 2005 400 14 13,300 8

Grade 8: 2005 200 8 10,900 7

NOTE: The numbers of students are rounded to the nearest hundred.

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center

for Education Statistics, National Assessment of Educational Progress (NAEP), 2003 and

2005 Mathematics Assessments.

Table 8.

Number and percentage of participating public school students

with disabilities who received accommodations for NAEP

mathematics assessment in Puerto Rico and the nation, by

grade and year

Grade and year
16

functioned as expected in the nation, the translated

items did not function as expected in Puerto Rico.

Changes made to the 2005 Puerto Rico

administration

In light of initial concerns about the quality of the

Puerto Rico data, important enhancements were

made to the translation procedures for the 2005

assessment. Key among these changes were: (1) the

addition of two Puerto Rican mathematics teachers

to the expert review panel and, (2) adaptation of

topics and language usage to incorporate the unique

linguistic and cultural characteristics of Puerto Rico.

In addition, a number of changes were implemented

to improve various components of the assessment

process. First, the administration script in Puerto

Rico was revised to give students explicit directions

to move on to the next question when they did not

know the answer. The revised script also included

extensive explanations of the different item types in

the assessment. These changes were made because

students in Puerto Rico may not have been familiar

with the NAEP assessment format.

Second, in an attempt to reduce the number of items

with missing responses, students were provided

with an additional 10 minutes to complete each of

the two timed mathematics sections. Thus, students

in Puerto Rico were given a total of 70 minutes to

complete the two mathematics sections in 2005,

compared to 50 minutes in 2003. Fifty minutes is the

standard time given to students in other jurisdictions.

Additional analysis of the 2003 and 2005 data

indicated that the Puerto Rico results were accurate

in terms of the NAEP scale and thus comparisons

could be made between Puerto Rico and the nation.

Because of translation changes made to the 2005

assessment, Puerto Rico results from 2003 cannot

be compared with those of 2005.

Quality of the Puerto Rico 2003

assessment results

This was the first attempt to conduct an entire NAEP

administration in a language other than English.

Preliminary analysis of data for the 2003 mathematics assessment showed three important differences between Puerto Rico and other jurisdictions:

High Percentage of Missing Data. Of those who

participated, there were more missing responses in

Puerto Rico than in the nation overall. Preliminary

analyses of the Puerto Rico 2003 data identified two

types of missing data: omitted items and not reached

items. An item was considered omitted if a student

skipped that question but answered one or more

questions following it. An item was considered as

not reached when neither that question nor any

question following it in the section was answered.

The percentage of omitted and not reached items

in Puerto Rico was not only substantially higher

than the national rate, but also higher than any

other jurisdiction’s rate. Note that the calculation

of an average score takes into account omitted and

incorrect responses, but not reached items are not

included in the calculation.

Low Percentage of Correct Items. In addition to the

high percentage of missing responses, there was

also a high percentage of incorrect responses in

Puerto Rico. The average percentage of correct

items was significantly lower for students in Puerto

Rico than for students in the nation. This could

indicate either a problem with how items functioned

in Puerto Rico or reflect knowledge and skills of

students in the nation.

High Levels of Item Misfit. The most problematic

result to emerge from the 2003 Puerto Rican mathematics assessment was item misfit. Item misfit is

defined as a mismatch between expected and actual

student performance on a test item. Although items

Technical Notes

About the NAEP administration in Puerto Rico

A full discussion of the content, administration, and data quality can be found in the forthcoming Technical Report of the NAEP Mathematics

Assessment in Puerto Rico: Focus on Statistical Issues. Published reports are available at http://nationsreportcard.gov/puertorico___2005/.
THE NATION’S REPORT CARD ● 17
Darvin M. Winick, Chair

President

Winick & Associates

Austin, Texas

Amanda P. Avallone, Vice Chair

Assistant Principal and

Eighth-Grade Teacher

Summit Middle School

Boulder, Colorado

Francie Alexander

Chief Academic Officer,

Scholastic, Inc.

Senior Vice President,

Scholastic Education

New York, New York

David J. Alukonis

Chairman

Hudson School Board

Hudson, New Hampshire

Barbara Byrd-Bennett

Executive Superintendentin-Residence

Cleveland State University

Cleveland, Ohio

Gregory Cizek

Professor of Educational

Measurement

University of North Carolina

Chapel Hill, North Carolina

Shirley V. Dickson

Educational Consultant

Aliso Viejo, California

Honorable David P. Driscoll

Commissioner of Education

Massachusetts Department

of Education

Malden, Massachusetts

John Q. Easton

Executive Director

Consortium on Chicago

School Research

University of Chicago

Chicago, Illinois

Alan J. Friedman

Consultant

Museum Development and

Science Communications

New York, New York

David W. Gordon

County Superintendent

of Schools

Sacramento County Office

of Education

Sacramento, California

Robin C. Hall

Principal

Beecher Hills Elementary School

Atlanta, Georgia

Kathi M. King

Twelfth-Grade Teacher

Messalonskee High School

Oakland, Maine

Honorable Keith King

Member

Colorado House

of Representatives

Denver, Colorado

Kim Kozbial-Hess

Fourth-Grade Teacher

Hawkins Elementary School

Toledo, Ohio

James S. Lanich

President

California Business for

Education Excellence

Sacramento, California

Honorable Cynthia Nava

Senator

New Mexico State Senate

Las Cruces, New Mexico

Andrew C. Porter

Director, Learning

Sciences Institute

Peabody College,

Vanderbilt University,

Nashville, Tennessee

Luis A. Ramos

Community Relations Manager

PPL Susquehanna

Berwick, Pennsylvania

Mary Frances Taymans, SND

Executive Director

Secondary Schools Department

National Catholic

Educational Association

Washington, D.C.

Oscar A. Troncoso

Principal

Socorro High School

El Paso, Texas

Honorable Thomas J. Vilsack

Former Governor of Iowa

Des Moines, Iowa

Honorable Michael E. Ward

Former North Carolina

Superintendent of

Public Instruction

Hattiesburg, Mississippi

Eileen L. Weiser

Former Member,

State Board of Education

Michigan Department of Education

Lansing, Michigan

Grover J. Whitehurst

(Ex officio)

Director

Institute of Education Sciences

U.S. Department of Education

Washington, D.C.

Charles E. Smith

Executive Director

National Assessment

Governing Board

Washington, D.C.

U . S . D E P A R T M E N T O F E D U C A T I O N

The National Assessment of Educational Progress (NAEP) is a congressionally mandated project sponsored by the U.S. Department of

Education. The National Center for Education Statistics, a department within the Institute of Education Sciences, administers NAEP.

The Commissioner of Education Statistics is responsible by law for carrying out the NAEP project.

Margaret Spellings

Secretary

U.S. Department

of Education

Grover J. Whitehurst

Director

Institute of

Education Sciences

Mark Schneider

Commissioner

National Center for

Education Statistics

Peggy Carr

Associate Commissioner

National Center for

Education Statistics

T H E N A T I O N A L A S S E S S M E N T G O V E R N I N G B O A R D

In 1988, Congress created the National Assessment Governing Board to set policy for the National

Assessment of Educational Progress, commonly known as the Nation’s Report Card TM. The Board

is an independent, bipartisan group whose members include governors, state legislators, local

and state school officials, educators, business representatives, and members of the general public.

T H E N A T I O N ' S

R E P O R T C A R D

Mathematics 2003 and 2005

Performance in Puerto Rico:

Highlights

March 2007

M O R E I N F O R M A T I O N

The report release site is

http://nationsreportcard.gov.

The NCES web electronic catalog

is http://nces.ed.gov/pubsearch.

For ordering information, write to

U.S. Department of Education

ED Pubs

P.O. Box 1398

Jessup, MD 20794-1398

or call toll free 1-877-4ED-Pubs

or order online at http://www.edpubs.org

S U G G E S T E D C I T A T I O N

Baxter, G.P., Bleeker, M.M.,

Waits, T.L., Salvucci, S. (2007).

The Nation’s Report Card:

Mathematics 2003 and 2005

Performance in Puerto Rico—

Highlights (NCES 2007–459).

U.S. Department of Education,

National Center for Education

Statistics, Washington, D.C.:

U.S. Government Printing Office.

C O N T E N T C O N T A C T

Emmanuel Sikali

202-502-7419

emmanuel.sikali@ed.gov

“O U R M I S S I O N I S T O E N S U R E E Q U A L A C C E S S T O E D U C A T I O N A N D

T O P R O M O T E E D U C A T I O N A L E X C E L L E N C E T H R O U G H O U T T H E N A T I O N .”

www.ed.gov