Geography 2001 The Nation’s Report Card

National Center for Education Statistics

U.S. Department of Education

Office of Educational Research and Improvement NCES 2002-484
What is The Nation’s Report Card?

THE NATION’S REPORT CARD, the National Assessment of Educational Progress (NAEP), is the only nationally

representative and continuing assessment of what America’s students know and can do in various subject areas. Since

1969, assessments have been conducted periodically in reading, mathematics, science, writing, history, geography, and

other fields. By making objective information on student performance available to policymakers at the national, state,

and local levels, NAEP is an integral part of our nation’s evaluation of the condition and progress of education. Only

information related to academic achievement is collected under this program. NAEP guarantees the privacy of

individual students and their families.

NAEP is a congressionally mandated project of the National Center for Education Statistics, the U.S. Department

of Education. The Commissioner of Education Statistics is responsible, by law, for carrying out the NAEP project

through competitive awards to qualified organizations. NAEP reports directly to the Commissioner, who is also

responsible for providing continuing reviews, including validation studies and solicitation of public comment, on

NAEP’s conduct and usefulness.

In 1988, Congress established the National Assessment Governing Board (NAGB) to formulate policy guidelines

for NAEP. The Board is responsible for selecting the subject areas to be assessed from among those included in the

National Education Goals; for setting appropriate student performance levels; for developing assessment objectives and

test specifications through a national consensus approach; for designing the assessment methodology; for developing

guidelines for reporting and disseminating NAEP results; for developing standards and procedures for interstate,

regional, and national comparisons; for determining the appropriateness of test items and ensuring they are free from

bias; and for taking actions to improve the form and use of the National Assessment.

The National Assessment Governing Board

Mark D. Musick, Chair

President

Southern Regional Education Board

Atlanta, Georgia

Michael T. Nettles, Vice Chair

Professor of Education

University of Michigan

Ann Arbor, Michigan

Melanie A. Campbell

Fourth-Grade Teacher

Topeka, Kansas

Honorable Wilmer S. Cody

Former Commissioner of Education

State of Kentucky

Frankfort, Kentucky

Daniel A. Domenech

Superintendent of Schools

Fairfax County Public Schools

Fairfax, Virginia

Edward Donley

Former Chairman

Air Products & Chemicals, Inc.

Allentown, Pennsylvania

Thomas H. Fisher

Director

Student Assessment Services

Florida Department of Education

Tallahassee, Florida

Edward H. Haertel

Professor, School of Education

Stanford University

Stanford, California

Juanita Haugen

Local School Board Member

Pleasanton, California

Honorable Dirk Kempthorne

Governor of Idaho

Boise, Idaho

Honorable Nancy Kopp

State Legislator

Annapolis, Maryland

Honorable Ronnie Musgrove

Governor of Mississippi

Jackson, Mississippi

Roy M. Nageak, Sr.

First Vice-Chair

Alaska Board of Education and

Early Development

Barrow, Alaska

Debra Paulson

Eighth-Grade Mathematics Teacher

El Paso, Texas

Honorable Jo Ann Pottorff

State Legislator

Wichita, Kansas

Diane Ravitch

Research Professor

New York University

New York, New York

Sister Lourdes Sheehan, R.S.M.

Secretary for Education

United States Catholic Conference

Washington, DC

John H. Stevens

Executive Director

Texas Business and Education

Coalition

Austin, Texas

Migdania D. Vega

Principal

Coral Way Elementary Bilingual

School

Miami, Florida

Deborah Voltz

Assistant Professor

Department of Special Education

University of Louisville

Louisville, Kentucky

Honorable Michael E. Ward

State Superintendent of Public

Instruction

North Carolina Public Schools

Raleigh, North Carolina

Marilyn A. Whirry

Twelfth-Grade English Teacher

Manhattan Beach, California

Dennie Palmer Wolf

Director, Annenberg Institute

Brown University

Providence, Rhode Island

Grover J. Whitehurst (Ex-Officio)

Assistant Secretary of Education

Office of Educational Research and

Improvement

U.S. Department of Education

Washington, DC

Roy Truby

Executive Director, NAGB

Washington, DC
Andrew R. Weiss

Anthony D. Lutkus

Barbara S. Hildebrant

Matthew S. Johnson

in collaboration with

Scott Davis

Wendy S. Grigg

Mei-Jang Lin

Frank Jenkins

Yuxin Tang

June 2002

U.S. Department of Education

Office of Educational Research and Improvement NCES 2002–484

Geography 2001

National Center for Education Statistics

The Nation’s Report Card
U.S. Department of Education

Rod Paige

Secretary

Office of Educational Research and Improvement

Grover J. Whitehurst

Assistant Secretary

National Center for Education Statistics

Gary W. Phillips

Deputy Commissioner

June 2002

SUGGESTED CITATION

U.S. Department of Education. Office of Educational Research and Improvement. National Center for Education Statistics.

The Nation’s Report Card: Geography 2001, NCES 2002–484, by A. R.Weiss, A. D. Lutkus, B. S. Hildebrant, &

M. S. Johnson. Washington, DC: 2002.

FOR MORE INFORMATION

Content contact:

Arnold Goldstein

202–502–7344

To obtain single copies of this report, limited number of copies available, or ordering information on other U.S.

Department of Education products, call toll free 1–877–4ED-PUBS (877–433–7827), or write:

Education Publications Center (ED Pubs)

U.S. Department of Education

P.O. Box 1398

Jessup, MD 20794–1398

TTY/TDD 1–877–576–7734

FAX 301–470–1244

Online ordering via the Internet: http://www.ed.gov/pubs/edpubs.html

Copies also are available in alternate formats upon request.

This report also is available on the World Wide Web: http://nces.ed.gov/nationsreportcard

The work upon which this publication is based was performed for

the National Center for Education Statistics by Educational Testing Service.
T A B L E O F C O N T E N T S • G E O G R A P H Y R E P O R T C A R D iii

Table of Contents

The

Nation’s

Report

Card

Executive Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix

Chapter 1

NAEP 2001 Geography Assessment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

Overview of the 2001 National Assessment of Educational Progress . . . . . . . . . . . . . . . 2

The Geography Framework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

Geography Assessment Instruments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

Description of School and Student Samples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

Reporting the Assessment Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

The Setting of Achievement Levels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

Achievement-Level Descriptions for Each Grade . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

The Trial Status of Achievement Levels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

Interpreting NAEP Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

Overview of the Remaining Report . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

Chapter 2

Average Scale Score and Achievement-Level Results for the Nation . . . . . . . . . . . . . . 17

Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

Average Scale Score Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

Scale Scores by Percentile . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

Achievement-Level Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

Chapter 3

Average Scale Scores and Achievement-Level Results for Selected Subgroups . . . . . . 23

Gender . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

Race/Ethnicity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

Region of the Country . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

Parents’ Highest Level of Education . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

Type of School . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

Type of Location . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

Free/Reduced-Price School Lunch Program Eligibility . . . . . . . . . . . . . . . . . . . . . . . . 48
iv T A B L E O F C O N T E N T S • GEOGRAPHY REPORT CARD

Chapter 4

Classroom Contexts for Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51

Teacher Background and Preparedness to Teach Geography . . . . . . . . . . . . . . . . . . . . 52

Geography Skills and Topics Taught in Grade 4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56

Geography Skills Taught in Grades 8 and 12 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58

The Extent of Students’ Social Studies and Geography Instruction . . . . . . . . . . . . . . . 61

The Use of Computers in the Social Studies Classroom in Grades 4 and 8 . . . . . . . . . 64

The Use of Computers in Grade 12 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67

Student Interest in Geography Grades 8 and 12 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69

Chapter 5

Becoming a More Inclusive National Assessment . . . . . . . . . . . . . . . . . . . . . . . . . . . 71

Two Sets of 2001 NAEP Geography Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72

Results for the Nation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75

National Results by Gender . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77

National Results by Race/Ethnicity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78

Chapter 6

Sample Assessment Questions and Student Responses . . . . . . . . . . . . . . . . . . . . . . . 81

Grade 4 Sample Assessment Questions and Results . . . . . . . . . . . . . . . . . . . . . . . . . 82

Grade 8 Sample Assessment Questions and Results . . . . . . . . . . . . . . . . . . . . . . . . . 92

Grade 12 Sample Assessment Questions and Results . . . . . . . . . . . . . . . . . . . . . . . 100

Maps of Selected Item Descriptions

on the NAEP Geography Scale—Grades 4, 8, and 12 . . . . . . . . . . . . . . . . . . . . . . . 108

Appendix A

Overview of Procedures Used for the NAEP 2001 Geography Assessment . . . . . . . . . 113

Appendix B

Data Appendix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141

Appendix C

Members of the NAEP Geography Standing Committee . . . . . . . . . . . . . . . . . . . . . . 181

Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182
TABLE OF CONTENTS • GEOGRAPHY REPORT CARD v

Chapter 1: Figures and Tables

Figure 1.1: Distribution of assessment time by geography content area,

grades 4, 8, and 12: 2001 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

Figure 1.2: Descriptions of the three geography content areas . . . . . . . . . . . . . . . . . . 4

Table 1.1: Distribution of geography assessment time across cognitive areas,

grades 4, 8, and 12: 2001 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

Figure 1.3: Policy definitions of the three NAEP achievement levels . . . . . . . . . . . . . . 9

Figure 1.4: Descriptions of NAEP geography achievement levels for grade 4 . . . . . . . 10

Figure 1.5: Descriptions of NAEP geography achievement levels for grade 8 . . . . . . . 11

Figure 1.6: Descriptions of NAEP geography achievement levels for grade 12 . . . . . . 12

Chapter 2: Figures and Tables

Figure 2.1: Average geography scale scores,

grades 4, 8, and 12: 1994 and 2001 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

Figure 2.2: Geography scale score percentiles,

grades 4, 8, and 12: 1994 and 2001 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

Figure 2.3: Percentage of students within and at or above

geography achievement levels, grades 4, 8, and 12: 1994 and 2001 . . . . . . . . . . . . 21

Chapter 3: Figures and Tables

Figure 3.1: Average geography scale scores by gender,

grades 4, 8, and 12: 1994 and 2001 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

Figure 3.2: Differences in average geography scale scores by gender,

grades 4, 8, and 12: 1994 and 2001 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

Figure 3.3: Percentage of students within and at or above

geography achievement levels by gender, grades 4, 8, and 12: 1994 and 2001 . . . . 26

Figure 3.4: Average geography scale scores by race/ethnicity,

grades 4, 8, and 12: 1994 and 2001 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

Figure 3.5: Differences in average geography scale scores by race/ethnicity,

grades 4, 8, and 12: 1994 and 2001 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

Figure 3.6a: Percentage of students within and at or above

geography achievement levels by race/ethnicity, grade 4: 1994 and 2001 . . . . . . . . 30

Figure 3.6b: Percentage of students within and at or above

geography achievement levels by race/ethnicity, grade 8: 1994 and 2001 . . . . . . . . 31

Figure 3.6c: Percentage of students within and at or above

geography achievement levels by race/ethnicity, grade 12: 1994 and 2001 . . . . . . . 32

Figure 3.7: Average geography scale scores by region of the country,

grades 4, 8, and 12: 1994 and 2001 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

Figure 3.8a: Percentage of students within and at or above

geography achievement levels by region of the country, grade 4: 1994 and 2001 . . . . . . 34

Figure 3.8b: Percentage of students within and at or above

geography achievement levels by region of the country, grade 8: 1994 and 2001 . . . . . . 35

Figure 3.8c: Percentage of students within and at or above

geography achievement levels by region of the country, grade 12: 1994 and 2001 . . . . . 36

Figure 3.9: Average geography scale scores by

parents’ highest level of education, grades 8 and 12: 1994 and 2001 . . . . . . . . . . . 37
vi T A B L E O F C O N T E N T S • GEOGRAPHY REPORT CARD

Figure 3.10a: Percentage of students within and at or above geography

achievement levels by parents’ highest level of education, grade 8: 1994 and 2001 . . . . 39

Figure 3.10b: Percentage of students within and at or above geography

achievement levels by parents’ highest level of education, grade 12: 1994 and 2001 . . . 40

Figure 3.11: Average geography scale scores by type of school,

grades 4, 8, and 12: 1994 and 2001 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42

Figure 3.12a: Percentage of students within and at or above

geography achievement levels by type of school, grade 4: 1994 and 2001 . . . . . . . . 43

Figure 3.12b: Percentage of students within and at or above

geography achievement levels by type of school, grade 8: 1994 and 2001 . . . . . . . . 44

Figure 3.12c: Percentage of students within and at or above

geography achievement levels by type of school, grade 12: 1994 and 2001 . . . . . . . 45

Table 3.1: Average geography scale scores by type of school location,

grades 4, 8, and 12: 2001 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

Figure 3.13: Percentage of students within and at or above geography

achievement levels by type of school location, grades 4, 8, and 12: 2001 . . . . . . . . 47

Table 3.2: Average geography scale scores by student eligibility for

Free/Reduced-Price School Lunch program, grades 4, 8, and 12: 2001 . . . . . . . . . . 48

Figure 3.14: Percentage of students within and at or above

geography achievement levels by Free/Reduced-Price School Lunch

program eligibility, grades 4, 8, and 12: 2001 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

Chapter 4: Figures and Tables

Table 4.1: Percentage of students and average geography scale scores

by teachers’ reported undergraduate/graduate major and minor/special emphasis,

grades 4 and 8: 2001 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53

Table 4.2: Percentage of students and average geography scale scores

by teachers’ reports on how well prepared they felt they were to teach geography,

grades 4 and 8: 1994 and 2001 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55

Table 4.3: Percentage of students and average geography scale scores

by teachers’ reports on frequency of instruction of selected skills and topics,

grade 4: 1994 and 2001 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57

Table 4.4a: Percentage of students and average geography scale scores

by students’ reports on frequency of instruction of selected skills and topics,

grade 8: 1994 and 2001 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59

Table 4.4b: Percentage of students and average geography scale scores

by students’ reports on frequency of instruction of selected skills and topics,

grade 12: 1994 and 2001 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60

Table 4.5: Percentage of students and average geography scale scores

by students’ reports on grades in which geography was taken since the 6th grade,

grade 8: 1994 and 2001 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62

Table 4.6: Percentage of students and average geography scale scores

by students’ reports on grades in which geography was taken since 9th grade,

grade 12: 1994 and 2001 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63

Table 4.7a: Percentage of students and average geography scale scores

by teachers’ reports on computer use for social studies instruction,

grade 4: 2001 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
TABLE OF CONTENTS • G E O G R A P H Y R E P O R T C A R D vii

Table 4.7b: Percentage of students and average geography scale scores

by teachers’ reports on computer use for social studies instruction,

grade 8: 2001 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66

Table 4.8: Percentage of students and average geography scale scores

by students’ reports on computer use for history and geography,

grade 12: 2001 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68

Table 4.9: Percentage of students and average geography scale scores

by students’ reports on how much they like studying geography,

grades 8 and 12: 1994 and 2001 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69

Chapter 5: Figures and Tables

Figure 5.1: The two sets of NAEP results based on a split-sample design . . . . . . . . . 74

Table 5.1: National average geography scale scores by type of results,

grades 4, 8, and 12: 2001 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76

Table 5.2: Percentage of students within and at or above geography

achievement levels by type of results, grades 4, 8, and 12: 2001 . . . . . . . . . . . . . . 77

Table 5.3: National average geography scale scores by gender and type of results,

grades 4, 8, and 12: 2001 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77

Table 5.4: Percentage of students within and at or above geography

achievement levels by gender and type of results, grades 4, 8, and 12: 2001 . . . . . . 78

Table 5.5: National average geography scale scores by race/ethnicity

and type of results, grades 4, 8, and 12: 2001 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79

Table 5.6: Percentage of students within and at or above geography achievement

levels by race/ethnicity and type of results, grades 4, 8, and 12: 2001 . . . . . . . . . . . 80

Chapter 6: Figures and Tables

Table 6.1: Grade 4 Sample Question 1 Results (Multiple-Choice) . . . . . . . . . . . . . . . 83

Table 6.2: Grade 4 Sample Question 2 Results (Multiple-Choice) . . . . . . . . . . . . . . . 84

Table 6.3a: Grade 4 Sample Question 3 Results

(“Complete” Short-Constructed-Response) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86

Table 6.3b: Grade 4 Sample Question 3 Results

(“Partial” Short-Constructed-Response) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87

Table 6.4a: Grade 4 Sample Question 4 Results

(“Complete” Extended-Constructed-Response) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89

Table 6.4b: Grade 4 Sample Question 4 Results

(“Essential” Extended-Constructed-Response) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90

Table 6.4c: Grade 4 Sample Question 4 Results

(“Partial” Extended-Constructed-Response) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91

Table 6.5: Grade 8 Sample Question 5 Results (Multiple-Choice) . . . . . . . . . . . . . . . 93

Table 6.6: Grade 8 Sample Question 6 Results (Multiple-Choice) . . . . . . . . . . . . . . . 94

Table 6.7: Grade 8 Sample Question 7 Results (Multiple-Choice) . . . . . . . . . . . . . . . 95

Table 6.8: Grade 8 Sample Question 8 Results (Multiple-Choice) . . . . . . . . . . . . . . . 96

Table 6.9a: Grade 8 Sample Question 9 Results

(“Complete” Short-Constructed-Response) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98
viii T A B L E O F C O N T E N T S • GEOGRAPHY REPORT CARD

Table 6.9b: Grade 8 Sample Question 9 Results

(“Partial” Short-Constructed-Response) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99

Table 6.10: Grade 12 Sample Question 10 Results (Multiple-Choice) . . . . . . . . . . . 100

Table 6.11: Grade 12 Sample Question 11 Results (Multiple-Choice) . . . . . . . . . . . 101

Table 6.12a: Grade 12 Sample Question 12 Results

(“Complete” Short-Constructed-Response) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103

Table 6.12b: Grade 12 Sample Question 12 Results

(“Partial” Short-Constructed-Response) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104

Table 6.13a: Grade 12 Sample Question 13 Results

(“Complete” Short-Constructed-Response) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106

Table 6.13b: Grade 12 Sample Question 13 Results

(“Partial” Short-Constructed-Response) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107

Figure 6.1: Grade 4 Item Map

Map of selected item descriptions on the

National Assessment of Educational Progress geography scale for grade 4 . . . . . . . 109

Figure 6.2: Grade 8 Item Map

Map of selected item descriptions on the

National Assessment of Educational Progress geography scale for grade 8 . . . . . . . 110

Figure 6.3: Grade 12 Item Map

Map of selected item descriptions on the

National Assessment of Educational Progress geography scale for grade 12 . . . . . . 111
EXECUTIVE SUMMARY • G E O G R A P H Y R E P O R T C A R D ix

Executive Summary

The

Nation’s

Report

Card

Major Findings at

Grades 4, 8,

and 12

Results for

Student

Subgroups

Classroom

Contexts for

Learning

Becoming a

More Inclusive

NAEP

The National Assessment of Educational Progress (NAEP) is

the nation’s only ongoing representative sample survey of

student achievement in core subject areas. In 2001, NAEP

conducted a geography assessment of the nation’s fourth-,

eighth-, and twelfth-grade students.

Authorized by Congress and administered by the National

Center for Education Statistics (NCES) in the U.S.

Department of Education, NAEP regularly reports to the

public on the educational progress of students in grades 4, 8,

and 12. This report presents the results of the NAEP 2001

geography assessment for the nation. Results in 2001 are

compared to results of the 1994 NAEP geography

assessment, which was the preceding NAEP geography

assessment and the only other geography assessment

conducted under the current framework. Students’

performance on the assessment is described in terms of

average scores on a 0–500 scale and in terms of the

percentage of students attaining three achievement levels:

Basic, Proficient, and Advanced. The achievement levels are

performance standards adopted by the National Assessment

Governing Board (NAGB) as part of its statutory

responsibilities. They represent collective judgments of what

students should know and be able to do.
x EXECUTIVE SUMMARY • GEOGRAPHY REPORT CARD

As provided by law, the Deputy Commissioner of Education Statistics, upon

review of a congressionally mandated

evaluation of NAEP, determined that the

achievement levels are to be used on a trial

basis and should be interpreted with

caution. However, both the Deputy Commissioner and the NAGB believe these

performance standards are useful for understanding trends in student achievement.

They have been widely used by national

and state officials as a common yardstick of

academic performance.

In addition to providing average scores

and achievement-level performance in

geography for the nation’s fourth-, eighth-,

and twelfth-graders, this report provides

results for subgroups of students at those

grade levels defined by various background

characteristics (such as gender, race/

ethnicity, region, parents’ education, etc.)

and classroom contexts for learning. A

summary of major findings from the 2001

NAEP geography assessment is presented

on the following pages. Differences between results across years or between

groups of students are discussed only if they

have been determined to be statistically

significant. Readers are cautioned that the

relationship between a contextual variable

and student performance is not necessarily

causal.

Major Findings at

Grades 4, 8, and 12

 Average geography scores for fourthand eighth-graders were higher in 2001

than in 1994, while the performance of

twelfth-graders was not significantly

different.

 At both grades 4 and 8, score increases

occurred among the lower-performing

students (at the 10th and 25th percentiles).

 The 2001 geography assessment showed

that 21 percent of fourth-graders, 30

percent of eighth-graders, and 25 percent of twelfth-graders performed at or

above the Proficient level for their

respective grades. These levels are

identified by NAGB as those at which

all students should perform.

 Both grades 4 and 8 showed an increase

from 1994 to 2001 in the percentage of

students at or above Basic. There were no

significant changes in the percentage at

or above Proficient at any grade.

Results for Student Subgroups

In addition to overall results, NAEP reports

on the performance of various subgroups

of students. Observed differences between

student subgroups in NAEP geography

performance may reflect a range of socioeconomic and educational factors not

addressed in this report or by NAEP.

Gender

 There was no statistically significant

change at any grade in the average scores

of either male or female students between 1994 and 2001.

 In 2001 as in 1994, male students at

grades 4, 8, and 12 had higher average

scores than female students.

Race/Ethnicity

 At grade 4, Black students had higher

average scores in 2001 than in 1994.

 In 2001, White, Asian/Pacific Islander,

and American Indian students had

higher average scores than Black and

Hispanic students at all three grades.

 The 2001 results show a narrowing of

the average score point difference between White students and Black students

at grade 4.
EXECUTIVE SUMMARY • G E O G R A P H Y R E P O R T C A R D xi

Region of the Country

 Between 1994 and 2001, the average

scores of fourth-graders increased in the

Northeast, and the average scores of

eighth-graders increased in the Southeast.

 Fourth- and eighth-grade students in the

Northeast and Central regions outperformed students in the West in 2001, and

students in the Central region also

outperformed their counterparts in the

Southeast. Twelfth-graders in the Central

region had higher average scores than

twelfth-graders in the Southeast.

Parents’ Highest Level of Education

 Twelfth-graders whose parents had not

graduated from high school had higher

average scores in 2001 than in 1994.

 The higher the parental education level

reported, the higher the average score

attained by students at both grades 8 and

12 in 2001.

Type of School

 Eighth-grade public school students had

higher average scores in 2001 than in

1994.

 In 2001, nonpublic school students

outperformed public school students at

all three grades.

 In 2001, Catholic school students outperformed public school students at

grades 4, 8, and 12. Apparent differences

between public school and other

nonpublic school students were not

statistically significant.

Type of Location

 In 2001, students in rural and urban

fringe locations had higher average

scores than central city students at grades

4, 8, and 12.

Eligibility for Free/Reduced-Price

Lunch

 At every grade in 2001, the average

score for students who were eligible for

the Free/Reduced-Price School Lunch

program was lower than the average for

students who were not eligible for the

program (i.e., those above the poverty

guidelines).

Classroom Contexts for Learning

NAEP collects information about the

contexts for student learning by administering questionnaires to assessed students,

their teachers, and their school administrators. Using the student as the unit of

analysis, NAEP examines the relationship

between selected contextual variables

drawn from these questionnaires and

students’ average scores on the geography

assessment.

Teacher Preparation

 Ninety-three percent of fourth-grade

students had teachers who indicated

their graduate/undergraduate major or

minor was elementary education, and

about one-quarter (28 percent) of

eighth-grade students had teachers who

indicated they had a graduate/undergraduate major or minor in geography

or geography education.

 A higher percentage of fourth-grade

students in 2001 had teachers who

reported they were very prepared to

teach geography than did students in

1994. Forty-four percent of eighth-grade

students in 2001 had teachers who

reported they were very prepared to

teach geography.
xii E X E C U T I V E S U M M A R Y • GEOGRAPHY REPORT CARD

Geography Skills Taught

 The percentage of eighth-grade students

who studied maps and globes at least

once or twice a week increased in 2001

as compared to 1994.

 There was an increase in the percentage

of eighth- and twelfth-grade students

who studied natural resources once or

twice a week in 2001 as compared

with 1994.

 The percentages of eighth-grade students who studied countries and cultures

in their geography instruction at least

once or twice a week were greater in

2001 than in 1994.

Geography Course-Taking

 A higher percentage of eighth-graders in

2001 reported taking geography in sixth,

seventh, and eighth grades than did their

counterparts in 1994.

 The percentage of twelfth-grade

students taking geography courses at

each grade level during their high

school years increased in 2001 from

the percentage reported in 1994.

 In 2001 at grade 8, students who reported taking two or three years of

geography had higher scores than those

who took it for fewer years. Twelfthgraders who reported taking one year or

less of geography had higher average

scores than those who took 3 or 4 years

of geography.

Use of Computers

 Students at grades 4, 8, and 12 who used

the Internet or CD-ROM materials to a

small or moderate extent had higher

scores than students who did not use

these tools at all.

Becoming a More Inclusive NAEP

In the 2001 geography assessment, the

NAEP program used a split-sample design,

so that trends in students’ geography

achievement could be reported across

assessment years and, at the same time, the

program could continue to examine the

effects of including special-needs students

assessed with accommodations. Included in

this report is an overview of the second set

of results that include special-needs students who required and were provided

accommodations during the assessment

administration.

 In the sample where accommodations

were not permitted, between 44 and 48

percent of the special-needs students at

each of the three grade levels (between

5 and 8 percent of all students) were

excluded from NAEP testing by their

schools. In the sample where accommodations were offered, between 23 and

24 percent of the special-needs students

were excluded from the assessment

(between 2 and 4 percent of the total

sample).

 At grade 8, the average score when

accommodations were permitted was

lower than the average score when

accommodations were not permitted.

At grades 4 and 12, there were no

statistically significant differences between the average scores of students

when accommodations were permitted

and when accommodations were not

permitted.
CHAPTER 1 • GEOGRAPHY REPORT CARD 1

1

Chapter

Contents

Overview

Geography

Framework

Geography

Assessment

School and

Student Samples

Reporting

Results

NAEP

Achievement

Levels

Interpreting

NAEP Results

This Report

What is the NAEP

geography

assessment?

How does the

NAEP geography

assessment

measure and

report student

progress?

Chapter

Focus

NAEP 2001 Geography Assessment

Introduction

After more than 50 years during which geography was

largely replaced by social studies in American public schools,

geography education began to experience a revival during

the 1980s and 1990s.1

 Contributing to the change was a

growing belief in the relevance of geography to addressing

economic, political, and environmental issues at the

national and global level. Moreover, geography

education was increasingly seen as an essential tool in

the creation of effective citizens. This process gained

momentum through the work of various

organizations concerned with geography and

geography education. These groups encouraged a

more positive attitude toward geography and

provided important guidance for reestablishing

geography in the school curriculum.2 Two surveys of

geographic literacy, in 1988 and 1994, provided

statistical evidence that student knowledge and skills

fell far short of what was needed for responsible

citizenship.3 By the end of 1990, Congress had authorized

development of a broad-based National Assessment of

Educational Progress (NAEP) geography assessment at

1 Salter, C. L. (1990). Missing the magic carpet: The real significance of geographic ignorance.

Princeton, NJ: Educational Testing Service.

2 Joint Committee on Geographic Education. (1984). Guidelines for geographic education:

Elementary and secondary schools. Washington, DC: Association of American Geographers

and the National Council for Geographic Education.

3 Allen, R., Bettis, N., Kurfman, D., MacDonald, W., Mullis, I. V. S., & Salter, C. (1990).

The geography learning of high school seniors. Princeton, NJ: National Assessment of

Educational Progress, Educational Testing Service.

Persky, H. R., Reese, C. M., O’Sullivan, C. Y., Lazer, S., Moore, J. D., & Shakrani, S.

(1996). NAEP 1994 geography report card. Washington, DC: National Center for

Education Statistics, Office of Educational Research and Improvement,

U.S. Department of Education.
2 CHAPTER 1 • GEOGRAPHY REPORT CARD

grades 4, 8, and 12, and the President and

nation’s governors had declared geography

to be one of five core subjects in their

National Education Goals.

 Progress toward increasing the prominence of geography in the elementary and

secondary school curriculum has generally

been good. The 1990s saw the publication

of the Geography Framework for the 1994

National Assessment of Educational Progress

and the NAEP geography assessment in

1994, the introduction of the National

Geography Standards, and the institution of

the National Geographic Alliance Network.4 The alliance is a professional organization encouraged and supported with

grants from the National Geographic

Society Education Foundation. Geographic Alliances are present in all 50 states,

and are comprised of primary, secondary,

community college, and university geography educators interested in the enhancement of geography education. The number

of states with geography standards has been

increasing steadily as well. According to

recent data collected by the National

Geographic Society, 48 states plus the

District of Columbia now have geography

standards in place, 37 of which are based

on the National Geography Standards.

However, only 13 states require a geography course as a requirement for high

school graduation. Moreover, in 27 states

geography is not tested in mandated state

examinations, while in some other states

the portion of mandated tests devoted to

geography is very small. As a result, there

could be little incentive for teachers to

emphasize geography instruction when

higher stakes are attached to other subjects.5

The results from the 2001 NAEP geography

assessment provide policymakers, educators,

and the general public with a new, objective

tool with which to evaluate the country’s

progress toward geographic literacy.

Overview of the 2001

National Assessment of

Educational Progress

For over 30 years, the National Assessment

of Educational Progress (NAEP) has been

authorized by Congress to collect, analyze,

and report reliable and valid information

about what American students know and

can do in core subject areas. NAEP assesses

the performance of public and nonpublic

school students in grades 4, 8, and 12. In

2001, student performance in geography

and U.S. history was assessed at all three

grades. This report deals only with the

results of the geography assessment.

All NAEP assessments are based on

content frameworks developed through a

national consensus process. The NAEP

2001 geography assessment was the second

administration of an assessment based on

the Geography Framework for the 1994

National Assessment of Educational Progress,

which was originally developed for the

1994 assessment.6 In both 1994 and 2001,

4 National Assessment Governing Board. (1994). Geography framework for the 1994 National Assessment of Educational

Progress. Washington, DC: Author.

Geography Education Standards Project. (1994). Geography for life: National geography standards. Washington, DC:

National Geographic Research and Exploration.

5 Munroe, S. and Smith, T. (1998). State geography standards. Fordham Report, 2(2), http://www.edexcellence.net/

standards/geography/geograph.htm.

Dean, A. (2002). Unpublished data. National Geographic Education Foundation.

Council of Chief State School Officers. (2000). Key state education policies on K-12 education: 2000. Washington,

DC: Author.

6 National Assessment Governing Board. (1994). Geography framework for the 1994 National Assessment of Educational

Progress. Washington, DC: Author.
CHAPTER 1 • GEOGRAPHY REPORT CARD 3

assessments based on the framework were

administered to national samples of fourth-,

eighth-, and twelfth-graders.

This report describes the results of the

2001 geography assessment at grades 4, 8,

and 12 and compares results in 2001 to

those in 1994. Comparisons across assessment years are possible because the assessments were developed under the same

basic framework and share a common set

of geography questions. In addition, the

populations of students were sampled and

assessed using comparable procedures.

The Geography Framework

Although NAEP had conducted a geography assessment at grade 12 in 1988, a more

comprehensive NAEP geography framework was developed for the 1994 assessment. The new framework provided the

operational specifications for both the 1994

and 2001 assessments. The development of

the framework was managed by the Council of Chief State School Officers

(CCSSO) and adopted by the National

Assessment Governing Board (NAGB).

Approximately 50 professional geographers,

educators, administrators, and other interested individuals worked to achieve consensus on the general goals as well as the

specific language of the framework. In

addition, several hundred educational

experts and interested members of the

public contributed to the process, either by

participating in public hearings or by

reviewing drafts. The framework document

produced by this consensus process called

for the assessment of a broad range of

outcomes. It represented an ambitious

vision both of what students should know

and be able to do in geography, and of the

ways in which those competencies should

be tested.

The geography framework is organized

along two dimensions, a content dimension

and a cognitive dimension. The content

dimension forms the heart of the framework. It is divided into three main content

areas covering the breadth of geography

learning outcomes—knowledge and

skills—that would flow from good geography instruction.

The geography framework specifies the

percentage of assessment time to be

devoted to each content area. Figure 1.1

shows how the assessment time is distributed for each of the three grades: 40

Distribution of assessment time by geography content area,

grades 4, 8, and 12: 2001

Figure 1.1

Distribution of

Assessment Time

Space and Place

Environment and Society

Spatial Dynamics and Connections

40% 30%

30%

SOURCE: National Assessment Governing Board, Geography Framework for the 2001 National Assessment of Educational Progress.
4 CHAPTER 1 • GEOGRAPHY REPORT CARD

percent of assessment time goes to Space

and Place, and 30 percent each to Environment and Society and to Spatial Dynamics

and Connections. The percentages are

important both because they guide the

development of test questions and because

they determine how much weight each

content area receives in computing overall

test scores. Figure 1.2 provides descriptions

of each content area.

SOURCE: National Assessment Governing Board, Geography Framework for the 1994 and 2001 National Assessment of Educational Progress.

Space and Place:

Knowledge of geography as it relates to particular places on Earth, to spatial patterns on Earth’s surface,

and to physical and human processes that shape such spatial patterns.

Space is the basic resource and organizing element for geography. Patterns that are illustrated

on maps reflect both natural features and human activities. This content area requires students

to distinguish between and understand the spatial distribution of physical and human characteristics. Students must locate significant features and places on Earth, recognize existing

patterns in the distribution of features and places, and comprehend the reasons for the

development and existence of these patterns.

Environment and Society:

Knowledge of geography as it relates to the interactions between environment and society.

Geography is an integrative discipline that focuses on the interrelationships between the physical

environment and society. Human adaptation to and modification of the environment have

economic and political implications. Understanding the nature, scale, and ramifications of such

environmental transformations is fundamental in geography education, and is the core of this

content area. Students must be aware that every environmental issue lends itself to many

interpretations, depending on the people’s perspectives. Students must consider such multiple

perspectives as they evaluate decisions about issues, such as land use and resource development, because the results of such decisions often have complicated and unpredictable consequences. Learning to make wise decisions concerning the costs and benefits of environmental

modification is an expressed goal of geography education.

Spatial Dynamics and Connections:

Knowledge of geography as it relates to spatial connections among people, places, and regions.

This content area explores critical problems in human interaction. It requires students to

demonstrate comprehension of cultural, economic, and political regions and the connections

among them. Students must understand how peoples and places are alike and how they differ.

They should know that people of every country and every nation are increasingly connected to and

dependent upon other peoples and places of the world for both human and natural resources. In

this content area, students must demonstrate the knowledge that the world’s resources are

unevenly distributed, and an understanding of how this contributes to the movements of people,

patterns of trade, and conflict.

Descriptions of the three geography content areas Figure 1.2

Content Area

Descriptions
CHAPTER 1 • GEOGRAPHY REPORT CARD 5

Three cognitive areas or levels comprise

the cognitive dimension of the geography

assessment. The framework labels them as

Knowing, Understanding, and Applying,

and defines them as follows.

Knowing—What is it? Where is it?

In this area, students are assessed on their

ability to perform two related functions

with respect to information: a) an observation function and b) a recall function.

Students should be able to observe different elements of the landscape and answer

questions by recalling, for example, the

name of a place or a resource indigenous to

a particular country or by finding information about trading patterns among several

countries.

Understanding—Why is it there? How did

it get there? What is its significance?

In this area, students attribute meaning

to what has been observed and explain

events. Putting events in context and

explaining them requires students to see

connections among diverse pieces of

geographic information and to use that

information to explain existing patterns

and processes on Earth.

Applying—How can knowledge and

understanding be used to solve geographic

problems?

Applying geography knowledge and

understanding requires a range of higherorder thinking skills. Students classify,

hypothesize, use inductive and deductive

reasoning, and form problem-solving

models. They use many tools and skills of

geography as they attempt to develop a

comprehensive understanding en route to

proposing viable solutions.

Student performance in the three cognitive areas was not reported on separate

subscales. Rather, the three areas were used

to help guide development of the assessment instrument. The percentages of

assessment time to be devoted to each

cognitive area, as specified in the framework, are displayed in table 1.1.

Together the content and cognitive

dimensions of the assessment form a matrix

in which each content area is measured at

each cognitive level.

Table 1.1 Geography Assessment Time Across Cognitive Areas

Distribution of geography assessment time across cognitive areas, grades 4, 8, and 12: 2001

Knowing Understanding Applying

Grade 4 45% 30% 25%

Grade 8 40% 30% 30%

Grade 12 30% 30% 40%

SOURCE: National Assessment Governing Board, Geography Framework for the 1994 and 2001 National Assessment of Educational Progress.
6 CHAPTER 1 • GEOGRAPHY REPORT CARD

Geography Assessment

Instruments

As the only federally authorized ongoing

assessment of geography achievement,

NAEP must reflect the spirit of the framework as well as the specifications provided

by it. In order to achieve those goals, the

assessment development process involved

stages of review by measurement experts

and a committee of teachers, teacher

educators, and curriculum specialists expert

in geography. All components of the

assessment were evaluated for curricular

relevance, developmental appropriateness,

and fairness. The National Assessment

Governing Board (NAGB) gave final

approval for NAEP test questions. A list of

the geography development committee

members for the 2001 assessment is provided in appendix C.

The 2001 geography assessment booklets

at grades 4, 8, and 12 contained either three

or four sections: a set of general background

questions, a set of subject-related background questions, and one or two sets, or

“blocks,” of cognitive questions assessing

knowledge and skills in geography. The

general background questions are used to

collect some important basic information

about students. These questions tend to

remain fairly constant across different

NAEP assessments. The subject-related

questions are designed for specific assessments or for assessments given in an individual year. The questions in the geography

assessment asked students to give information about their school practices, such as

the frequency with which they used the

Internet or a CD-ROM to study geography, how often they received instruction in

using maps and globes, and when they had

taken a geography course. All students

participating in the geography assessment at

a particular grade received the same background questions.

The geography assessment as a whole

contained 91 questions at grade 4, 124

questions at grade 8, and 123 questions at

grade 12. The grade 4 assessment was

divided into six 25-minute blocks, while

both the grade 8 and grade 12 assessments

contained nine blocks, eight of which were

25-minute blocks and one of which was a

50-minute block. However, to reduce the

burden on individuals, each student answered only a small portion of the total

number of questions—either two 25-

minute blocks or one 50-minute block.

The 50-minute blocks administered at

grades 8 and 12 focused on a particular

geographic topic. In addition, one block at

each grade was based entirely upon a

student atlas that was provided to students.

The assessment time for each grade, therefore, was 50 minutes plus the 10-15 minutes needed to complete the background

questions.

Each block of geography questions

consisted of both multiple-choice and

“constructed-response” questions. (“Constructed response” is the term used to

describe test questions in which students

produce their own response, as distinct

from multiple-choice questions, in which

students choose an answer from one of

several options.) Typically, a block will

contain about 16–18 questions, but there is

considerable variation depending on the

balance between multiple-choice and

constructed-response questions. Overall,

more than 50 percent of student assessment

time was devoted to the latter question
CHAPTER 1 • GEOGRAPHY REPORT CARD 7

type. In addition, of the time reserved for

constructed-response questions approximately 20 percent was used for “production” questions in which students engaged

in such tasks as indicating place locations

on outline maps, drawing routes between

points on a map, and drawing maps and

diagrams based upon written descriptions.

Two types of constructed-response questions were used:

■ short-constructed-response questions

that required students to provide brief

written answers of one or two sentences

or complete a limited production task;

and

■ extended-constructed-response questions that required students to provide

answers of a paragraph or more in length

or engage in an extensive production

task like producing a map.

Examples of multiple-choice, short- and

extended-constructed-response and production questions are provided in chapter

6. Additional information about the design

of the 2001 geography assessment is presented in appendix A.

Description of School

and Student Samples

The NAEP 2001 geography assessment

included representative samples of both

public and nonpublic schools. For the

reporting sample, approximately 7,000

fourth-graders, 9,000 eighth-graders, and

9,000 twelfth-graders were assessed. The

number of schools in the reporting sample

were 365 at fourth grade, 369 at eighth

grade, and 374 at twelfth grade. Each

selected school that participated in the

assessment and each student assessed

represent a portion of the population of

interest. For additional information on

sample sizes and participation rates, see

appendix A.

This report contains two different sets of

national results based on two reporting

samples that differed in terms of whether

or not accommodations were made available to special-needs students. The national

results presented in chapters 2, 3, 4, and 6

of this report are based on a nationally

representative sample that included specialneeds students only if they could be assessed meaningfully without accommodations. These results can be compared to

those from 1994 because accommodations

were also not made available in that assessment year. Chapter 5 presents a second set

of national results from 2001 for a representative sample that includes the performance of students who required and were

provided with accommodations (e.g.,

bilingual dictionary, extended time, small

group testing). No comparison of these

results to those from 1994 can be made

because of the inclusion of these accommodated special-needs students.

In the sample that did not permit accommodations, 8 percent of fourth-graders,

8 percent of eighth-graders, and 5 percent

of twelfth-graders were excluded from the

geography assessment in 2001. School staff

familiar with these students made the

determination, based upon NAEP’s inclusion criteria, that these students could not

be assessed meaningfully without accommodations because of their disability and/

or limited English proficiency. In 1994, 5

percent at both the fourth- and eighthgrades, and 3 percent at the twelfth-grade

were excluded. Additional information

regarding exclusion rates is provided in

appendix A.
8 CHAPTER 1 • GEOGRAPHY REPORT CARD

Reporting the

Assessment Results

Student performance on the NAEP geography assessment is presented in two ways:

as average scores on the NAEP geography

scale, and in terms of the percentage of

students attaining NAEP geography

achievement levels. The average scale

scores are a measure of what students know

and can do in geography. The achievementlevel results indicate the degree to which

students’ performance meets expectations of

what they should know and be able to do.

Average scale score results are presented

on the NAEP geography composite scale,

which ranges from 0–500. Students’ responses on the NAEP 2001 geography

assessment were analyzed to determine the

percentages of students that responded

correctly to each multiple-choice question

and the percentages of students that responded at each score level for the constructed-response questions. Scales that

summarize results for each of the three

content areas described earlier were created. The composite scale is a weighted

average of the separate subscales for the

three content areas. The weight for each

content area corresponds to its relative

importance as prescribed in the NAEP

geography framework. A full description of

NAEP scale procedures can be found in

the forthcoming NAEP 2001 Technical

Report.

Achievement-level results are presented

in terms of geography achievement levels

7 No Child Left Behind Act of 2001: Reauthorization of the Elementary and Secondary Education Act. Pub. L. No.

107-110 (H.R. 1).

National Assessment of Educational Progress Improvement Act of 1988. Pub. L. No. 100-297, 20, U.S.C. 1211.

8 National Assessment of Educational Progress Improvement Act of 1988. Pub. L. No. 100-297, 20, U.S.C. 1211.

9 No Child Left Behind Act of 2001: Reauthorization of the Elementary and Secondary Education Act. Pub. L. No.

107-110 (H.R. 1).

as authorized by the NAEP legislation and

adopted by the National Assessment

Governing Board (NAGB).7 For each

grade tested, NAGB has adopted three

achievement levels: Basic, Proficient, and

Advanced. For reporting purposes, the

achievement-level cut scores are placed on

the geography scale, resulting in four

ranges: below Basic, Basic, Proficient, and

Advanced.

The Setting of

Achievement Levels

The 1988 NAEP legislation that created

the National Assessment Governing Board

directed the Board to identify “appropriate

achievement goals...for each subject area”

that NAEP measures.8 The 2001 NAEP

reauthorization reaffirmed many of the

Board’s statutory responsibilities, including

developing “appropriate student achievement levels for each grade or age in each

subject area to be tested . . . ”9 To follow

this directive and achieve the mandate of

the 1988 statute to “improve the form and

use of NAEP results,” NAGB undertook

the development of student performance

standards called “achievement levels.” Since

1990 the Board has adopted achievement

levels in mathematics, reading, U.S. history,

geography, science, writing, and civics.

The Board defined three levels for each

grade: Basic, Proficient, and Advanced. The

Basic level denotes partial mastery of the

knowledge and skills that are fundamental

for proficient work at a given grade. The

Proficient level represents solid academic
CHAPTER 1 • GEOGRAPHY REPORT CARD 9

SOURCE: National Assessment Governing Board.

This level denotes partial mastery of prerequisite knowledge and skills that are

fundamental for proficient work at each grade.

This level represents solid academic performance for each grade assessed. Students

reaching this level have demonstrated competency over challenging subject matter,

including subject-matter knowledge, application of such knowledge to real-world

situations, and analytical skills appropriate to the subject matter.

This level signifies superior performance.

Figure 1.3

Achievement Levels

Basic

Proficient

Advanced

Policy definitions of the three NAEP achievement levels

performance. Students reaching this level

demonstrate competency over challenging

subject matter. The Advanced level presumes mastery of both the Basic and Proficient levels and superior performance.

Figure 1.3 presents the policy definitions of

the achievement levels that apply across all

grades and subject areas. The policy definitions guided the development of the

geography achievement levels, as well as

the achievement levels established in all

other subject areas. Adopting three levels

of achievement for each grade signals the

importance of looking at more than one

standard of performance. The Board

believes, however, that all students should

reach the Proficient level: the Basic level is

not the desired goal, but rather represents

partial mastery that is a step toward Proficient.

The achievement levels in this report

were adopted by the Board based on a

standard-setting process designed and

conducted under a contract with ACT, Inc.

To develop these levels, ACT convened a

cross section of educators and interested

citizens from across the nation and asked

them to judge what students should know

and be able to do relative to a body of

content reflected in the NAEP framework

for geography. This achievement-levelsetting process was reviewed by a variety of

individuals including policymakers, representatives of professional organizations,

teachers, parents, and other members of the

general public. Prior to adopting these

levels of student achievement, NAGB

engaged a large number of persons to

comment on the recommended levels and

to review the results.

The results of the achievement-levelsetting process, after NAGB’s approval,

became a set of achievement-level descriptions and a set of achievement-level cut

points on the 0–500 NAEP geography

scale. The cut points are the scores that

define the boundaries between below

Basic, Basic, Proficient, and Advanced performance at grades 4, 8, and 12. The Board

established these geography achievement

levels based upon the geography content

framework.
10 C H A P T E R 1 • GEOGRAPHY REPORT CARD

Achievement-Level

Descriptions for Each Grade

Specific definitions of the Basic, Proficient,

and Advanced geography achievement levels

for grades 4, 8, and 12 are presented in

figures 1.4 through 1.6. As noted previously, the achievement levels are cumulative. Therefore, students performing at the

Proficient level also display the competencies

associated with the Basic level, and students

at the Advanced level also demonstrate the

skills and knowledge associated with both

the Basic and the Proficient levels. For each

achievement level listed in figures 1.4

through 1.6, the scale score that corresponds to the beginning of that level is

shown in parentheses. For example, in

figure 1.4 the scale score of 240 corresponds to the beginning of the grade 4

Proficient level of achievement.

Figure 1.4

Achievement

Levels

Descriptions of NAEP geography achievement levels for grade 4

SOURCE: National Assessment Governing Board, Geography Framework for the 1994 and 2001 National Assessment of Educational Progress.

Basic Students should be able to use words or diagrams to define basic geography vocabulary;

(187) identify personal behaviors and perspectives related to the environment, and describe some

environmental and cultural issues in their community; use visual and technology tools to

access information; identify major geographic features on maps and globes; be able to read

and draw simple maps, map keys, and legends; demonstrate how people depend upon, use,

and adapt to the environment; and give examples of the movement of people, goods, services,

and ideas from one place to another. In addition to demonstrating an understanding of how

individuals are alike and different, they should demonstrate a knowledge of the ways people

depend on each other.

Proficient Students should be able to use fundamental geographic knowledge and vocabulary to identify

(240) basic geographic patterns and processes; describe an environmental or cultural issue from

more than one perspective; and read and interpret information from visual and technological

tools such as photograph maps and globes, aerial photography, and satellite images. They

should be able to use number and letter grids to plot specific locations; understand relative

location terms; and sketch simple maps and describe and/or draw landscapes they have

observed or studied. Proficient students should be able to illustrate how people depend upon,

adapt to, and modify the environment, describe and/or illustrate geographic aspects of a

region using fundamental geographic vocabulary and give reasons for current human

migration; discuss the impact a location has upon cultural similarities and differences; and

be able to demonstrate how an event in one location can have an impact upon another

location.

Advanced Students should be able to use basic geographic knowledge and vocabulary to describe global

(276) patterns and processes; describe ways individuals can protect and enhance environmental

quality; describe how modifications to the environment may have a variety of consequences;

explain differing perspectives that apply to local environmental or cultural issues; and

demonstrate an understanding of forces that result in migration, changing demographics,

and boundary changes. They should be able to solve simple problems by applying information

learned through working with visual and technological tools such as aerial and other

photographs, maps and globes, atlases, news media, and computers. They should be able to

construct models and sketch and label maps of their own state, the United States, and the

world; use them to describe and compare differences, similarities, and patterns of change in

landscapes; and be able to predict the impact a change one location can have on another.

They should be able to analyze the ways individuals and groups interact.
CHAPTER 1 • G E O G R A P H Y R E P O R T C A R D 11

Figure 1.5

Achievement

Levels

Descriptions of NAEP geography achievement levels for grade 8

SOURCE: National Assessment Governing Board, Geography Framework for the 1994 and 2001 National Assessment of Educational Progress.

Basic Students should possess fundamental knowledge and vocabulary of concepts relating to

(242) patterns, relationships, distance, directions, scale, boundary, site, and situation; solve

fundamental locational questions using latitude and longitude; interpret simple map scales;

identify continents and their physical features, oceans, and various cities; respond accurately

to descriptive questions using information obtained by use of visual and technological tools

such as geographic models and/or translate that information into words; explain differences

between maps and globes; and find a wide range of information using an atlas or almanac.

Students should be able to recognize and illustrate the relationships that exist between

humans and their environments, and provide evidence showing how physical habitat can

influence human activity. They should be able to define a region and identify its distinguishing

characteristics. Finally, they should be able to demonstrate how the interaction that takes place

between and among regions is related to the movement of people, goods, services, and ideas.

Proficient Students should possess a fundamental geographic vocabulary; understand geography’s

(282) analytical concepts; solve locational questions requiring integration of information from two

or more sources, such as atlases or globes; compare information presented at different

scales; and identify a wide variety of physical and cultural features and describe regional

patterns. Students should be able to respond accurately to interpretive questions using

geography’s visual and technological tools and translate that information into patterns;

identify differences in map projections and select proper projections for various purposes; and

develop a case study working with geography’s analytical concepts. In addition, students

should be able to describe the physical and cultural characteristics of places; explain how

places change due to human activity; and explain and illustrate how the concept of regions

can be used as a strategy for organizing and understanding Earth’s surface. Students should

be able to analyze and interpret data bases and case studies, as well as use information from

maps to describe the role that regions play in influencing trade and migration patterns and

cultural and political interaction.

Advanced Students should have a command of extensive geographic knowledge, analytical concepts,

(315) and vocabulary; be able to analyze spatial phenomena using a variety of sources with

information presented at a variety of scales and show relationships between them; and use

case studies for special analysis and to develop maps and other graphics. Students should be

able to identify patterns of climate, vegetation, and population across Earth’s surface and

interpret relationships between and among these patterns, and use one category of a map or

aerial photograph to predict other features of a place such as vegetation based on climate or

population density based on topographic features. Students should also be able to relate the

concept of region to specific places and explain how regions change over time due to a variety

of factors. They should be able to profile a region of their own design using geographic

concepts, tools, and skills.
12 C H A P T E R 1 • GEOGRAPHY REPORT CARD

Figure 1.6

Achievement

Levels

Descriptions of NAEP geography achievement levels for grade 12

SOURCE: National Assessment Governing Board, Geography Framework for the 1994 and 2001 National Assessment of Educational Progress.

Basic Students should possess a knowledge of concepts and terms commonly used in physical and

(270) human geography as well as skills enabling them to employ applicable units of measurement

and scale when solving simple locational problems using maps and globes. They should be

able to read maps; provide examples of plains, plateaus, hills, and mountains; and locate

continents, major bodies of water, and selected countries and cities. They should be able to

interpret geographic data and use visual and technological tools such as charts, tables,

cartograms, and graphs; know the nature of and be able to identify several basic types of

map projection; understand the basic physical structure of the planet; explain and apply

concepts such as continental drift and plate tectonics; and describe geography’s analytical

concepts using case studies. Students should have a comprehensive understanding of spatial

relationships including the ability to recognize patterns that exist across Earth in terms of

phenomena, including climate regions, time zones, population distributions, availability of

resources, vegetation zones, and transportation and communication networks. They should be

able to develop data bases about specific places and provide a simple analysis about their

importance.

Proficient Students should have an extensive understanding and knowledge of the concepts and

(305) terminology of physical and human geography. They should be able to use geographic

concepts to analyze spatial phenomena and to discuss economic, political, and social factors

that define and interpret space. They should be able to do this through the interpretation of

maps and other visual and technological tools, through the analysis of case studies, the

utilization of data bases, and the selection of appropriate research materials. Students

should be able to design their own maps based on descriptive data; describe the physical and

cultural attributes of major world regions; relate the spatial distribution of population to

economic and environmental factors; and report both historical and contemporary events

within a geographic framework using tools such as special purpose maps, and primary and

secondary source materials.

Advanced Students should possess a comprehensive understanding of geographic knowledge and

(339) concepts; apply this knowledge to case studies; formulate hypotheses and test geographic

models that demonstrate complex relationships between physical and human phenomena;

apply a wide range of map skills; develop maps using fundamental cartographic principles

including translating narratives about places and events into graphic representations, and

use other visual and technological tools to perform locational analysis and interpret spatial

relationships. Students should also be able to undertake sophisticated analysis from aerial

photographs or satellite imagery and other visuals. Advanced students should be able to

develop criteria assessing issues relating to human spatial organization and environmental

stability and, through research skills and the application of critical thinking strategies,

identify alternative solutions. They should be able to compile data bases from disparate

pieces of information and from these data bases develop generalizations and speculations

about outcomes when data change.
CHAPTER 1 • G E O G R A P H Y R E P O R T C A R D 13

10 No Child Left Behind Act of 2001: Reauthorization of the Elementary and Secondary Education Act. Pub. L. No.

107-110 (H.R. 1).

11 United States General Accounting Office. (1993). Education achievement standards: NAGB’s approach yields misleading

interpretations. U.S. General Accounting Office Report to Congressional Requestors. Washington, DC: Author.

National Academy of Education. (1993). Setting performance standards for achievement: A report of the National Academy

of Education Panel on the evaluations of the NAEP Trial State Assessment: An evaluation of the 1992 achievement levels.

Stanford, CA: Author.

12 Cizek, G. (1993). Reactions to National Academy of Education report. Washington, DC: National Assessment Governing

Board.

Kane, M. (1993). Comments on the NAE evaluation of the NAGB achievement levels. Washington, DC: National

Assessment Governing Board.

13 American College Testing. (1995). NAEP reading revisited: An evaluation of the 1992 achievement level descriptions.

Washington, DC: National Assessment Governing Board.

14 National Academy of Education. (1996). Reading achievement levels. In Quality and utility: The 1994 Trial State

Assessment in reading. The fourth report of the National Academy of Education Panel on the evaluation of the NAEP Trial

State Assessment. Stanford, CA: Author.

15 National Academy of Education. (1997). Assessment in transition: Monitoring the nation’s educational progress (p. 99).

Mountain View, CA: Author.

The Trial Status of

Achievement Levels

The 2001 NAEP reauthorization law

requires that the achievement levels be

used on a trial basis until the Commissioner of Education Statistics determines

that the achievement levels are “reasonable,

valid, and informative to the public.”10

Until that determination is made, the law

requires the Commissioner and the Board

to state clearly the trial status of the

achievement levels in all NAEP reports.

In 1993, the first of several congressionally mandated evaluations of the achievement-level-setting process concluded that

the procedures used to set the achievement

levels were flawed and that the percentage

of students at or above any particular

achievement-level cutpoint may be underestimated.11 Others have critiqued these

evaluations, asserting that the weight of the

empirical evidence does not support such

conclusions.12

In response to the evaluations and

critiques, NAGB conducted an additional

study of the 1992 reading achievement

levels before deciding to use those reading

achievement levels for reporting 1994

NAEP results.13 When reviewing the

findings of this study, the National Academy of Education (NAE) Panel expressed

concern about what it saw as a “confirmatory bias” in the study and about the

inability of this study to “address the panel’s

perception that the levels had been set too

high.”14 In 1997, the NAE Panel summarized its concerns with interpreting NAEP

results based on the achievement levels as

follows:

First, the potential instability of the levels

may interfere with the accurate portrayal of

trends. Second, the perception that few American

students are attaining the higher standards we

have set for them may deflect attention to the

wrong aspects of education reform. The public has

indicated its interest in benchmarking against

international standards, yet it is noteworthy that

when American students performed very well on

a 1991 international reading assessment, these

results were discounted because they were

contradicted by poor performance against the

possibly flawed NAEP reading achievement

levels in the following year.15

The National Center for Education

Statistics and the National Assessment

Governing Board have sought and con-
14 C H A P T E R 1 • GEOGRAPHY REPORT CARD

tinue to seek new and better ways to set

performance standards on NAEP.16 For

example, NCES and NAGB jointly sponsored a national conference on standard

setting in large-scale assessments, which

explored many issues related to standard

setting.17 Although new directions were

presented and discussed, a proven alternative to the current process has not yet been

identified. The Deputy Commissioner of

Education Statistics and the Board continue to call on the research community to

assist in finding ways to improve standard

setting for reporting NAEP results.

The most recent congressionally mandated evaluation conducted by the National Academy of Sciences (NAS) relied

on prior studies of achievement levels,

rather than carrying out new evaluations,

on the grounds that the process has not

changed substantially since the initial

problems were identified. Instead, the NAS

Panel studied the development of the 1996

science achievement levels. The NAS Panel

basically concurred with earlier congressionally mandated studies. The Panel

concluded that “NAEP’s current achievement-level-setting procedures remain

fundamentally flawed. The judgment tasks

are difficult and confusing; raters’ judgments of different item types are internally

inconsistent; appropriate validity evidence

for the cut scores is lacking; and the process

has produced unreasonable results.”18

The NAS Panel accepted the continuing

use of achievement levels in reporting

NAEP results on a developmental basis,

until such time as better procedures can be

developed. Specifically, the NAS Panel

concluded that “....tracking changes in the

percentages of students performing at or

above those cut scores (or, in fact, any

selected cut scores) can be of use in describing changes in student performance

over time.”19

The National Assessment Governing

Board urges all who are concerned about

student performance levels to recognize

that the use of these achievement levels is a

developing process and is subject to various

interpretations. The Board and the Deputy

Commissioner believe that the achievement levels are useful for reporting trends

in the educational achievement of students

in the United States.20 In fact, achievementlevel results have been used in reports by

the President of the United States, the

Secretary of Education, state governors,

legislators, and members of Congress.

Government leaders in the nation and in

more than 40 states use these results in

their annual reports.

16 Reckase, Mark, D. (2000). The evolution of the NAEP achievement levels setting process: A summary of the research and

development efforts conducted by ACT. Iowa City, IA: ACT, Inc.

17 National Assessment Governing Board and National Center for Education Statistics. (1995). Proceedings of the joint

conference on standard setting for large-scale assessments of the National Assessment Governing Board (NAGB) and the

National Center for Education Statistics (NCES). Washington, DC: Government Printing Office.

18 Pellegrino, J.W., Jones, L.R., & Mitchell, K.J. (Eds.). (1998). Grading the nation’s report card: evaluating NAEP and

transforming the assessment of educational progress. Committee on the Evaluation of National Assessments of Educational Progress, National Research Council. (p.182). Washington, DC: National Academy Press.

19 Ibid., page 176.

20 Forsyth, Robert A. (2000). A description of the standard-setting procedures used by three standardized test

publishers. In Student performance standards on the National Assessment of Educational Progress: Affirmations and

improvements. Washington, DC: National Assessment Governing Board.

Nellhaus, Jeffrey M. (2000). States with NAEP-like performance standards. In Student performance standards on the

National Assessment of Educational Progress: Affirmations and improvements. Washington, DC: National Assessment

Governing Board.
CHAPTER 1 • G E O G R A P H Y R E P O R T C A R D 15

However, based on the congressionally

mandated evaluations so far, the Deputy

Commissioner agrees with the National

Academy’s recommendation that caution

needs to be exercised in the use of the

current achievement levels. Therefore, the

Deputy Commissioner concludes that

these achievement levels should continue

to be used on a trial basis and should

continue to be interpreted with caution.

Interpreting NAEP Results

The average scores and percentages presented in this report are estimates based on

samples of students rather than on entire

populations. Moreover, the collection of

questions used at each grade level is but a

sample of the many questions that could

have been asked to assess student knowledge of the framework content. As such,

the results are subject to a measure of

uncertainty, reflected in the standard error

of the estimates—a range of a few points

plus or minus the score—which accounts

for potential score fluctuation due to

sampling error and measurement error.

The standard errors for the estimated scale

scores and percentages in this report are

provided in appendix B.

The differences between scale scores and

between percentages discussed in the

following chapters take into account the

standard errors associated with the estimates. Comparisons are based on statistical

tests that consider both the magnitude of

the difference between the group average

scores or percentages and the standard

errors of those statistics. Estimates based on

smaller subgroups are likely to have relatively large standard errors. As a consequence, some seemingly large differences

may not be statistically significant. When

this is the case, the term “apparent differences” is used in this report. Throughout

this report, differences between scores or

between percentages are pointed out only

when they are significant from a statistical

perspective. All differences reported are

significant at the 0.05 level with appropriate adjustments for multiple comparisons.

The term “significant” identifies statistically

dependable population differences to help

inform dialogue among policymakers,

educators, and the public.

Readers are cautioned against interpreting NAEP results in a causal sense. Inferences related to student subgroup performance or to the effectiveness of public and

nonpublic schools, for example, should take

into consideration the many socioeconomic and educational factors that may

also affect performance in geography.

Overview of the

Remaining Report

The results in chapters 2, 4 and 6 of this

report are based on the set of data with no

accommodations offered to students.

Findings are presented for the nation and

for all the major reporting subgroups

included in all NAEP report cards. Comparisons with results from the 1994 assessment are noted where the data permit.

Chapter 4 examines contexts for learning

geography in terms of classroom practices

and student variables.
16 C H A P T E R 1 • GEOGRAPHY REPORT CARD

NAEP has sought to assess samples that

are as inclusive as possible. Nevertheless,

there has always been some exclusion of

students with disabilities (SD) and limited

English proficient (LEP) students who

could not be assessed meaningfully without

accommodations. Local school officials

have made decisions about exclusion in

accordance with explicit criteria provided

by NAEP. In order to expand the proportion of students who can be assessed

meaningfully, NAEP began in recent

assessments to explore the use of accommodations with special-needs students.

Chapter 5 presents an overview of a second

set of results—those that include students

who were provided accommodations

during the test administration. By including

these results in the nation’s geography

report card, NAEP continues a phased

transition toward a more inclusive reporting sample. Future assessment results will

be based solely on a student and school

sample in which accommodations are

permitted.

Chapter 6 provides sample assessment

questions and student responses from the

2001 assessment. Also presented in chapter

6 are item maps that position selected

question descriptions along the NAEP

geography scale where they are likely to be

answered successfully by students. The

descriptions used on these item maps focus

on the geography skills or knowledge

needed to answer the question. The data

presented in both chapters 4 and 6 are

based on the set of results that did not

include accommodated special-needs

students.

This report also contains appendices that

support or augment the results presented.

Appendix A contains an overview of the

NAEP geography framework and specifications, information on the national sample,

and a more detailed description of the

major reporting subgroups featured in

chapters 2 and 3. Appendix B contains the

full data with standard errors for all tables

and figures in this report. Appendix C

contains a list of the NAEP geography

committee members.
CHAPTER 2 • G E O G R A P H Y R E P O R T C A R D 17

2Average Scale Score and

Achievement-Level Results for the Nation

Overview

This chapter presents the NAEP 2001 geography assessment

results for the nation at grades 4, 8, and 12. Student

performance is described by average scale scores on the

NAEP geography composite scale, which ranges from 0 to

500, and in terms of percentages of students who attained

each of the three geography achievement levels: Basic,

Proficient, and Advanced. Results of the NAEP 2001

geography assessment are compared with results from

the NAEP geography assessment given in 1994. This

comparison is possible because the assessments share a

common set of geography exercises based on the

current geography framework and because the

populations of students were sampled and assessed

using comparable procedures. The results for this

chapter are based on testing conditions comparable to

those offered in 1994 when accommodations for

special-needs students were not offered. Special-needs

students who could participate without

accommodations were included. A second set of

results were obtained in 2001 that includes the

performance of students who required and were provided

accommodations. Results for the 2001 assessment that

include special-needs students tested with accommodations

are presented in chapter 5.

Are the nation’s

fourth-, eighth-,

and twelfthgraders making

progress in

geography?

Chapter

Focus

Chapter

Contents

Overview

Average Scale

Score Results

Scale Scores by

Percentile

AchievementLevel Results
18 C H A P T E R 2 • GEOGRAPHY REPORT CARD

Average Scale Score Results

The results of the 2001 geography assessment show higher average scores than the

results in 1994 at grades 4 and 8, and no

statistically significant change at grade 12.

As seen in figure 2.1, the average score of

fourth-graders rose from 206 to 209, and

the average score of eighth-graders rose

from 260 to 262.

Figure 2.1 Average geography scale scores, grades 4, 8, and 12: 1994 and 2001

National Scale Score

Results

 Significantly different from 1994.

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of

Educational Progress (NAEP), 1994 and 2001 Geography Assessments.

500 

290 

280 

270 

260 

1994 2001

260

285

262

285

250 

240 

230 

220 

210 

200 

0 

206 209

Grade 12

Grade 8

Grade 4
CHAPTER 2 • G E O G R A P H Y R E P O R T C A R D 19

Scale Scores by Percentile

An examination of percentile scores provides additional information about student

performance across the score distribution.

The percentile indicates the percentage of

students whose scores fell below a particular point on the NAEP geography scale.

The advantage of viewing percentile scores

is that they show how students with lower

or higher ability performed compared to

the national average. In addition, the

percentile data show whether trends in the

national average scores are reflected in

scores at other levels of the performance

distribution. Figure 2.2 shows the geography scale scores for grades 4, 8, and 12 at

the 10th, 25th, 50th, 75th, and 90th percentiles for the 1994 and 2001 assessments.

 Significantly different from 1994.

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of

Educational Progress (NAEP), 1994 and 2001 Geography Assessments.

Figure 2.2

National Performance

Distribution

Geography scale score percentiles, grades 4, 8, and 12: 1994 and 2001

270 

260 

250 

240 

230 

220 

210 

0 

244

265

247

267

500 

330 

320 

310 

300 

290 

280 

1994 2001

287

306

321

287

305

319

250 

240 

230 

220 

210 

200 

190 

0 

213

237

217

241

500 

310 

300 

290 

280 

270 

260 

1994 2001

263

285

302

265

286

303

200 

190 

180 

170 

160 

150 

140 

0 

146

179

158

185

500 

260 

250 

240 

230 

220 

210 

1994 2001

211

237

257

212

236

254

Grade 4 Grade 8 Grade 12 Percentiles

90th

75th

50th

25th

10th

Percentiles

90th

75th

50th

25th

10th

Percentiles

90th

75th

50th

25th

10th
20 C H A P T E R 2 • GEOGRAPHY REPORT CARD

At grades 4 and 8, scores at the two

lowest percentiles (10th and 25th) were

higher in 2001 than in 1994, suggesting

that much of the improvement seen at

grades 4 and 8 was concentrated among

the lower-performing students. Other

apparent changes at these two grades were

not statistically significant. At grade 12,

consistent with national average score

results, none of the apparent differences in

percentile scores was statistically significant.

Achievement-Level Results

The results of student performance are not

only reported using scores on the NAEP

geography scale, but also using geography

achievement levels. The achievement levels

are performance standards adopted by the

National Assessment Governing Board

(NAGB), based on the collective judgments

of experts about what students should be

expected to know and be able to do.

Viewing student performance from this

perspective provides some insight into the

adequacy of students’ knowledge and skills

and the extent to which they achieved

expected levels of performance. A discussion of the trial status of achievement levels

is in chapter 1.

Figure 2.3 presents achievement-level

results for grades 4, 8, and 12. The results

are shown in two ways: 1) the percentage

of students within each achievement-level

interval, and 2) the percentage of students

at or above the Basic level and at or above

the Proficient level. The text that follows

discusses significant differences at or above

Basic and Proficient, which are marked with

* in the figures. Differences within

achievement levels are not discussed although they are shown in the figures. In

reading figure 2.3, it is necessary to keep in

mind that the percentages at or above

specific achievement levels are cumulative.

Included among the percentage of students

at or above the Basic level are also those

who have achieved the Proficient and

Advanced levels of performance, and included among students at or above the

Proficient level are also those who have

attained the Advanced level of performance.
CHAPTER 2 • G E O G R A P H Y R E P O R T C A R D 21

Figure 2.3

National AchievementLevel Results

Percentage of students within and at or above geography achievement levels,

grades 4, 8, and 12: 1994 and 2001

 Significantly different from 1994.

NOTE: Percentages within each geography achievement level may not add to 100, or to the exact percentages at or above achievement levels, due to rounding.

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of

Educational Progress (NAEP), 1994 and 2001 Geography Assessments.

3%

19% 22%

48%

70%

30%

1994

2%

19% 21%

53%

74%

26%

2001

Grade 4

Advanced

At or above

Proficient

At or above

Basic

Proficient

Basic

Below

Basic

4%

24% 28%

43%

71%

29%

1994

4%

26%

30%

44%

74%

26%

2001

Advanced 

Proficient 

Grade 8

Below

Basic

Basic

At or above

Proficient

At or above

Basic

2%

25%

27%

43%

70%

30%

1994

1%

23% 25%

47%

71%

29%

2001

Grade 12

Advanced

At or above

Proficient

At or above

Basic

Proficient

Basic

Below

Basic
22 C H A P T E R 2 • GEOGRAPHY REPORT CARD

The NAGB has identified the Proficient

achievement level as the minimum level at

which all students should perform. In 2001,

21 percent of fourth-graders, 30 percent of

eighth-graders, and 25 percent of twelfthgraders were at or above the Proficient level.

Across years, the improvement in performance seen in the fourth- and eighthgrade average scale scores is reflected in

achievement-level performance. Both

grades show an increase from 1994 to 2001

in the percentage of students at or above

Basic and a decrease in the percentage of

students below Basic. As in 1994, only a

small percentage of students at any grade

reached the Advanced level in 2001: 2

percent at fourth grade, 4 percent at eighth

grade, and 1 percent at twelfth grade.
CHAPTER 3 • G E O G R A P H Y R E P O R T C A R D 23

3

Chapter

Contents

Gender

Race/Ethnicity

Region of the

Country

Parents’

Education

Type of School

Type of Location

Eligibility for

Free/ReducedPrice School

Lunch Program

Are selected

subgroups of

students making

progress in

geography?

Chapter

Focus

Average Scale Scores and AchievementLevel Results for Selected Subgroups

In addition to reporting on the performance of all students,

NAEP also provides results for various subgroups of students

at each grade. Examining subgroup results provides insight,

not only into how these groups of students performed in

comparison to one another, but also into how each group

has progressed over time. In light of recent educational

reform efforts that focus on improving the achievement of

all students, the information presented in this chapter

serves as a valuable indicator on the nation’s progress

in meeting its educational goals.

Results for the NAEP 2001 geography

assessment are presented by gender, race/ethnicity,

region of the country, parents’ highest level of

education, type of school, type of location, students’

eligibility for the Free/Reduced-Price School Lunch

program. For all subgroups except type of location

and free/reduced-price school lunch eligibility,

results of the 2001 assessment are compared with

those of the 1994 assessment.

Differences reported in this chapter between

demographic subgroups for the 2001 assessment and

between the 2001 and 1994 assessments are based on

statistical tests that consider both the magnitude of the

difference between the group average scores or percentages

and the standard errors of those statistics. Differences
24 C H A P T E R 3 • GEOGRAPHY REPORT CARD

Figure 3.1 Average geography scale scores by gender, grades 4, 8, and 12: 1994 and 2001

National Scale Score

Results by Gender

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of

Educational Progress (NAEP), 1994 and 2001 Geography Assessments.

500 

250 

240 

230 

220 

210 

200 

190 

0 

1994 2001

208

203

212

207

Grade 4 Grade 8 Grade 12

500 

280 

270 

260 

250 

240 

230 

220 

0 

1994 2001

262

258

264

260

500 

290 

280 

270 

260 

250 

240 

230 

0 

1994 2001

288

281

287

282

Male

Female

between groups and between assessment

years are discussed only if they have been

determined to be statistically significant.

Throughout this chapter, differences

between 1994 and 2001 are marked in the

figures. Differences within 2001 are not

marked, but where such differences are

discussed in the text, they are statistically

significant. Furthermore, the reader should

bear in mind that differences in performance among subgroups of students may

reflect a range of socioeconomic and

educational factors not addressed in this

report or by NAEP.

Gender

Figure 3.1 presents average geography

scores for male and female students across

assessment years. There was no statistically

significant change from 1994 to 2001 in

the average scores of either male or female

students at any of the three grades. Although the scale score differences across

years by gender were similar to the change

across years for the population as a whole,

the smaller sample sizes and the generally

larger standard errors in the two subgroups

prevented the results of the statistical tests

from reaching the “significant” level in
CHAPTER 3 • G E O G R A P H Y R E P O R T C A R D 25

NOTE: Score differences are calculated based on differences between unrounded average scale scores.

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of

Educational Progress (NAEP), 1994 and 2001 Geography Assessments.

Figure 3.2

National Scale Score

Differences by Gender

Differences in average geography scale scores by gender, grades 4, 8, and 12:

1994 and 2001

0 20 30 40 50 10

2001

1994

Score Differences

0 20 30 40 50 10

2001

1994

Score Differences

5 4 7

5 4 4

0 20 30 40 50 10

2001

1994

Score Differences

Male–Female

Grade 4 Grade 8 Grade 12

these comparisons. In 2001, male students

at all three grades had higher average scores

than female students, just as they had in 1994.

The performance of subgroups on the

geography assessment can also be compared

by determining if a difference or “gap”

exists between subgroups’ average scores

and, if it does, whether that gap increases or

decreases between assessment years. Figure

3.2 shows that there was no statistically

significant change since 1994 in the differences between the average scores of male

and female students at any of the three

grades.

The percentages of male and female

students at or above the geography

achievement levels and within each

achievement-level range are presented in

figure 3.3. None of the apparent changes

across years in the percentages of male and

female students at or above Basic and at or

above Proficient were statistically significant.

Looking at the differences in performance of male and female students in 2001

shows higher percentages of fourth- and

eighth-grade male students at or above

Proficient and at Advanced than their female

counterparts. Among twelfth-graders, a

higher percentage of male students than

female students were at or above the Basic

and Proficient levels.
26 C H A P T E R 3 • GEOGRAPHY REPORT CARD

Figure 3.3

National AchievementLevel Results by Gender

Percentage of students within and at or above geography achievement levels by

gender, grades 4, 8, and 12: 1994 and 2001

 Significantly different from 1994.

NOTE: Percentages within each geography achievement-level range may not add to 100, or to the exact percentages at or above achievement levels, due to rounding.

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of

Educational Progress (NAEP), 1994 and 2001 Geography Assessments.

Grade 4

Grade 12

Grade 8

4%

22% 26%

46%

71%

29%

1994

3%

21% 24%

51%

75%

25%

2001

Male

Advanced

At or above

Proficient

At or above

Basic

Proficient

Basic

Below

Basic 25%

2%

17% 19%

49%

68%

32%

1994

1%

17% 18%

54%

72%

28%

2001

Female

Advanced

At or above

Proficient

At or above

Basic

Proficient

Basic

Below

Basic

5%

25%

30%

42%

72%

28%

1994

5%

29%

33%

42%

75%

25%

2001

Male

Advanced

At or above

Proficient

At or above

Basic

Proficient

Basic

Below

Basic

3%

22% 25%

44%

69%

31%

1994

3%

24% 26%

47%

73%

27%

2001

Female

Advanced

At or above

Proficient

At or above

Basic

Proficient

Basic

Below

Basic

2%

29%

32%

41%

73%

27%

1994

2%

26%

28%

45%

73%

27%

2001

Male

Advanced

At or above

Proficient

At or above

Basic

Proficient

Basic

Below

Basic

1%

21% 22%

45%

67%

33%

1994

1%

20% 21%

48%

70%

30%

2001

Female

Advanced

At or above

Proficient

At or above

Basic

Proficient

Basic

Below

Basic
CHAPTER 3 • G E O G R A P H Y R E P O R T C A R D 27

Figure 3.4

National Scale Score

Results by Race/Ethnicity

Average geography scale scores by race/ethnicity, grades 4, 8, and 12:

1994 and 2001

 Significantly different from 1994.

NOTE: Sample size was insufficient to permit a reliable estimate for American Indian students at grade 12 in 1994.

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of

Educational Progress (NAEP), 1994 and 2001 Geography Assessments.

500 

220 

210 

200 

190 

180 

170 

160 

0 

1994 2001

218

183

214

168

193

184

212

181

199

222

500 

280 

270 

260 

250 

240 

230 

220 

0 

1994 2001

270

239

264

229

248

273

240

266

234

261

500 

300 

290 

280 

270 

260 

250 

240 

0 

1994 2001

291

268

285

258

291

270

286

260

288

White

Hispanic

Asian/Pacific

Islander

Black

American Indian

Grade 4 Grade 8 Grade 12

Race/Ethnicity

The background questionnaire administered with the NAEP geography assessment asked students to indicate the racial/

ethnic subgroup that best described them.

The mutually exclusive subgroup categories were White, Black, Hispanic, Asian/

Pacific Islander, and American Indian

(including Alaska Native). Figure 3.4 shows

the average scores for the five subgroups at

grades 4, 8, and 12. Only the results from

the 2001 assessment are reported for

twelfth-grade American Indian students

because the sample size in 1994 was

insufficient to permit a reliable estimate.

At grade 4, the average score of Black

students increased from 168 in 1994 to 181

in 2001. There were no other statistically

significant changes in average scores among

the five racial/ethnic groups. The significance of the apparent gains for American

Indian students at grades 4 and 8 could not

be determined because of insufficient

sample sizes.

Scale score differences among subgroups

were evident at all three grades in 2001. At

fourth grade, White students had higher

scores, on average, than students from the

other four racial/ethnic groups. Asian/

Pacific Islander students outperformed

Black, Hispanic, and American Indian

students, and American Indian students

performed better than Black and Hispanic

students.

At grade 8, White students had higher

average scores than Black, Hispanic, and

Asian/Pacific Islander students. Asian/

Pacific Islander and American Indian
28 C H A P T E R 3 • GEOGRAPHY REPORT CARD

 Significantly different from 1994.

NOTE: Score differences are calculated based on differences between unrounded average scale scores.

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of

Educational Progress (NAEP), 1994 and 2001 Geography Assessments.

Figure 3.5

National Scale Score

Differences by Race/

Ethnicity

Differences in average geography scale scores by race/ethnicity, grades 4, 8,

and 12: 1994 and 2001

0 20 30 40 50 10

2001

1994

Score Differences

0 20 30 40 50 10

2001

1994

Score Differences

50 40 33

0 20 30 40 50 10

2001

1994

Score Differences

White–Black

Grade 4 Grade 8 Grade 12

0 20 30 40 50 10

2001

1994

Score Differences

0 20 30 40 50 10

2001

1994

Score Differences

35 31 23

38 33 21

0 20 30 40 50 10

2001

1994

Score Differences

White–Hispanic

Grade 4 Grade 8 Grade 12

40 38 32

students outperformed Black and Hispanic

students. Hispanic students had higher

average scores than Black students.

Among twelfth-graders, White students,

Asian/Pacific Islander students, and

American Indian students had higher

average scores than Black students and

Hispanic students, and Hispanic students

outperformed Black students. These

differences should, however, be interpreted

with caution. The average score of a

selected subgroup does not represent the

entire range of performance within that

group. Furthermore, differences between

groups of students cannot be attributed

solely to group identification. A complex

array of educational and social factors

interacts to affect average student

performance.

Score differences between White

students and Black students and between

White students and Hispanic students are

presented in figure 3.5. Results from the

2001 geography assessment show a

narrowing of the score difference between White students and Black students

at grade 4. Other apparent changes were

not statistically significant.
CHAPTER 3 • G E O G R A P H Y R E P O R T C A R D 29

Achievement-level results for the racial/

ethnic subgroups are presented in figures

3.6a, b, and c. At grade 4, the percentages

of Black students and White students

performing at or above the Basic level were

higher in 2001 than in 1994. At grade 8,

the percentage of White students at or

above Basic was higher in 2001. There was

no statistically significant change in the

percentage of twelfth-grade students at or

above the Basic and Proficient levels among

any of the racial/ethnic groups.

Comparing the performance of subgroups in 2001 shows higher percentages

of White and Asian/Pacific Islander

students at or above the Basic and Proficient

levels than Black and Hispanic students at

all three grades.
30 C H A P T E R 3 • GEOGRAPHY REPORT CARD

Figure 3.6a

National AchievementLevel Results by Race/

Ethnicity

Percentage of students within and at or above geography achievement levels

by race/ethnicity, grade 4: 1994 and 2001

 Significantly different from 1994.

# Percentage is between 0.0 and 0.5.

NOTE: Percentages within each geography achievement-level range may not add to 100, or to the exact percentages at or above achievement levels, due to rounding.

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of

Educational Progress (NAEP), 1994 and 2001 Geography Assessments.

4%

25% 29%

53%

81%

19%

1994

3%

26%

29%

58%

87%

13%

2001

White

Advanced

At or above

Proficient

At or above

Basic

Proficient

Basic

Below

Basic

(2%) 3%

32%

34%

66%

1994

(5%) 5%

39%

44%

56%

2001

Black

Advanced

At or above

Proficient

At or above

Basic

Proficient

Basic

Below

Basic

# #

1%

(9%) 10%

39%

49%

51%

1994

(6%) 6%

43%

49%

51%

2001

Hispanic

Advanced At or above

Proficient

At or above

Basic

Proficient

Basic

Below

Basic

# 4%

23% 27%

49%

76%

24%

1994

1%

23% 25%

52%

77%

23%

2001

Asian/Pacific Islander

Advanced

At or above

Proficient

At or above

Basic

Proficient

Basic

Below

Basic

(9%) 9%

53%

62%

38%

1994

13% 13%

53%

66%

34%

2001

American Indian

Advanced

At or above

Proficient

At or above

Basic

Proficient

Basic

Below

Basic

# #
CHAPTER 3 • G E O G R A P H Y R E P O R T C A R D 31

Figure 3.6b

National AchievementLevel Results by Race/

Ethnicity

Percentage of students within and at or above geography achievement levels

by race/ethnicity, grade 8: 1994 and 2001

 Significantly different from 1994.

# Percentage is between 0.0 and 0.5.

NOTE: Percentages within each geography achievement-level range may not add to 100, or to the exact percentages at or above achievement levels, due to rounding.

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of

Educational Progress (NAEP), 1994 and 2001 Geography Assessments.

5%

30%

36%

47%

82%

18%

1994

5%

34%

39%

48%

86%

14%

2001

White

Advanced

At or above

Proficient

At or above

Basic

Proficient

Basic

Below

Basic

(4%) 5%

30%

34%

66%

1994

(6%) 6%

34%

40%

60%

2001

Black

Advanced At or above

Proficient

At or above

Basic

Proficient

Basic

Below

Basic

# #

1%

(10%) 10%

39%

50%

50%

1994

1%

(9%) 10%

38%

48%

52%

2001

Hispanic

Advanced At or above

Proficient

At or above

Basic

Proficient

Basic

Below

Basic

6%

25%

30%

43%

73%

27%

1994

4%

28%

32%

47%

79%

21%

2001

Asian/Pacific Islander

Advanced

At or above

Proficient

At or above

Basic

Proficient

Basic

Below

Basic

2%

13% 15%

43%

59%

41%

1994

3%

29%

31%

41%

72%

28%

2001

American Indian

Advanced

At or above

Proficient

At or above

Basic

Proficient

Basic

Below

Basic
32 C H A P T E R 3 • GEOGRAPHY REPORT CARD

Figure 3.6c

National AchievementLevel Results by Race/

Ethnicity

Percentage of students within and at or above geography achievement levels

by race/ethnicity, grade 12: 1994 and 2001

Significantly different from 1994.

# Percentage is between 0.0 and 0.5.

Sample size was insufficient to permit a reliable estimate for American Indian students at grade 12 in 1994.

NOTE: Percentages within each geography achievement-level range may not add to 100, or to the exact percentages at or above achievement levels, due to rounding.

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of

Educational Progress (NAEP), 1994 and 2001 Geography Assessments.

2%

31%

33%

46%

78%

22%

1994

2%

29%

31%

51%

81%

19%

2001

White

Advanced

At or above

Proficient

At or above

Basic

Proficient

Basic

Below

Basic

(5%) 5%

27%

32%

68%

1994

(4%) 4%

31%

35%

65%

2001

Black

Advanced At or above

Proficient

At or above

Basic

Proficient

Basic

Below

Basic

# #

(10%) 10%

38%

48%

52%

1994

(10%) 10%

42%

52%

48%

2001

Hispanic

Advanced At or above

Proficient

At or above

Basic

Proficient

Basic

Below

Basic

# # 3%

25%

28%

41%

69%

31%

1994

1%

25%

26%

45%

72%

28%

2001

Asian/Pacific Islander

Advanced

At or above

Proficient

At or above

Basic

Proficient

Basic

Below

Basic

1%

31%

32%

41%

74%

26%

2001

American Indian

Advanced

At or above

Proficient

At or above

Basic

Proficient

Basic

Below

Basic

1994

No

data

reported
CHAPTER 3 • G E O G R A P H Y R E P O R T C A R D 33

 Significantly different from 1994.

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of

Educational Progress (NAEP), 1994 and 2001 Geography Assessments.

Figure 3.7

National Scale Score

Results by Region of

the Country

Average geography scale scores by region of the country, grades 4, 8, and 12:

1994 and 2001

500 

250 

240 

230 

220 

210 

200 

190 

0 

1994 2001

203

200

215

205

214

207

219

200

Northeast

Southeast

Central

West

500 

280 

270 

260 

250 

240 

230 

220 

0 

1994 2001

266

252

268

255

266

260

270

255

500 

290 

280 

270 

260 

250 

240 

230 

0 

1994 2001

284

278

289

286 286

281

287

283

Grade 4 Grade 8 Grade 12

Region of the Country

NAEP assessments traditionally provide

results for four regions of the country:

Northeast, Southeast, Central, and West.

Appendix A (see page 137) contains a

description of the states and other jurisdictions that make up each region.

Figure 3.7 shows scale score results by

region of the country. Overall increases in

average scores at grades 4 and 8 were not

spread evenly across the four regions of the

country. At grade 4, students in the Northeast region showed a gain in the average

score between 1994 and 2001. At grade 8,

students in the Southeast region showed a

statistically significant increase. None of the

other apparent changes between 1994 and

2001 in regional average scores were

statistically significant.

In 2001, some differences in performance among regions are evident at all

three grades. At grades 4 and 8, students in

the Northeast and Central regions had

higher average scores than their counterparts in the West. Students in the Central

region outperformed students in the

Southeast. Twelfth-graders in the Central

region had higher average scores than

twelfth-graders in the Southeast region.
34 C H A P T E R 3 • GEOGRAPHY REPORT CARD

Achievement-level results for the four

regions appear in figures 3.8a, b, and c. The

percentage of fourth-grade students from

the West performing at or above the Proficient level decreased between 1994 and

2001. Over the same period, the percentage of both eighth- and twelfth-graders in

the Southeast performing at or above the

Basic level increased.

Figures 3.8a, b, and c also show a number of differences in achievement-level

performance among the four regions for

the year 2001. At grades 4 and 8, there

were higher percentages of students in the

Northeast region and the Central region at

or above Basic and at or above Proficient

than in the West. At all three grades, the

percentage of students at or above Basic and

at or above Proficient was higher in the

Central region than in the Southeast.

 Significantly different from 1994.

NOTE: Percentages within each geography achievement-level range may not add to 100, or to the exact percentages at or above achievement levels, due to rounding.

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of

Educational Progress (NAEP), 1994 and 2001 Geography Assessments.

Figure 3.8a

National AchievementLevel Results by

Region of the Country

Percentage of students within and at or above geography achievement levels

by region of the country, grade 4: 1994 and 2001

3%

19% 22%

45%

67%

33%

1994

3%

22% 24%

54%

78%

22%

2001

Northeast

Advanced

At or above

Proficient

At or above

Basic

Proficient

Basic

Below

Basic

2%

14% 17%

48%

64%

36%

1994

1%

17% 18%

54%

72%

28%

2001

Southeast

Advanced

At or above

Proficient

At or above

Basic

Proficient

Basic

Below

Basic

3%

19% 21%

48%

70%

30%

1994

1%

13% 14%

52%

66%

34%

2001

West

Advanced

At or above

Proficient

At or above

Basic

Proficient

Basic

Below

Basic

4%

25%

28%

49%

78%

22%

1994

3%

27%

30%

51%

82%

18%

2001

Central

Advanced

At or above

Proficient

At or above

Basic

Proficient

Basic

Below

Basic
CHAPTER 3 • G E O G R A P H Y R E P O R T C A R D 35

Figure 3.8b

National AchievementLevel Results by

Region of the Country

Percentage of students within and at or above geography achievement levels

by region of the country, grade 8: 1994 and 2001

 Significantly different from 1994.

NOTE: Percentages within each geography achievement-level range may not add to 100, or to the exact percentages at or above achievement levels, due to rounding.

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of

Educational Progress (NAEP), 1994 and 2001 Geography Assessments.

6%

28%

33%

43%

76%

24%

1994

4%

29%

34%

44%

78%

22%

2001

Northeast

Advanced

At or above

Proficient

At or above

Basic

Proficient

Basic

Below

Basic

3%

19% 21%

40%

62%

38%

1994

3%

24% 26%

46%

73%

27%

2001

Southeast

Advanced

At or above

Proficient

At or above

Basic

Proficient

Basic

Below

Basic

6%

30%

36%

44%

80%

20%

1994

6%

32%

38%

43%

82%

18%

2001

Central

Advanced

At or above

Proficient

At or above

Basic

Proficient

Basic

Below

Basic

3%

20% 23%

45%

67%

33%

1994

2%

21% 23%

44%

66%

34%

2001

West

Advanced

At or above

Proficient

At or above

Basic

Proficient

Basic

Below

Basic
36 C H A P T E R 3 • GEOGRAPHY REPORT CARD

Figure 3.8c

National AchievementLevel Results by

Region of the Country

Percentage of students within and at or above geography achievement levels

by region of the country, grade 12: 1994 and 2001

 Significantly different from 1994.

NOTE: Percentages within each geography achievement-level range may not add to 100, or to the exact percentages at or above achievement levels, due to rounding.

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of

Educational Progress (NAEP), 1994 and 2001 Geography Assessments.

2%

23% 25%

44%

69%

31%

1994

2%

24% 26%

46%

71%

29%

2001

Northeast

Advanced

At or above

Proficient

At or above

Basic

Proficient

Basic

Below

Basic

1%

19% 20%

41%

60%

40%

1994

1%

20% 21%

46%

67%

33%

2001

Southeast

Advanced

At or above

Proficient

At or above

Basic

Proficient

Basic

Below

Basic

2%

30%

32%

43%

75%

25%

1994

1%

27%

28%

48%

76%

24%

2001

Central

Advanced

At or above

Proficient

At or above

Basic

Proficient

Basic

Below

Basic

2%

27%

29%

43%

72%

28%

1994

1%

22% 23%

47%

70%

30%

2001

West

Advanced

At or above

Proficient

At or above

Basic

Proficient

Basic

Below

Basic

Parents’ Highest Level

of Education

Eighth- and twelfth-grade students who

participated in the NAEP geography

assessment were asked to indicate the

highest level of education completed by

each parent. Students chose from among

four options: did not finish high school,

graduated from high school, some education after high school, and graduated from

college. Students could also choose the

response, “I don’t know.” The analysis here

uses the highest education level reported

for either parent. Data were not collected

at grade 4 because in previous NAEP

assessments fourth-graders’ responses about

their parents’ education were unreliable and

contained a large percentage of “I don’t

know” responses.
CHAPTER 3 • G E O G R A P H Y R E P O R T C A R D 37

Figure 3.9

National Scale Score

Results by Parents’

Education

Average geography scale scores by parents’ highest level of education,

grades 8 and 12: 1994 and 2001

 Significantly different from 1994.

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of

Educational Progress (NAEP), 1994 and 2001 Geography Assessments.

500 

290 

280 

270 

260 

250 

240 

230 

0 

1994 2001

238

250

265

272

234

241

253

266

274

245

Less than high school

Graduated high school

Some education after high school

Graduated college

Unknown

Grade 8 Grade 12

500 

300 

290 

280 

270 

260 

250 

240 

0 

1994 2001

263

274

286

294

257

269

276

284

293

257

The scale score results for studentreported parent education level appear in

figure 3.9. In 2001, nearly one-half of all

students at both grades reported that at

least one of their parents graduated from

college (48 percent at grade 8 and 46

percent at grade 12). In the case of grade 8,

that percentage is a statistically significant

increase over 1994. Only a small percentage of students at either grade reported that

their parents had not graduated from high

school (6 percent at grade 8 and 7 percent

at grade 12). Additional information

on the percentage of students reporting

parents’ highest level of education is available in appendix B.

Twelfth-graders who reported that their

parents had not graduated from high

school had higher average scores in 2001

than in 1994. The results for both grades

in 2001 reveal a pattern similar to that from

the 1994 geography assessment and from

other NAEP assessments. Overall there is a

positive relationship between studentreported parental education and student

achievement: the higher the parental

education level reported, the higher the

average score.
38 C H A P T E R 3 • GEOGRAPHY REPORT CARD

Achievement-level results by level of

parental education are presented in figures

3.10a and b. As with the average scale

score results, the 2001 achievement-level

results show a general pattern of higher

percentages of eighth- and twelfth-grade

students at or above the Basic and Proficient

levels as the reported level of parental

education level increased. The one exception to the pattern was at grade 12, where

the percentage of students at or above

Proficient did not differ significantly between those students who reported that

their parents graduated high school and

those who reported their parents did not.
CHAPTER 3 • G E O G R A P H Y R E P O R T C A R D 39

Figure 3.10a

National AchievementLevel Results by

Parents’ Education

Percentage of students within and at or above geography achievement levels

by parents’ highest level of education, grade 8: 1994 and 2001

 Significantly different from 1994.

# Percentage is between 0.0 and 0.5.

NOTE: Percentages within each geography achievement-level range may not add to 100, or to the exact percentages at or above achievement levels, due to rounding.

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of

Educational Progress (NAEP), 1994 and 2001 Geography Assessments.

1%

(8%) 8%

39%

47%

53%

1994

(8%) 8%

41%

48%

52%

2001

Less than high school

Advanced At or above

Proficient

At or above

Basic

Proficient

Basic

Below

Basic

# 1%

14% 15%

47%

62%

38%

1994

1%

16% 18%

48%

66%

34%

2001

Graduated high school

Advanced

At or above

Proficient

At or above

Basic

Proficient

Basic

Below

Basic

3%

26%

29%

50%

79%

21%

1994

2%

27%

30%

51%

80%

20%

2001

Some education after high school

Advanced

At or above

Proficient

At or above

Basic

Proficient

Basic

Below

Basic

7%

34%

41%

41%

82%

18%

1994

6%

37%

43%

42%

86%

14%

2001

Graduated college

Advanced

At or above

Proficient

At or above

Basic

Proficient

Basic

Below

Basic

1%

(8%) 8%

35%

44%

56%

1994

1%

(11%) 12%

44%

56%

44%

2001

Unknown

Advanced

At or above

Proficient

At or above

Basic

Proficient

Basic

Below

Basic
40 C H A P T E R 3 • GEOGRAPHY REPORT CARD

Figure 3.10b

National AchievementLevel Results by

Parents’ Education

Percentage of students within and at or above geography achievement levels

by parents’ highest level of education, grade 12: 1994 and 2001

 Significantly different from 1994.

# Percentage is between 0.0 and 0.5.

NOTE: Percentages within each geography achievement-level range may not add to 100, or to the exact percentages at or above achievement levels, due to rounding.

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of

Educational Progress (NAEP), 1994 and 2001 Geography Assessments.

0%

(7%) 7%

34%

41%

59%

1994

(10%) 10%

38%

48%

52%

2001

Less than high school

Advanced

At or above

Proficient

At or above

Basic

Proficient

Basic

Below

Basic

#

13% 14%

42%

56%

44%

1994

12% 12%

50%

62%

38%

2001

Graduated high school

Advanced

At or above

Proficient

At or above

Basic

Proficient

Basic

Below

Basic

# #

1%

23% 24%

51%

75%

25%

1994

1%

20% 21%

52%

73%

27%

2001

Some education after high school

Advanced

At or above

Proficient

At or above

Basic

Proficient

Basic

Below

Basic

3%

37%

40%

41%

81%

19%

1994

3%

34%

36%

46%

82%

18%

2001

Graduated college

Advanced

At or above

Proficient

At or above

Basic

Proficient

Basic

Below

Basic

0%

(7%) 7%

29%

36%

64%

1994

0%

(5%) 5%

28%

33%

67%

2001

Unknown

Advanced At or above

Proficient

At or above

Basic

Proficient

Basic

Below

Basic
CHAPTER 3 • G E O G R A P H Y R E P O R T C A R D 41

Type of School

The schools that participate in the NAEP

assessment are classified as either public or

nonpublic. A further distinction is then

made within the nonpublic classification

between nonpublic Catholic schools and

other nonpublic schools. In 2001, as in

previous NAEP assessments, fourth-,

eighth-, and twelfth-grade students attending nonpublic schools had higher average

scores than did their public school peers.

However, readers are cautioned against

making assumptions about the relative

quality of public- and nonpublic-school

instruction based on these findings. Socioeconomic and sociological factors that may

affect student performance should also be

considered when interpreting the results.

Figure 3.11 shows average geography

scores by type of school. At grade 8, public

school students had higher average scores

in 2001 than in 1994. All other apparent

differences across years were not statistically

significant. The data for performance in

2001 reveal that, at all three grades, students

from Catholic schools had higher average

scores than students from public schools.

The apparent differences between students

in public schools and other nonpublic

schools were not statistically different.
42 C H A P T E R 3 • GEOGRAPHY REPORT CARD

500 

250 

240 

230 

220 

210 

200 

190 

0 

1994 2001

204

221

207

226

500 

250 

240 

230 

220 

210 

200 

190 

0 

1994 2001

222

220

230

221

Grade 4

Public

Nonpublic Nonpublic: Other

Nonpublic: Catholic

500 

290 

280 

270 

260 

250 

240 

230 

0 

1994 2001

258

276

261

274

Grade 8

Grade 12

500 

310 

300 

290 

280 

270 

260 

250 

0 

1994 2001

283

294

284

291

500 

290 

280 

270 

260 

250 

240 

230 

0 

1994 2001

277

271

500 

310 

300 

290 

280 

270 

260 

250 

0 

1994 2001

291

298 294

287

 Significantly different from 1994.

NOTE: Italicized scale score values indicate that two or more groups had the same rounded average score. The average scores, when rounded, were the same

for Nonpublic: Catholic and Nonpublic: Other at grade 8 in 1994.

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of

Educational Progress (NAEP), 1994 and 2001 Geography Assessments.

Figure 3.11

National Scale Score

Results by Type of

School

Average geography scale scores by type of school, grades 4, 8, and 12:

1994 and 2001
CHAPTER 3 • G E O G R A P H Y R E P O R T C A R D 43

Achievement-level results by type of

school are presented in figures 3.12a, b, and

c. At grade 4, the percentage of Catholic

school students performing at or above the

Basic level increased between 1994 and

2001. At grade 8, the percentage of public

school students at or above the Basic level

increased across years.

In 2001 there were a number of differences in achievement-level performance

between subgroups. At all three grades,

there were higher percentages of nonpublic

school students and, more specifically,

Catholic school students at or above the

Basic and Proficient levels than public school

students.

Figure 3.12a

National AchievementLevel Results by Type

of School

Percentage of students within and at or above geography achievement levels

by type of school, grade 4: 1994 and 2001

 Significantly different from 1994.

NOTE: Percentages within each geography achievement-level range may not add to 100, or to the exact percentages at or above achievement levels, due to rounding.

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of

Educational Progress (NAEP), 1994 and 2001 Geography Assessments.

3%

19% 21%

47%

68%

32%

1994

2%

18% 20%

52%

72%

28%

2001

Public

Advanced

At or above

Proficient

At or above

Basic

Proficient

Basic

Below

Basic

5%

26%

30%

53%

84%

16%

1994

3%

28%

31%

59%

91%

2001

(9%)

Nonpublic

Advanced

At or above

Proficient

At or above

Basic

Proficient

Basic

Below

Basic

5%

(7% )

25%

30%

54%

85%

15%

1994

4%

32%

36%

56%

93%

2001

Nonpublic: Catholic

Advanced

At or above

Proficient

At or above

Basic

Proficient

Basic

Below

Basic

4%

26%

30%

52%

82%

18%

1994

2%

23% 25%

63%

88%

12%

2001

Nonpublic: Other

Advanced

At or above

Proficient

At or above

Basic

Proficient

Basic

Below

Basic
44 C H A P T E R 3 • GEOGRAPHY REPORT CARD

Figure 3.12b

National AchievementLevel Results by Type

of School

Percentage of students within and at or above geography achievement levels

by type of school, grade 8: 1994 and 2001

 Significantly different from 1994.

NOTE: Percentages within each geography achievement-level range may not add to 100, or to the exact percentages at or above achievement levels, due to rounding.

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of

Educational Progress (NAEP), 1994 and 2001 Geography Assessments.

4%

22% 26%

43%

69%

31%

1994

3%

25%

28%

44%

72%

28%

2001

Public

Advanced

At or above

Proficient

At or above

Basic

Proficient

Basic

Below

Basic

8%

36%

44%

43%

87%

13%

1994

5%

36%

41%

46%

87%

13%

2001

Nonpublic

Advanced

At or above

Proficient

At or above

Basic

Proficient

Basic

Below

Basic

8%

35%

44%

45%

89%

11%

1994

6%

40%

46%

44%

89%

11%

2001

Nonpublic: Catholic

Advanced

At or above

Proficient

At or above

Basic

Proficient

Basic

Below

Basic

7%

38%

45%

41%

86%

14%

1994

4%

33%

37%

48%

85%

15%

2001

Nonpublic: Other

Advanced

At or above

Proficient

At or above

Basic

Proficient

Basic

Below

Basic
CHAPTER 3 • G E O G R A P H Y R E P O R T C A R D 45

Figure 3.12c

National AchievementLevel Results by Type

of School

Percentage of students within and at or above geography achievement levels

by type of school, grade 12: 1994 and 2001

 Significantly different from 1994.

NOTE: Percentages within each geography achievement-level range may not add to 100, or to the exact percentages at or above achievement levels, due to rounding.

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of

Educational Progress (NAEP), 1994 and 2001 Geography Assessments.

1%

24% 26%

42%

68%

32%

1994

1%

23% 24%

47%

71%

29%

2001

Public

Advanced

At or above

Proficient

At or above

Basic

Proficient

Basic

Below

Basic

3%

33%

36%

47%

83%

17%

1994

2%

30%

32%

48%

80%

20%

2001

Nonpublic

Advanced

At or above

Proficient

At or above

Basic

Proficient

Basic

Below

Basic

1%

32%

33%

47%

80%

20%

1994

2%

32%

34%

51%

85%

15%

2001

Nonpublic: Catholic

Advanced

At or above

Proficient

At or above

Basic

Proficient

Basic

Below

Basic

5%

35%

40%

47%

87%

13%

1994

2%

26%

29%

44%

73%

27%

2001

Nonpublic: Other

Advanced

At or above

Proficient

At or above

Basic

Proficient

Basic

Below

Basic
46 C H A P T E R 3 • GEOGRAPHY REPORT CARD

Average geography scale scores by type of school location, grades 4, 8, and 12: 2001

Central city Urban fringe/large town Rural/small town

Table 3.1 National Scale Score Results by Type of Location

Grade 4 199 212 215

Grade 8 255 265 265

Grade 12 279 288 284

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of

Educational Progress (NAEP), 2001 Geography Assessment.

Type of Location

The schools from which NAEP draws its

samples of students are classified according

to their type of location. Based on Census

Bureau definitions of metropolitan statistical areas, including population size and

density, the three mutually exclusive categories are central city, rural/small town,

and urban fringe/large town. Because of

new methods used by NCES to identify

the type of location assigned to each school

in the Common Core of Data, schools

were not classified in exactly the same way

in 2001 as in 1994. Therefore, comparisons

between the two assessment years are not

possible, and only the data from the 2001

assessment are reported. More information

on the definitions of the 2001 assessment

classifications for location type appears in

appendix A.

The performance of students by type of

school location is shown in table 3.1. At all

three grades, students attending schools in

rural and urban fringe locations had higher

average scale scores than students in central

city schools.

Achievement-level results by type of

school location appear in figure 3.13. At

grade 4, higher percentages of urban fringe

and rural students performed at or above

Basic and at or above Proficient than did

their central city counterparts. There were

higher percentages of eighth-graders from

urban fringe and rural locations at or above

Basic than those from central city locations.

There were also higher percentages of

students at or above Proficient in urban

fringe locations than central city locations.

At grade 12, higher percentages of urban

fringe and rural students than central city

students performed at or above the Basic

level, and there were higher percentages of

students at or above Proficient in urban

fringe locations than in central city or rural

locations.
CHAPTER 3 • G E O G R A P H Y R E P O R T C A R D 47

Figure 3.13

National AchievementLevel Results by Type

of Location

Percentage of students within and at or above geography achievement levels

by type of school location, grades 4, 8, and 12: 2001

2%

14% 16%

46%

62%

38%

2%

21% 23%

53%

76%

24%

Central city Urban fringe/

large town

Grade 4

Rural/

small town

At or above

Proficient

At or above

Basic

Advanced

Basic

Proficient

Below

Basic

2%

21% 23%

58%

81%

19%

3%

22% 25%

39%

64%

36%

4%

29%

32%

45%

78%

22%

Central city Urban fringe/

large town

Grade 8

Rural/

small town

At or above

Proficient

At or above

Basic

Advanced

Basic

Proficient

Below

Basic

3%

27%

30%

48%

78%

22%

1%

19% 20%

42%

63%

37%

2%

28%

30%

45%

75%

25%

Central city Urban fringe/

large town

Grade 12

Rural/

small town

At or above

Proficient

At or above

Basic

Advanced

Basic

Proficient

Below

Basic

1%

21% 22%

52%

74%

26%

NOTE: Percentages within each geography achievement-level range may not add to 100, or to the exact percentages at or above achievement levels, due to rounding.

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of

Educational Progress (NAEP), 2001 Geography Assessment.
48 C H A P T E R 3 • GEOGRAPHY REPORT CARD

Table 3.2 National Scale Score Results by Free/Reduced-Price School Lunch Eligibility

Average geography scale scores by student eligibility for Free/Reduced-Price School Lunch

program, grades 4, 8, and 12: 2001

Eligible Not eligible Info not available

Grade 4 186 221 218

Grade 8 242 270 266

Grade 12 269 287 289

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of

Educational Progress (NAEP), 2001 Geography Assessment.

Free/Reduced-Price School Lunch

Program Eligibility

Funded by the U.S. Department of Agriculture (USDA) as part of the National

School Lunch Program, the Free/Reduced-Price School Lunch program is

designed to assure that children at or near

the poverty line receive nourishing meals.

Eligibility guidelines for the lunch program

are based on the federal income poverty

guidelines and are stated by household

size.1 NAEP first began collecting data on

student eligibility for this program in 1996;

therefore cross-year comparisons back to

1994 are not possible.

Table 3.2 presents the average scale score

results for grades 4, 8, and 12. The scores

for the substantial number of students for

whom eligibility information is not available appear in the “Info not available”

column (see the percentages for each

category in the table B.18). Students whose

schools do not participate in the Free/

Reduced-Price School Lunch program are

included in this category. At each grade,

students eligible for the Free/ReducedPrice School Lunch program (i.e., those

meeting the poverty guidelines) had lower

average scores than did ineligible students

and students for whom information was

not available.

Achievement-level results reflected the

scale score results, as seen in figure 3.14. At

all three grades, higher percentages of

students who were ineligible for the Free/

Reduced-Price School Lunch program or

for whom information was not available

were at or above Basic and at or above

Proficient than were students who were

eligible for the lunch program. At grade 8,

this pattern extended to the Advanced

achievement level.

1 U.S. General Services Administration. (2001). Catalog of federal domestic assistance. Washington, DC: Executive Office

of the President, Office of Management and Budget. http://www.cfda.gov/default.htm.
CHAPTER 3 • G E O G R A P H Y R E P O R T C A R D 49

# Percentage is between 0.0 and 0.5.

NOTE: Percentages within each geography achievement-level range may not add to 100, or to the exact percentages at or above achievement levels, due to rounding.

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of

Educational Progress (NAEP), 2001 Geography Assessment.

Figure 3.14

National AchievementLevel Results by Free/

Reduced-Price School

Lunch Program Eligibility

Percentage of students within and at or above geography achievement levels by

Free/Reduced-Price School Lunch program eligibility, grades 4, 8, and 12: 2001

(6%) 6%

45%

51%

49%

3%

27%

29%

56%

86%

14%

Eligible Not eligible

Advanced

At or above

Proficient

At or above

Basic

Proficient

Basic

Below

Basic

Grade 4

Info not

available

3%

24% 27%

57%

84%

16%

#

1%

(10%) 11%

39%

50%

50%

5%

32%

37%

46%

83%

17%

Eligible Not eligible

Advanced

At or above

Proficient

At or above

Basic

Proficient

Basic

Below

Basic

Grade 8

Info not

available

4%

29%

33%

46%

79%

21%

(10%) 11%

40%

51%

49%

1%

25%

26%

49%

75%

25%

Eligible Not eligible

Advanced

Proficient

Basic

Below

Basic

Grade 12

Info not

available

# 2%

29%

31%

45%

76%

24%

At or above

Proficient

At or above

Basic

CHAPTER 4 • G E O G R A P H Y R E P O R T C A R D 51

4

Chapter

Contents

Teacher

Background and

Preparedness

Geography Skills

Taught

Extent of

Geography

Instruction

Computer Use

Student Interest

in Geography

How does teacher

preparedness

relate to students’

geography

performance?

How do classroom

activities and

computer use

relate to student

achievement?

Chapter

Focus

Classroom Contexts for Learning

This chapter examines teacher and student variables related

to the teaching and learning of geography, such as teacher

preparedness and classroom practices, student interest in

geography, and the context in which students learn the

concepts of geography. The results presented in this chapter

are based on teachers’ and students’ responses to

questionnaires administered as part of the NAEP 2001

geography assessment. NAEP administers

background and instructional questionnaires to

students at grades 4, 8, and 12 and to the teachers of

participating fourth- and eighth-grade students.

Teachers at grade 12 were not administered a

questionnaire because of the difficulty of linking

students to teachers across the diversity of courses at

this grade level. In this examination of contexts for

learning, students are the unit of analysis. Thus, for

questions answered by students, the percentage of

students choosing each response option is presented.

For questions answered by teachers, the percentage of

students whose teachers chose each option is

presented. Students’ average NAEP geography scores

for each response are also presented in order to examine the

relationship between each variable and students’ geography

performance. Readers are reminded that the relationship

between a contextual variable and geography performance is

not necessarily causal. Many factors contribute to student

performance. NAEP data can identify relationships between

contextual variables and student performance, but cannot

explain why the relationships exist.
52 C H A P T E R 4 • GEOGRAPHY REPORT CARD

1 Geography Education Standards Project. (1994). Geography for life: national geography standards. Washington, DC:

National Geographic Research and Exploration.

2 Gregg, M. (2001). River views of beginning pre-service teachers: content knowledge use. Journal of Geography

100, 61–68.

Brophy, J. (1991). Advances in research on teaching. (Vol. 2) Teacher’s knowledge of subject matter as it relates to their

teaching practice. Greenwich, CT: JAI Press.

Reynolds, M. C., Ed. (1989). Knowledge base for the beginning teacher. Oxford: Pergamon Press.

3 Rynne, E. (1997) The continuing mismatch between student’s undergraduate experience and the teaching

demands of the geography classroom: experience of pre-service secondary geography teachers. Journal of

Geography in Higher Education 21, 65–77.

Teacher Background and

Preparedness to Teach Geography

Competency in geography is positively

associated with extent of geography education and begins with well-prepared teachers.1 Teachers of geography in grades 4, 8,

and 12 approach instruction with varied

backgrounds in geography or social studies,

which necessarily results in diverse learning

experiences for their students. Research on

teaching and learning shows that the extent

of a teacher’s knowledge of a subject can

have a marked effect on the quality of

teaching or even the willingness to teach

that subject.2 A number of educators are

concerned that poorly trained geography

teachers—those teaching “out of field”—

may lack the knowledge or skills to provide

students with the knowledge necessary to

meet geography standards.3 Thus, it is

instructive to explore the educational

backgrounds of the teachers who are

currently teaching geography to the

nation’s fourth- and eighth-grade students.

As part of the NAEP 2001 assessment,

teachers of participating students in grades

4 and 8 were asked about their undergraduate and graduate majors and minors.

The NAEP teacher questionnaire gave

fourth- and eighth-grade teachers a number of different majors/minors from which

to choose. Table 4.1 shows results for five

majors/minors asked of elementary teachers and four majors/minors asked of

eighth-grade teachers. Both the fourthand eighth-grade teachers were asked if

they had a major or minor in geography or

geography education, history or history

education, general social science or social

studies education, or other social science

(for example, political science, economics,

sociology, psychology, anthropology).

Fourth-grade teachers were also asked

whether they had a major or minor in

elementary education. Although teachers

were asked separately about their undergraduate and graduate education, and

about whether they had majored or minored in each subject, the data are presented here in a simplified form. The first

column in table 4.1 shows the percentages

and average scores of students whose

geography teachers either majored or

minored or had a special emphasis in a

subject at either the undergraduate or

graduate level. The second column shows

the corresponding data for students whose

teachers did not indicate that major or

minor. Note that the columns can sum to

more than 100 percent because it is possible for college students to complete more

than one major or minor.
CHAPTER 4 • G E O G R A P H Y R E P O R T C A R D 53

At grade 4, nearly all students (93 percent)

had teachers who majored or minored in

elementary education in undergraduate or

graduate school. While 15 to 20 percent of

students were taught by teachers who had a

social studies or history major or minor,

only 7 percent of fourth-graders were

taught by teachers who had majored or

minored in geography. Average geography

scores for fourth-grade students taught by

teachers with an elementary education

major or minor were higher than those

taught by teachers who did not.

At grade 8, just over one-quarter (28

percent) of students were taught by teachers

with a graduate or undergraduate major or

minor in geography. Within this grade,

there was no statistically significant relationship between teachers’ major/minor and

their students’ NAEP geography scores.

Table 4.1

Percentage of students and average

geography scale scores by teachers’

reported undergraduate/graduate

major and minor/special emphasis,

grades 4 and 8: 2001

Grades4&8

Grade 4

Geography or geography education 7 93

204 210

History or history education 15 85

206 211

Social science or social studies education 20 80

208 210

Other social science 57 43

210 209

Elementary education 93 7

211 197

Grade 8

Geography or geography education 28 72

263 263

History or history education 71 29

263 261

Social science or social studies education 55 45

263 262

Other social science 51 49

261 264

The percentage of students is listed first with the corresponding average scale score presented below.

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of

Educational Progress (NAEP), 2001 Geography Assessment.

Yes No Did you have a major, minor, or special emphasis in any

of the following subjects as part of your undergraduate

or graduate course work?

Teachers’

Majors/Minors

At grade 4, students

whose teachers

majored in elementary education had

higher average

scores than those

whose teachers did

not.
54 C H A P T E R 4 • GEOGRAPHY REPORT CARD

Teachers of fourth- and eighth-grade

students participating in the geography

assessment were asked how prepared they

felt they were to teach geography. The

question was asked in both 1994 and 2001,

so cross-year comparisons are possible.

Table 4.2 shows that a higher percentage of

the fourth-grade students in 2001 had

teachers who reported they were very

prepared to teach geography (31 percent)

than did students in 1994 (23 percent).

These results may be associated with the

increase in workshops in geography offered

to teachers through various geographic

organizations, and an increase in the literature available to teachers related to teaching geographic concepts.4 In addition,

there has been a call for geographers as a

group, specifically college and university

faculty, to develop an open dialogue with

K-12 teachers that would aid in providing

elementary and high school teachers with

the teaching tools necessary for them to

enter the classroom with some degree of

confidence.5

In 2001, approximately 84 percent of

fourth-grade students were taught by

teachers who reported that they were very

prepared or adequately prepared to teach

geography. Only 1 percent of fourth-grade

students had teachers who reported they

were unprepared to teach geography.

Approximately 87 percent of eighth-grade

students had teachers who reported they

were at least adequately prepared to teach

geography, and only 2 percent had teachers

who felt unprepared. The level of teachers’

self-reported preparedness had no statistically significant relationship to students’

average geography scores.

4 Gibbs, G. (1999). Improving teaching, learning, and assessment. Journal of Geography in Higher Education 23, 147–

155.

McDougall, W. A. (2001). Why geography matters. American Educator 25,10–15.

McAlonan, S. Hotchkiss, H., Roark, K., Kenney, M., & Jackson, J. (2001) Making standards work! Geography. A

teacher’s guide to contextual learning: integrating academic content standards with career development and workplace competencies. Denver: Colorado State Department of Education.

5 Bettis, N. C. (2001). Assessment issues in geographic education for the twenty-first century. Journal of Geography

100, 172–174.

Welford, M. & Fouberg, E.H. (2000). Theory and research in geography education. Journal of Geography 99,

183–184.
CHAPTER 4 • G E O G R A P H Y R E P O R T C A R D 55

Grades4&8

Table 4.2

Percentage of students and average

geography scale scores by teachers’

reports on how well prepared they

felt they were to teach geography,

grades 4 and 8: 1994 and 2001

Teachers’

Preparedness to

Teach Geography

The percentage of

fourth-graders

whose teachers

reported they were

very prepared to

teach geography

increased between

1994 and 2001.

Grade 4

Very prepared 23 31 *

209 211

Adequately prepared 57 53

206 210

Somewhat prepared 18 15

207 206

Unprepared 2 1

200 209

Grade 8

Very prepared 36 44

260 263

Adequately prepared 48 43

262 262

Somewhat prepared 13 11

265 261

Unprepared 2 2

260 264

The percentage of students is listed first with the corresponding average scale score presented below.

* Significantly different from 1994.

NOTE: Percentages may not add to 100 due to rounding.

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of

Educational Progress (NAEP), 1994 and 2001 Geography Assessments.

1994 2001 Regardless or whether you are currently teaching the

topic, how well prepared do you feel you are to teach

geography at the elementary /middle school level?
56 C H A P T E R 4 • GEOGRAPHY REPORT CARD

Geography Skills and

Topics Taught in Grade 4

Many of the instructional guides and

standards for geography education

emphasize the importance of using maps

and globes as tools to visualize space and

gain a spatial perspective—necessary skills

for an understanding of and competency in

geography.6 A number of articles also

discuss the importance of exposing

children to cultural geography,and

environmental geography.7, 8 As part of the

1994 and 2001 NAEP geography

assessments, fourth-grade teachers were

asked about the frequency with which they

taught about maps and globes, natural

resources, foreign countries and cultures,

and environmental issues as a part of

geography instruction. The results are

displayed in table 4.3.

There were a few changes between 1994

and 2001 in the frequency of instruction

reported by teachers of fourth-grade

students in these four skill and topic areas.

The percentage of fourth-grade students

whose teachers reported they never taught

about natural resources increased from 9

percent in 1994 to 14 percent in 2001,

while the percentage of students whose

teachers reported teaching about natural

resources as frequently as once or twice

weekly decreased from 38 percent to 31

percent. These data may suggest that the

topic of natural resources has lost favor

during the past decade in teaching geography to fourth-graders.

Of the four skills and topics, maps and

globes were most frequently used during

geography instruction during 2001. Approximately three-quarters of fourth-grade

students received instruction about maps

and globes on at least a weekly basis,

according to their teachers. However, less

than one-half of the students had teachers

who included natural resources as part of

instruction once a week or more, and only

about one-quarter received instruction in

foreign countries and cultures and environmental issues at least once a week. Twentynine percent of students received no

instruction in foreign countries and cultures. Readers should be aware that teachers were asked only to indicate the frequency and not the total amount of time

they devoted to the skills and topics discussed here “as part of geography instruction.” Therefore, students may have received

more instruction in these four areas than is

readily apparent from the percentages

shown, though not necessarily from a

geographic perspective.

6 Oldakowski, R. K. (2001). Activities to develop a spatial perspective among students in introductory geography

courses. Journal of Geography 100, 243–250.

Thompson, G. (1999). I thought the world was flat, like the maps showed it! Social Education 63, 269–271.

Trifonoff, K. M. (1998). Introducing thematic maps in the primary grades. Social Studies and the Young Learner 11,

17–22.

7 Crampton, J. (1998). A regional geography class in a distributed learning environment. Journal of Higher Education

22, 417–423.

8 Graf, M. (2000). The world’s best places: classroom explorations in geography & environmental science. Portsmouth, NH:

Heinemann.

Steinberg, P. E. (1997). Political geography and the environment. Journal of Geography 96, 113–118.
CHAPTER 4 • G E O G R A P H Y R E P O R T C A R D 57

Grade 4

Table 4.3

Percentage of students and average

geography scale scores by teachers’

reports on frequency of instruction of

selected skills and topics, grade 4:

1994 and 2001

Using maps and globes

Almost every day 29 28

210 213

Once or twice a week 54 47

208 209

Once or twice a month 17 22

199 206

Never or hardly ever 1 3

*** 209

Natural resources

Almost every day 9 9

201 217

Once or twice a week 38 31 *

209 208

Once or twice a month 44 46

208 210

Never or hardly ever 9 14 *

198 208

Foreign countries and cultures

Almost every day 6 3

206 206

Once or twice a week 19 23

203 208

Once or twice a month 43 45

208 209

Never or hardly ever 32 29

209 212

Environmental issues

Almost every day 4 7

201 212

Once or twice a week 27 21

206 205

Once or twice a month 56 56

208 211

Never or hardly ever 13 16

208 211

The percentage of students is listed first with the corresponding average scale score presented below.

* Significantly different from 1994.

*** Sample size is insufficient to permit a reliable estimate.

NOTE: Percentages may not add to 100 due to rounding.

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of

Educational Progress (NAEP), 1994 and 2001 Geography Assessments.

1994 2001 How often do you teach the following skills and topics as

a part of geography instruction with this class?

Frequency of

Instruction in

Fourth-Grade

Geography Skills

and Topics

Natural resources

topics were taught

less frequently in

2001.
58 C H A P T E R 4 • GEOGRAPHY REPORT CARD

Geography Skills Taught in

Grades 8 and 12

Eighth-and twelfth-grade students participating in the 1994 and 2001 geography

assessments were asked about the frequency

with which they studied the following

skills and topics in school: using maps and

globes, natural resources (for example, oil,

forests, and water), countries and cultures,

and environmental issues (for example,

pollution and recycling). The results of

these questions are shown in

tables 4.4a and 4.4b.

At grade 8, students in 2001 reported

having devoted more time to two of the

four areas than did their counterparts in

1994. Approximately 46 percent of students reported using maps and globes at

least once or twice a week compared to 39

percent in 1994. At the same time, the

percentage that reported never using maps

and globes dropped from 28 percent to 21

percent. For the topic of countries and

cultures, approximately 63 percent of

students in 2001 said they studied it one or

two times per week or more compared to

52 percent in 1994, and the percentage that

reported never studying it fell from 20 to 13

percent.

The geography performance of eighthgraders varied somewhat depending on the

amount of time they spent studying each

topic, but in general, higher frequency of

study did not translate into significantly

higher performance. In the case of maps

and globes and natural resources, students

performed best when they studied the

topics once or twice a month. Students

who studied natural resources and environmental issues almost every day had lower

average scores than students who studied

these topics less frequently or never. Students who never studied countries and

cultures had lower average scores than

students who did study that topic.

Twelfth-graders’ reports indicated a

general increase between 1994 and 2001 in

the amount of time they spent studying the

four geography skills and topics listed in

the beginning of this section. The easiest

way to see this change among the data in

table 4.4b is to note that the percentage of

students responding “never” declined for

each of the four topics (from 40 percent to

36 percent for maps and globes, from 45

percent to 39 percent for natural resources,

from 28 percent to 19 percent for countries and cultures, and from 37 percent to

30 percent for environmental issues, respectively). But as these results also show, a

substantial percentage of twelfth-graders

still did not study these topics in 2001.

Again, as was seen at the eighth-grade

level, more instruction was not necessarily

linked to better student performance.

Students who reported studying maps and

globes and natural resources almost every

day had lower average scores than students

who reported less frequent study. The same

was true with the study of environmental

issues except that the performance of

students in the “almost every day” and

“never” categories did not differ significantly. The one instructional topic that did

have a somewhat positive relationship to

twelfth-graders’ geography scores was

countries and cultures. Students studying

countries and cultures at least once a

month outperformed those who never

studied countries and cultures.
CHAPTER 4 • G E O G R A P H Y R E P O R T C A R D 59

Grade 8

Table 4.4a

Percentage of students and average

geography scale scores by students’ reports

on frequency of instruction of selected skills

and topics, grade 8: 1994 and 2001

Using maps and globes

Almost every day 9 12 *

261 259

Once or twice a week 30 34 *

264 264

Once or twice a month 33 33

263 268

Never or hardly ever 28 21 *

253 258

Natural resources

Almost every day 9 9

251 249

Once or twice a week 21 24 *

259 262

Once or twice a month 36 35

265 269

Never or hardly ever 34 32

260 263

Countries and cultures

Almost every day 23 31 *

260 264

Once or twice a week 29 32 *

261 266

Once or twice a month 28 24 *

264 263

Never or hardly ever 20 13 *

256 254

Environmental issues

Almost every day 12 11

258 254

Once or twice a week 21 24 *

260 265

Once or twice a month 33 33

263 267

Never or hardly ever 34 32

260 262

1994 2001 How often have you studied the following geography skills

and topics in school?

The percentage of students is listed first with the corresponding average scale score presented below.

* Significantly different from 1994.

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of

Educational Progress (NAEP), 1994 and 2001 Geography Assessments.

Frequency of

Instruction in

Eighth-Grade

Geography Skills

and Topics

Students performed

best when instruction on these topics

was once or twice a

month.

Eighth-graders who

never studied

countries and

cultures had lower

scores than those

who did.
60 C H A P T E R 4 • GEOGRAPHY REPORT CARD

Grade 12

The percentage of students is listed first with the corresponding average scale score presented below.

* Significantly different from 1994.

NOTE: Percentages may not add to 100 due to rounding.

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of

Educational Progress (NAEP), 1994 and 2001 Geography Assessments.

Using maps and globes

Almost every day 7 6

284 277

Once or twice a week 22 24

288 285

Once or twice a month 31 34 *

286 287

Never or hardly ever 40 36 *

283 284

Natural resources

Almost every day 7 7

282 275

Once or twice a week 18 22 *

286 283

Once or twice a month 31 32

288 288

Never or hardly ever 45 39 *

284 285

Countries and cultures

Almost every day 16 20 *

287 286

Once or twice a week 26 32 *

288 288

Once or twice a month 30 29

286 286

Never or hardly ever 28 19 *

280 277

Environmental issues

Almost every day 11 11

284 279

Once or twice a week 22 26 *

288 286

Once or twice a month 30 33 *

288 289

Never or hardly ever 37 30 *

282 282

Table 4.4b

Percentage of students and average

geography scale scores by students’ reports

on frequency of instruction of selected skills

and topics, grade 12: 1994 and 2001

Frequency of

Instruction in

Twelfth-Grade

Geography Skills

and Topics

All topics received

more frequent

instruction in 2001.

Students who never

studied countries

and cultures scored

lower than those

who did.

1994 2001 How often have you studied the following geography skills

and topics in school?
CHAPTER 4 • G E O G R A P H Y R E P O R T C A R D 61

The Extent of Students’

Social Studies and

Geography Instruction

As part of the NAEP 2001 geography

assessment, eighth- and twelfth-grade

students were asked a series of questions

concerning the amount of geography

instruction they had received. The results

for the student responses are shown in

tables 4.5 and 4.6 and are summarized

below.

Geography Course Taking

at Grades 8 and 12

Eighth-grade students were asked whether

they had previously taken a geography

course in grades 6 and 7, and whether they

were currently enrolled in a geography

course in eighth grade. The results of their

answers to these questions are presented in

cumulative form in table 4.5. This table

shows the percentages and average scores of

eighth-grade students who reported taking

zero, one, two, and three years of geography

from the sixth through eighth grades. In

total, about 59 percent of eighth-grade

students in 2001 took two or three years of

geography. In 2001, 63 percent of eighthgraders reported taking a geography course

in eighth grade (data not shown). A higher

percentage of students in 2001 than in

1994 reported taking three years of geography and, conversely, a lower percentage in

2001 than in 1994 reported taking no

geography courses. In addition, there was a

positive association in 2001 between more

course-taking and higher geography scores.

Students who took three years of course

work had higher scores, on average, than

those who took two years. Those students

who took two years had higher scores than

those who took one year.
62 C H A P T E R 4 • GEOGRAPHY REPORT CARD

Twelfth-grade students were asked

whether they had previously taken a

geography course in grades 9, 10, 11, or

were currently enrolled in a geography

course. Table 4.6 displays percentages of

twelfth-grade students and their average

geography scores by their cumulative

amount of high school geography coursetaking. Twelfth-graders in 2001 were more

likely to report having taken three and four

courses than their counterparts in 1994.

The majority of twelfth-grade students (73

percent) indicated they were not currently

taking geography in twelfth grade (data not

shown). While taking more years of

geography was related to higher geography

scores for the eighth-graders in table 4.5,

this pattern did not hold for the twelfthgrade data in table 4.6. In fact, those who

reported taking no geography courses in

high school had higher average scores than

those who had taken 2, 3, or 4 years, and

were not significantly different from those

who took one year of geography. About

one-half of the twelfth-grade students (53

percent) took one year or less of geography

in high school. This group may represent

students who are following a different

academic curriculum than the students

who reported taking geography in multiple

years.

Grade 8

Table 4.5

Percentage of students and average

geography scale scores by students’

reports on grades in which geography

was taken since the 6th grade, grade 8:

1994 and 2001

The percentage of students is listed first with the corresponding average scale score presented below.

* Significantly different from 1994.

NOTE: Percentages may not add to 100 due to rounding.

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of

Educational Progress (NAEP), 1994 and 2001 Geography Assessments.

Number of grades selected

None 18 12 *

250 255

One 30 20 *

257 256

Two 14 16

269 263

Three 26 43 *

274 272

Don’t know 13 9 *

243 246

1994 2001 Did you take or do you expect to take a geography course

in 6th, 7th, or 8th grade?

Eighth-Grade

Frequency of

Geography Course

Taking

More frequent

course taking was

associated with

higher scores for

eighth-graders.
CHAPTER 4 • G E O G R A P H Y R E P O R T C A R D 63

Number of grades selected

None 31 21 *

286 289

One 35 32

288 288

Two 16 15

286 285

Three 10 18 *

281 280

Four 5 10 *

277 281

Don’t know 3 3

268 265

Grade 12

Table 4.6

Percentage of students and average

geography scale scores by students’

reports on grades in which geography

was taken since 9th grade, grade 12:

1994 and 2001

Twelfth-Grade

Frequency of

Geography Course

Taking

Students taking one

year or less of

geography had

higher scores, on

average, than

students taking 3 or

4 years.

The percentage of students is listed first with the corresponding average scale score presented below.

* Significantly different from 1994.

NOTE: Percentages may not add to 100 due to rounding.

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of

Educational Progress (NAEP), 1994 and 2001 Geography Assessments.

1994 2001 Did you take or do you expect to take a geography course

in 9th, 10th, 11th, or 12th grade?
64 C H A P T E R 4 • GEOGRAPHY REPORT CARD

9 Shepard, I. (1998). Teaching and learning geography with information and communication technologies. Cheltenham, UK:

Cheltenham and Gloucester College of Higher Education

Crampton, J. W. (1999). Integrating the web and the geography curriculum: the Bosnian virtual fieldtrip. Journal of

Geography 98, 155–168.

10 Mosely, W. G. (2001). Computer assisted comprehension of distant worlds: understanding hunger dynamics in

Africa. Journal of Geography 100, 32–45.

11 Sharma, M. B. & Elbow, G. S. (2000). Using internet primary sources to teach critical thinking skills in geography.

Westport, CT: Greenwood Professional Guides in School Librarianship.

Cross, J.A. (1997). Natural hazards and disaster information on the internet. Journal of Geography 96, 307–314.

Barta-Smith, N.A. & Hathaway, J.T. (2000). Making cyberspaces into cyberplaces. Journey of Geography 99, 253–265.

The Use of Computers in

the Social Studies Classroom

in Grades 4 And 8

Computers can be used to enhance geography instruction. The role of information

and communication technologies in the

classroom is to effectively supplement, not

replace, human contact in the teaching and

learning process.9 Some research has

shown that there is a quantifiable improvement in student understanding of geographic concepts when computers are used

to enhance the learning experience.10 The

use of computers in geography instruction

can also be employed to increase critical

thinking skills for oral and written presentations,as well as for specific topics in

geography.11 While most geography instruction takes place under the curriculum

category of “social studies,” such instruction also takes place in classes with geography titles. The 2001 NAEP geography

assessment asked teachers of fourth- and

eighth-grade students how frequently

computers were used when working on

social studies to locate and retrieve social

studies information through the Internet,

look up social studies information in CDROM reference works, use exploration or

simulation software, and organize social

studies information using spreadsheets or

databases. Table 4.7 presents results corresponding with teachers’ reports on the

frequency of these activities.

According to their teachers, the majority

of fourth-grade students used computers

for the four tasks either to a small extent or

not at all. At fourth grade, students occasionally retrieved information through the

Internet and used CD-ROMs for reference, but only rarely used exploration/

simulation software or spreadsheets and

databases. Nearly two-thirds of fourthgraders used the Internet or CD-ROMs to

at least a small extent while less than half

used simulation software or spreadsheets/

databases at least to a small extent. However, 34 percent and 37 percent of students,

respectively, did not engage in these computer activities at all. Students who used

the Internet and CD-ROMs to either a

small or a moderate extent had higher

average scores than students who did not

use them at all, and students who used

simulation software to a small extent

outperformed students who did not use

such software.
CHAPTER 4 • G E O G R A P H Y R E P O R T C A R D 65

Grade 4

Table 4.7a

Percentage of students and average

geography scale scores by teachers’

reports on computer use for social

studies instruction, grade 4: 2001

Grade 4

Use CD-ROM to look up reference works

Not at all 37

205

Small extent 47

211

Moderate extent 14

216

Large extent 2

214

Retrieve information through the Internet

Not at all 34

203

Small extent 45

212

Moderate extent 17

216

Large extent 4

211

Use exploration/simulation software

Not at all 54

207

Small extent 37

213

Moderate extent 8

211

Large extent 1

***

Organize information using spreadsheets/databases

Not at all 89

209

Small extent 9

213

Moderate extent 1

213

Large extent #

***

The percentage of students is listed first with the corresponding average scale score presented below.

*** Sample size is insufficient to permit a reliable estimate.

# Percentage is between 0.0 and 0.5.

NOTE: Percentages may not add to 100 due to rounding.

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of

Educational Progress (NAEP), 2001 Geography Assessment.

When students in this class work on social studies, to what 2001

extent do they use computers to do each of the following?

Fourth-Grade

Computer Use

Fourth-graders who

did not use these

technologies had

lower scores than

those who used

them to a small or

moderate extent.
66 C H A P T E R 4 • GEOGRAPHY REPORT CARD

The percentage of students is listed first with the corresponding average scale score presented below.

NOTE: Percentages may not add to 100 due to rounding.

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of

Educational Progress (NAEP), 2001 Geography Assessment.

Grade 8

Use CD-ROM to look up reference works

Not at all 31

258

Small extent 48

263

Moderate extent 17

266

Large extent 4

268

Retrieve information through the Internet

Not at all 20

255

Small extent 47

261

Moderate extent 29

266

Large extent 4

273

Use exploration/simulation software

Not at all 62

261

Small extent 32

265

Moderate extent 5

259

Large extent 1

257

Organize information using spreadsheets/databases

Not at all 74

261

Small extent 22

266

Moderate extent 2

262

Large extent 1

249

Grade 8

Table 4.7b

Percentage of students and average

geography scale scores by teachers’

reports on computer use for social

studies instruction, grade 8: 2001

When students in this class work on social studies, to what 2001

extent do they use computers to do each of the following?

Eighth-Grade

Computer Use

Eighth-graders who

did not use these

technologies had

lower scores than

those who used

them to a small or

moderate extent.
CHAPTER 4 • G E O G R A P H Y R E P O R T C A R D 67

At the eighth-grade level, the pattern of

frequency of use of the information technology tools was similar to that in grade 4:

information retrieval through the Internet

and use of CD-ROMs for reference occurred to a small or moderate extent for

two-thirds to three-quarters of the students,

while exploration/simulation software and

spreadsheets/databases were rarely used.

Twenty-nine percent of eighth-grade

students used the Internet to a moderate

extent for social studies and 47 percent

used it to a small extent, according to their

teachers. Forty-eight percent used CDROMs to a small extent, nearly one-third

(32 percent) used simulation software to a

small extent, and 22 percent used spreadsheets or databases to a small extent. About

three-quarters of students in eighth grade

(74 percent) did not use spreadsheets or

databases at all.

Greater usage of the Internet and CDROMs was generally associated with

higher performance among eighth-graders.

Students whose teachers reported a large

extent of Internet usage had higher average

scores than students who used the Internet

to a small extent or not at all. Students who

used the Internet or CD-ROMs to a

moderate or small extent had higher scores

than students who did not use these tools

at all.

The Use of Computers

in Grade 12

Twelfth-grade students participating in the

2001 NAEP geography assessment also

answered questions on the extent of use of

several types of computer technology. In

answering the questions, students were to

consider both work in class and homework

assignments. The results are shown in

table 4.8.

Forty-two percent of students used a

CD or the Internet for research to at least a

moderate extent. About one-third of

students (32 percent) used these tools to a

small extent, and 26 percent did not use

them at all. Students who reported using a

CD or the Internet for research to a small

or moderate extent had higher average

scores than those who never used them,

and moderate use was associated with

higher scores than a small amount of use.

There was no statistically significant difference between the average scores of students

using CDs and the Internet to a moderate

extent and those of students using them to

a large extent.

Thirty-four percent of twelfth-grade

students reported some use of simulation

software. Twenty-three percent used it to a

small extent, 9 percent to a moderate

extent, and 2 percent to a large extent. The

66 percent of the students who reported

not using simulation software at all had

higher average scores than students who

reported using it to any extent.

Thirty percent of the students reported

using a computer to a small extent to put

history or geography information into

tables, charts, or graphs. Sixteen percent of

students used a computer to a moderate or

large extent for this purpose and over onehalf (55 percent) of students never used a

computer for this purpose. Students who

reported carrying on these activities to a

small extent had higher average scores than

those who engaged in them either more or

not at all. Students who said they did not

do these activities at all outperformed

students who did them to a large extent.
68 C H A P T E R 4 • GEOGRAPHY REPORT CARD

Research projects using a CD or the Internet

Not at all 26

274

Small extent 32

285

Moderate extent 29

290

Large extent 13

292

Use exploration/simulation software

Not at all 66

287

Small extent 23

281

Moderate extent 9

276

Large extent 2

278

Tables, charts, or graphs on the computer

Not at all 55

284

Small extent 30

288

Moderate extent 12

281

Large extent 4

277

The percentage of students is listed first with the corresponding average scale score presented below.

NOTE: Percentages may not add to 100 due to rounding.

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of

Educational Progress (NAEP), 2001 Geography Assessment.

Grade 12

Table 4.8

Percentage of students and average

geography scale scores by students’

reports on computer use for history

and geography, grade 12: 2001

Twelfth-Grade

Computer Use

Students who

used electronic

information retrieval

had higher scores

than those who

did not.

2001 Think about all the courses since the 9th grade in which you

have studied history or geography. To what extent have you

used computers to do the following? For this question include

both work in class and homework assignments.
CHAPTER 4 • G E O G R A P H Y R E P O R T C A R D 69

Student Interest in Geography

Grades 8 and 12

Interest in geography may increase a

student’s efforts to learn the subject, either

in school or independently. The NAEP

2001 assessment asked students in grades 8

and 12 to indicate whether geography was

one of their favorite subjects, whether they

like most other subjects better than

geography, or whether they never studied

geography. As shown in table 4.9, most

eighth-grade students in 2001 preferred

subjects other than geography. Only 20

percent indicated that geography was one

of their favorites. However, these students

outperformed students who liked other

subjects better.

At grade 12, the percentage of students

who preferred subjects other than geography increased from 63 percent to 72

percent between 1994 and 2001 even as

the percentage of students who reported

never taking a geography class declined

(from 23 percent to 13 percent). Only 15

percent of students in 2001 chose geography as one of their favorite subjects. However, those students had higher average

scores than the students who did not favor

geography.

The percentage of students is listed first with the corresponding average scale score presented below.

* Significantly different from 1994.

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of

Educational Progress (NAEP), 1994 and 2001 Geography Assessments.

Grades8&12

Table 4.9

Percentage of students and

average geography scale scores

by students’ reports on how

much they like studying geography,

grades 8 and 12: 1994 and 2001

Grade 8

One of my favorite subjects 19 20

274 275

Like other subjects better 67 69

260 263

Never studied geography 14 11 *

241 247

Grade 12

One of my favorite subjects 14 15

297 293

Like other subjects better 63 72 *

285 285

Never studied geography 23 13 *

277 278

1994 2001 How much do you like studying geography?

How Much Eighthand Twelfth-Grade

Students Like

Geography

Students who

favored geography

had the highest

scores.

CHAPTER 5 • G E O G R A P H Y R E P O R T C A R D 71

5

Chapter

Contents

Two Sets of 2001

Geography

Results

Results for the

Nation

National Results

by Gender

National Results

by Race/Ethnicity

1 See appendix A for a description of specific criteria provided to assist them in making

exclusion decisions.

How would

the NAEP results

differ if

accommodations

were permitted

for special-needs

students?

Chapter

Focus

Becoming a More Inclusive

National Assessment

In its efforts to assess a representative sample of all students

in the nation, NAEP has consistently striven to include

special-needs students—those with disabilities (SD) and/or

limited English proficiency (LEP). A certain percentage of

such students, however, has always been excluded because

they could not be assessed meaningfully without

accommodations. Schools that participate in NAEP have

been permitted to exclude certain students who have

been classified as having a disability under the

Individuals with Disabilities Education Act (IDEA),

based upon their Individualized Education Programs

(IEP) and Section 504 of the Rehabilitation Act of

1973. Similarly, schools have been permitted to

exclude some students they identify as being limited

English proficient.1

In order to assess a more inclusive sample, and

in an attempt to remain consistent with state- and

district-level testing policies that increasingly offer

accommodations to special-needs students, NAEP

began to explore the use of accommodations in the

1996 and 1998 assessments. A split-sample design

was used to identify a portion of schools that could provide

accommodations to their special-needs students who

required them, and a portion of schools in which

accommodations would not be offered (the standard

administration procedure prior to 1996). The split-sample
72 C H A P T E R 5 • GEOGRAPHY REPORT CARD

2 Olson, J. F. & Goldstein, A. A. (1997). The inclusion of students with disabilities and limited-English-proficient students in

large-scale assessments: A summary of recent progress. (NCES Publication No. 97-482). Washington, DC: National

Center for Education Statistics.

Mazzeo, J., Carlson, J. E., Voelkl, K. E., & Lutkus, A. D. (1999). Increasing the participation of special needs students in

NAEP: A report on 1996 research activities. (NCES Publication No. 2000-473). Washington, DC: National Center

for Education Statistics.

design made it possible to study the effects

on NAEP results of including special-needs

students who required and were provided

accommodations, while at the same time,

obtaining results that were comparable to

those from previous assessments. Based on

research conducted and published since

that time, it was determined that NAEP

could begin a transition to reporting results

that included the performance of accommodated special-needs students.2

 It is

anticipated that in the near future, NAEP

will only report results that include accommodated special-needs students.

Two Sets of 2001 NAEP

Geography Results

This report is the first to display two

different sets of NAEP geography results

based on the split-sample design: 1) those

that reflect the performance of regular and

special-needs students when accommodations were not permitted; and 2) those that

reflect the performance of regular and

special-needs students—both those who

were accommodated and those who could

be tested without accommodations—when

accommodations were permitted. It should

be noted that accommodated students

make up a small proportion of the total

weighted number of students assessed (see

table A.6, page 124 in appendix A for

details). Making accommodations available

may change the overall assessment results in

subtle and different ways. For example,

when accommodations are permitted, there

may be some occurrences of students being

accommodated who might have taken the

test under standard conditions if accommodations were not permitted. This could

lead to an overall increase in the average

assessment results if accommodations were

to increase special-needs students’ performance. Conversely, when accommodations

are permitted, special-needs students who

could not have been tested without accommodations could be included in the

sample. Assuming that these are generally

lower-performing students, their inclusion

in the sample—even with accommodations—could result in an overall lower

average score.

The two sets of results presented in this

chapter were obtained by administering the

assessment to a nationally representative

sample of students and schools. In one

sample, no accommodations were permitted; all students were assessed under the

same conditions that were the basis for

reporting results from the 1994 NAEP

geography assessment. In another sample,

accommodations were permitted for SD

and/or LEP students who normally receive

accommodations in their district or state

assessment programs. Most accommodations that schools routinely provide for

their own testing programs were permitted.
CHAPTER 5 • G E O G R A P H Y R E P O R T C A R D 73

The permitted accommodations included,

but were not limited to, the following:

■ one-on-one testing;

■ bilingual dictionary;

■ small-group testing;

■ extended time;

■ oral reading of questions/directions; and

■ use of an aide for transcribing responses.

(See appendix A, table A.7, page 126, for

greater detail on the numbers and percentages of students accommodated by accommodation type in the 2001 assessment.)

Figure 5.1 provides a visual representation of how the two sets of results were

based on the two samples in 2001. Included in both sets of results (accommodations not permitted and accommodations

permitted) are those students from both

samples of schools who were not identified

as either SD and/or LEP. In addition, the

first set of results (accommodations not

permitted) includes SD and/or LEP students from the sample of schools where

accommodations were not permitted (see

middle portion of figure 5.1). This is the

set of results that allows for trend comparisons back to 1994 and are presented in the

other chapters of this report.

The second set of results, accommodations permitted (see bottom portion of

figure 5.1), includes SD and/or LEP

students from the sample of schools where

accommodations were permitted. This is

the set of results that form the new, more

inclusive baseline for future reporting of

trend comparisons for the NAEP geography assessment.
74 C H A P T E R 5 • GEOGRAPHY REPORT CARD

The two sets of NAEP results based on a split-sample design

Figure 5.1 Split-Sample Design

Split-sample design

The national sample was split. In part of the

schools, accommodations were not permitted

for students with disabilities (SD) and limited

English proficient (LEP) students. In the other

schools, accommodations were permitted for

SD and LEP students who routinely received

them in their school assessments.

Accommodations-not-permitted results

The accommodations-not-permitted results

include the performance of students from both

samples who were not classified as SD or LEP

and the performance of SD and LEP students

from the sample in which no accommodations

were permitted.

Accommodations-permitted results

The accommodations-permitted results also

include the performance of students from both

samples who were not classified as SD or LEP;

however, the SD and LEP students whose

performance is included in this set of

results were from the sample in which

accommodations were permitted. Since

students who required testing accommodations

could be assessed and represented in the

overall results, it was anticipated that these

results would include more special-needs

students and reflect a more inclusive sample.

Sample with no Sample with

accommodations permitted accommodations permitted

Non-SD/LEP Non-SD/LEP

students students

SD/LEP SD/LEP

students students

Sample with no Sample with

accommodations permitted accommodations permitted

Non-SD/LEP Non-SD/LEP

students students

SD/LEP SD/LEP

students students

Sample with no Sample with

accommodations permitted accommodations permitted

Non-SD/LEP Non-SD/LEP

students students

SD/LEP SD/LEP

students students
CHAPTER 5 • G E O G R A P H Y R E P O R T C A R D 75

In the NAEP 2001 sample where accommodations were not permitted, 16

percent of the students in fourth grade, 16

percent in eighth grade, and 11 percent in

twelfth grade were identified by their

schools as having special needs (i.e., either

as SD or LEP students). In the other sample

where accommodations were offered, 17

percent of the students in the fourth grade,

16 percent of students in the eighth grade,

and 10 percent in the twelfth grade were

identified as having special needs. In the

sample where accommodations were not

permitted, between 44 and 48 percent of

the special-needs students at each of the

three grade levels (between 5 and 8 percent

of all students—see appendix A, table A.5,

page 123) were excluded from NAEP

testing by their schools. In the sample

where accommodations were offered,

between 23 and 24 percent of the specialneeds students were excluded from the

assessment (between 2 and 4 percent of the

total sample).

Because the split-sample design was not

used in 1994, trend data for accommodated

students are not available. Therefore, this

chapter compares only the two sets of

results from the 2001 geography assessment. Overall results are provided for the

nation and for student subgroups by

gender and by race/ethnicity. These results

are discussed in terms of statistically significant differences between the two sets of

results and differences between subgroups

of students within each set of results.

Throughout this chapter, the assessment

results that include SD and/or LEP students for whom accommodations were not

permitted will be referred to as the “accommodations-not-permitted” results.

The set of results that includes SD and/or

LEP students for whom accommodations

were permitted will be referred to as the

“accommodations-permitted” results.

Results for the Nation

Accommodations Not Permitted and

Accommodations Permitted

Table 5.1 displays the average geography

scale scores for the nation in 2001 for two

sets of results: 1) accommodations not

permitted, and 2) accommodations permitted. There were no statistically significant

differences in the average scores between

the two sets of results at grades 4 and 12.

At grade 8, however, the average score

when accommodations were permitted

was lower than the average score when

accommodations were not permitted.
76 C H A P T E R 5 • GEOGRAPHY REPORT CARD

Table 5.1 Comparison of Two Sets of National Scale Score Results

National average geography scale scores by type of results, grades 4, 8, and 12: 2001

Accommodations not permitted Accommodations permitted

Grade 4 209 208

Grade 8 262 260 

Grade 12 285 284

 Significantly different from the sample where accommodations were not permitted.

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of

Educational Progress (NAEP), 2001 Geography Assessment.

As noted in the introduction to this

chapter, NAEP has always sought to include special-needs students proportional

to their representation in the U.S. population. Offering accommodations tends to

reduce exclusion rates for special-needs

students, and therefore allows NAEP to

offer a fairer and more accurate picture of

the status of American education. Because

special-needs students are typically classified as eligible for special educational

services after having shown some difficulty

in the regular learning environment, some

may assume that including the performance of these students would tend to

lower the overall results. This assumption

appears to have been justified only in the

observed difference between the two sets

of geography results in 2001 in grade 8,

where the accommodations-permitted

results, which included slightly more

special-needs students because of the

availability of accommodations, were lower

than the accommodations-not-permitted

results. It is important to examine the

percentages of students attaining the NAEP

achievement levels, however, to see if there

were higher percentages at the lower

performance ranges (i.e., below Basic and

Basic), when students were assessed with

accommodations.

Table 5.2 shows the percentages of

students attaining each of the achievement

levels. The percentages are similar across

the two sets of results for grades 4 and 12;

apparent differences between the accommodations-not-permitted and the accommodations-permitted results were not

significantly different. At grade 8, however,

the percentage of students at or above

Basic was higher when accommodations

were not permitted than when they were

permitted.
CHAPTER 5 • G E O G R A P H Y R E P O R T C A R D 77

Percentage of students within and at or above geography achievement levels by type of results,

grades 4, 8, and 12: 2001

Table 5.2 Comparison of Two Sets of National Achievement-Level Results

At or above At or above

Below Basic At Basic At Proficient At Advanced Basic Proficient

Grade 4

Accommodations were not permitted 26 53 19 2 74 21

Accommodations were permitted 27 52 19 2 73 20

Grade 8

Accommodations were not permitted 26 44 26 4 74 30

Accommodations were permitted 28  43 25 4 72  29

Grade 12

Accommodations were not permitted 29 47 23 1 71 25

Accommodations were permitted 29 47 23 1 71 24

 Significantly different from the sample where accommodations were not permitted.

NOTE: Percentages within each geography achievement-level range may not add to 100, or to the exact percentages at or above achievement levels, due to rounding.

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of

Educational Progress (NAEP), 2001 Geography Assessment.

National average geography scale scores by gender and type of results, grades 4, 8, and 12: 2001

Table 5.3 Comparison of Two Sets of National Scale Score Results by Gender

Male Female

Grade 4

Accommodations were not permitted 212 207

Accommodations were permitted 210 206

Grade 8

Accommodations were not permitted 264 260

Accommodations were permitted 262  258

Grade 12

Accommodations were not permitted 287 282

Accommodations were permitted 287 281

Significantly different from the sample where accommodations were not permitted.

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of

Educational Progress (NAEP), 2001 Geography Assessment.

National Results by Gender

Accommodations Not Permitted and

Accommodations Permitted

The average geography scale scores by

gender for both sets of results in 2001 are

provided in table 5.3. Male students at

grade 8 had higher geography scores when

accommodations were not permitted than

when accommodations were permitted.
78 C H A P T E R 5 • GEOGRAPHY REPORT CARD

As noted in chapter 3, in 2001, male

students at all three grades where accommodations were not offered had higher

scale scores than female students. The

same pattern continued where accommodations were offered—male students had

higher average scale scores than female

students in all three grades.

The percentages of male and female

students attaining the Basic, Proficient, and

Advanced levels are provided in table 5.4.

Comparing the two sets of results in 2001,

there were no statistically significant differences by accommodation status in the

percentages of male or female students

attaining each of the achievement levels at

grades 4, 8, or 12.

National Results by

Race/Ethnicity

Accommodations Not Permitted and

Accommodations Permitted

NAEP assessments across academic subjects

have typically reported large score differTable 5.4 Comparison of Two Sets of National Achievement-Level Results by Gender

At or above At or above

Below Basic At Basic At Proficient At Advanced Basic Proficient

Percentage of students within and at or above geography achievement levels by gender and

type of results, grades 4, 8, and 12: 2001

Grade 4

Male

Accommodations were not permitted 25 51 21 3 75 24

Accommodations were permitted 26 51 21 3 74 23

Female

Accommodations were not permitted 28 54 17 1 72 18

Accommodations were permitted 29 54 16 1 71 18

Grade 8

Male

Accommodations were not permitted 25 42 29 5 75 33

Accommodations were permitted 27 41 27 4 73 32

Female

Accommodations were not permitted 27 47 24 3 73 26

Accommodations were permitted 29 45 23 3 71 26

Grade 12

Male

Accommodations were not permitted 27 45 26 2 73 28

Accommodations were permitted 26 45 26 2 74 28

Female

Accommodations were not permitted 30 48 20 1 70 21

Accommodations were permitted 32 48 19 1 68 20

NOTE: Percentages within each geography achievement-level range may not add to 100, or to the exact percentages at or above achievement levels, due to rounding.

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of

Educational Progress (NAEP), 2001 Geography Assessment.

ences according to race and ethnic group

membership. If SD and/or LEP students

are over-represented in a particular racial or

ethnic group, that group’s assessment scores

may decrease. Table 5.5 provides the
CHAPTER 5 • G E O G R A P H Y R E P O R T C A R D 79

National average geography scale scores by race/ethnicity and type of results, grades 4, 8, and 12:

2001

Table 5.5 Comparison of Two Sets of National Scale Score Results by Race/Ethnicity

Asian/Pacific American

White Black Hispanic Islander Indian

Grade 4

Accommodations were not permitted 222 181 184 212 199

Accommodations were permitted 220 181 185 216 199

Grade 8

Accommodations were not permitted 273 234 240 266 261

Accommodations were permitted 271 232 238 267 259

Grade 12

Accommodations were not permitted 291 260 270 286 288

Accommodations were permitted 292 258 269 285 286

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of

Educational Progress (NAEP), 2001 Geography Assessment.

average geography scale scores for each of

the race/ethnicity categories for the two

sets of results in 2001. There were no

statistically significant differences in average

As noted in chapter 3, a pattern of

performance differences by race/ethnicity

can be seen in the accommodations-notpermitted results in 2001. Both White and

Asian/Pacific Islander students at all three

grades scored higher, on average, than

Black and Hispanic students. The same

pattern can be observed at all three grades

in the accommodations-permitted results.

In addition, Hispanic students had higher

scores than Black students regardless of

accommodations condition at grades 8 and

12. At grade 4, however, regardless of

accommodation conditions, there was no

significant difference between the scores of

Black and Hispanic students.

The relative standing of the performance

of American Indian students changes by

grade level. At grade 4, American Indian

students had lower average scores than

White students regardless of accommodation condition. At grades 8 and 12, however, the scores of American Indian students were not significantly different than

White students within either accommodation condition.

The percentages of students in each

race/ethnicity category who attained the

Basic, Proficient, and Advanced levels are

provided in table 5.6. No statistically

significant differences were found at any of

the three grades between the accommodations-not-permitted results and the accommodations-permitted results for the percentages of students attaining each of the

achievement levels in 2001.

scores for any racial/ethnic group at any

grade, between the samples where accommodations were not permitted and where

accommodations were permitted.
80 C H A P T E R 5 • GEOGRAPHY REPORT CARD

Table 5.6 Comparison of Two Sets of National Achievement-Level Results by Race/Ethnicity

Percentage of students within and at or above geography achievement levels by race/ethnicity and

type of results, grades 4, 8, and 12: 2001

At or above At or above

Below Basic At Basic At Proficient At Advanced Basic Proficient

# Percentage is between 0.0 and 0.5.

NOTE: Percentages within each geography achievement-level range may not add to 100, or to the exact percentages at or above achievement levels, due to rounding.

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of

Educational Progress (NAEP), 2001 Geography Assessment.

Grade 4

White

Accommodations were not permitted 13 58 26 3 87 29

Accommodations were permitted 15 57 25 3 85 28

Black

Accommodations were not permitted 56 39 5 # 44 5

Accommodations were permitted 56 40 4 # 44 4

Hispanic

Accommodations were not permitted 51 43 6 # 49 6

Accommodations were permitted 49 45 5 # 51 6

Asian/Pacific Islander

Accommodations were not permitted 23 52 23 1 77 25

Accommodations were permitted 18 57 24 2 82 25

American Indian

Accommodations were not permitted 34 53 13 # 66 13

Accommodations were permitted 37 51 12 # 63 12

Grade 8

White

Accommodations were not permitted 14 48 34 5 86 39

Accommodations were permitted 16 46 33 5 84 38

Black

Accommodations were not permitted 60 34 6 # 40 6

Accommodations were permitted 62 32 6 # 38 6

Hispanic

Accommodations were not permitted 52 38 9 1 48 10

Accommodations were permitted 54 37 9 1 46 9

Asian/Pacific Islander

Accommodations were not permitted 21 47 28 4 79 32

Accommodations were permitted 20 49 28 4 80 32

American Indian

Accommodations were not permitted 28 41 29 3 72 31

Accommodations were permitted 30 46 21 3 70 24

Grade 12

White

Accommodations were not permitted 19 51 29 2 81 31

Accommodations were permitted 19 51 29 2 81 31

Black

Accommodations were not permitted 65 31 4 # 35 4

Accommodations were permitted 67 30 3 # 33 3

Hispanic

Accommodations were not permitted 48 42 10 # 52 10

Accommodations were permitted 50 42 9 # 50 9

Asian/Pacific Islander

Accommodations were not permitted 28 45 25 1 72 26

Accommodations were permitted 29 46 23 1 71 25

American Indian

Accommodations were not permitted 26 41 31 1 74 32

Accommodations were permitted 29 41 29 1 71 30
CHAPTER 6 • G E O G R A P H Y R E P O R T C A R D 81

6

Chapter

Contents

Sample

Questions

Student

Responses

Item Maps

Sample materials

from the 2001

geography

assessment

Chapter

Focus

Sample Assessment Questions

And Student Responses

This chapter presents sample questions from the 2001

NAEP geography assessment. Four sample questions at each

grade are provided, including multiple-choice and

constructed-response questions. Each sample is classified

according to its geography content area, as described in the

geography framework. The constructed-response questions

are accompanied by actual student responses, reproduced

from test booklets, that illustrate work at different

rating levels. The constructed-response samples were

rated using either a three-point or four-point scoring

rubric. Three-point questions were rated as

“Complete,” “Partial,” or “Inappropriate.” Four-point

questions were rated as “Complete,” “Essential,”

“Partial,” or “Inappropriate. ” Sample responses are

included for each level except “Inappropriate.”

The table accompanying each sample question

presents two types of performance data: the overall

percentage of students who answered successfully,

and the percentage of students who answered

successfully within a specific score range on the

NAEP geography scale. The score ranges correspond

to the three achievement-level intervals—Basic, Proficient, and

Advanced—as well as the range below Basic. These

percentages give some indication of how difficult the

question was for students who performed within each of the

achievement-level ranges.
82 C H A P T E R 6 • GEOGRAPHY REPORT CARD

Many additional sample questions

released from the 1994 and 2001 NAEP

geography assessments are available for

viewing on the NAEP Web Site at http://

www.nces.ed.gov/nationsreportcard/

itmrls/. The item-viewing feature of the

Web Site includes student performance

data for all questions, detailed scoring

guides (rubrics), and sample student responses for the constructed-response

questions.

Grade 4 Sample Assessment

Questions and Results

Questions in the grade 4 assessment cover a

wide variety of geographic concepts and

skills across the three geography content

areas. A somewhat higher percentage of

questions is devoted to United States

geography than at the two higher grades

where increasing emphasis is placed on

world geography.

Many of the questions at all three grades

are based upon visual or textual stimuli

designed to make the assessment more

interesting and more authentic. Visual

stimuli include maps, charts, graphs,

diagrams, cartoons, and, as in sample

question 1, photographs.

The sample questions are also marked

on the item maps on pages 110-112. The

item map location of each question identifies the scale score at which that question

was answered successfully by at least 65

percent of the students for constructedresponse questions and 74 percent of the

students for four-option multiple-choice

questions.
CHAPTER 6 • G E O G R A P H Y R E P O R T C A R D 83

Look at the photograph above. What would help farmers in this area grow

more food?

A Cutting down forests

B Making terraces

C Building houses

Irrigating the land

In sample question 1, students are assessed on whether they can recognize a photographic

representation of a landscape and associate irrigation with the landscape depicted. This

question is mapped at scale score 216.

Table 6.1 Sample Question 1 Results (Multiple-Choice)

Overall percentage correct and percentages correct within each achievement-level range: 2001

*NAEP geography composite scale range.

***Sample size is insufficient to permit a reliable estimate (see appendix A).

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education

Statistics, National Assessment of Educational Progress (NAEP), 2001 Geography Assessment.

Percentage correct within

achievement-level intervals

Overall percentage Below Basic Basic Proficient Advanced

correct 186 and below* 187–239* 240–275* 276 and above*

70 50 74 84 ***

Grade 4

© Fritz Henle/Photo Researchers, Inc.

Geography Content Area: Environment and Society

Grade 4 Sample Question 1:
84 C H A P T E R 6 • GEOGRAPHY REPORT CARD

Which two nations are most likely to have a conflict over mineral resources?

A Nation A and Nation B

B Nation A and Nation C

Nation A and Nation D

D Nation C and Nation D

Sample question 2 measures students’ understanding of how geography plays a role in

conflict among countries. Students found this question to be quite difficult, with only

one-third answering correctly. This question appears on the item map at scale score 271.

Table 6.2 Sample Question 2 Results (Multiple-Choice)

Overall percentage correct and percentages correct within each achievement-level range: 2001

*NAEP geography composite scale range.

***Sample size is insufficient to permit a reliable estimate (see appendix A).

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education

Statistics, National Assessment of Educational Progress (NAEP), 2001 Geography Assessment.

Percentage correct within

achievement-level intervals

Overall percentage Below Basic Basic Proficient Advanced

correct 186 and below* 187–239* 240–275* 276 and above*

33 22 28 56 ***

Grade 4

Geography Content Area: Spatial Dynamics and Connections

Grade 4 Sample Question 2:
CHAPTER 6 • G E O G R A P H Y R E P O R T C A R D 85

Sample question 3 is one of a number of production tasks included in the NAEP geography

assessment in which students are asked to locate a place on a map or draw a map in their test

booklet. Responses to this question were scored with a three-level rubric as “Complete,” “Partial,” or “Inappropriate.” Two-thirds of students could correctly identify where they lived. This

question appears on the item map at scale score 192. (Note that the circled numbers on the

map were used in a different question that was also based on this map.)

Write down the name of the state or district where you live.

I live in______________________________________________ .

Directly on the map, draw an “X” on the state or district where you live.

Geography Content Area: Space and Place

Grade 4 Sample Question 3:
86 C H A P T E R 6 • GEOGRAPHY REPORT CARD

To earn a score of “Complete” on this question, students had to write the name of the state

or district where they live and correctly mark the location on the map.

Sample “Complete” Response:

Write down the name of the state or district where you live.

I live in______________________________________________ .

Directly on the map, draw an “X” on the state or district where you live.

Table 6.3a Sample Question 3 Results (“Complete” Short-Constructed-Response)

Overall percentage “Complete” and percentages “Complete” within each achievement-level range:

2001

*NAEP geography composite scale range.

***Sample size is insufficient to permit a reliable estimate (see appendix A).

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education

Statistics, National Assessment of Educational Progress (NAEP), 2001 Geography Assessment.

Percentage “Complete” within

achievement-level intervals

Overall percentage Below Basic Basic Proficient Advanced

“Complete” 186 and below* 187–239* 240–275* 276 and above*

66 38 71 88 ***

Grade 4
CHAPTER 6 • G E O G R A P H Y R E P O R T C A R D 87

Write down the name of the state or district where you live.

I live in______________________________________________ .

Directly on the map, draw an “X” on the state or district where you live.

To earn a score of “Partial,” students could indicate their state or district and mark a bordering state, or they could indicate the city or town in which they live and mark the correct state

in which that city lies. In the sample below, the student lives in North Carolina but marked

Virginia on the map.

Sample “Partial” Response:

Table 6.3b Sample Question 3 Results (“Partial” Short-Constructed-Response)

Overall percentage “Partial” or better and percentages “Partial” or better within each achievementlevel range: 2001

*NAEP geography composite scale range.

***Sample size is insufficient to permit a reliable estimate (see appendix A).

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education

Statistics, National Assessment of Educational Progress (NAEP), 2001 Geography Assessment.

Percentage “Partial” or better within

achievement-level intervals

Overall percentage Below Basic Basic Proficient Advanced

“Partial” or better 186 and below* 187–239* 240–275* 276 and above*

72 43 78 93 ***

Grade 4
88 C H A P T E R 6 • GEOGRAPHY REPORT CARD

LITTLE TOWN

– Width: 4.0 miles east to west

– Length: 3.0 miles north to south

– Main Street runs east to west through the town.

– The school is on the northeast side of town.

– Phelps Park is on the southwest side of town.

– Runt River runs north to south through the town.

On the grid below, each square is one mile wide and one mile long. Draw a

map of Little Town on the grid. Draw the town’s borders. Then, use the

symbols in the key below to draw the features listed above.

A more complex production task is seen in sample question 4. Here, students must use

written descriptions of features of a town to sketch a map on a grid. They must understand

scale, distance, and direction, and be able to read and use a map key. Responses were

scored with a four-level rubric as “Complete,” “Essential,” “Partial,” or “Inappropriate.” The

question was difficult for fourth-graders, with only 28 percent providing an “Essential” or

better response. The item map scale score point for this question is 295.

N

S

W E

Scale

= 1 mile

Key

S

P

School

Street

Park

River

Geography Content Area: Space and Place

Grade 4 Sample Question 4:
CHAPTER 6 • G E O G R A P H Y R E P O R T C A R D 89

Responses scored “Complete” correctly located all four features and drew the length and

width to scale in the correct directions.

Sample “Complete” Response:

N

S

W E

Scale

= 1 mile

Key

S

P

School

Street

Park

River

Overall percentage “Complete” and percentages “Complete” within each achievement-level range:

2001

Table 6.4a Sample Question 4 Results (“Complete” Extended-Constructed-Response)

Percentage “Complete” within

achievement-level intervals

Overall percentage Below Basic Basic Proficient Advanced

“Complete” 186 and below* 187–239* 240–275* 276 and above*

11 0 6 32 ***

*NAEP geography composite scale range.

***Sample size is insufficient to permit a reliable estimate (see appendix A).

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education

Statistics, National Assessment of Educational Progress (NAEP), 2001 Geography Assessment.

Grade 4
90 C H A P T E R 6 • GEOGRAPHY REPORT CARD

Responses scored “Essential” correctly located four features but not to scale, or correctly

located three features and had the scale correct.

Sample “Essential” Response:

N

S

W E

Scale

= 1 mile

Key

S

P

School

Street

Park

River

Overall percentage “Essential” or better and percentages “Essential” or better within each achievementlevel range: 2001

Table 6.4b Sample Question 4 Results (“Essential” Extended-Constructed-Response)

Percentage “Essential” or better within

achievement-level intervals

Overall percentage Below Basic Basic Proficient Advanced

“Essential” or better 186 and below* 187–239* 240–275* 276 and above*

28 1 25 65 ***

*NAEP geography composite scale range.

***Sample size is insufficient to permit a reliable estimate (see appendix A).

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education

Statistics, National Assessment of Educational Progress (NAEP), 2001 Geography Assessment.

Grade 4
CHAPTER 6 • G E O G R A P H Y R E P O R T C A R D 91

N

S

W E

Scale

= 1 mile

Key

S

P

School

Street

Park

River

Overall percentage “Partial” or better and percentages “Partial” or better within each achievement-level

range: 2001

Table 6.4c Sample Question 4 Results (“Partial” Extended-Constructed-Response)

Percentage “Partial” or better within

achievement-level intervals

Overall percentage Below Basic Basic Proficient Advanced

“Partial” or better 186 and below* 187–239* 240–275* 276 and above*

38 4 36 78 ***

*NAEP geography composite scale range.

***Sample size is insufficient to permit a reliable estimate (see appendix A).

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education

Statistics, National Assessment of Educational Progress (NAEP), 2001 Geography Assessment.

Grade 4

Responses scored “Partial” located only one or two features and had the scale correct, or

located three features with an incorrect scale.

Sample “Partial” Response:
92 C H A P T E R 6 • GEOGRAPHY REPORT CARD

Grade 8 Sample Assessment

Questions and Results

The assessment at grade 8, like that at

grade 4, covers a wide range of geography

skills and concepts. The questions, on

average, look for a deeper understanding of

the material and require students to grapple

with more sophisticated stimuli, compare

multiple maps, and apply geographic

understanding to solving problems.
CHAPTER 6 • G E O G R A P H Y R E P O R T C A R D 93

Table 6.5 Sample Question 5 Results (Multiple-Choice)

Overall percentage correct and percentages correct within each achievement-level range: 2001

*NAEP geography composite scale range.

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education

Statistics, National Assessment of Educational Progress (NAEP), 2001 Geography Assessment.

Percentage correct within

achievement-level intervals

Overall percentage Below Basic Basic Proficient Advanced

correct 241 and below* 242–281* 282–314* 315 and above*

70 37 74 91 97

Grade 8

Which question could you answer based only on the information in the map?

A At what times do the public trains arrive?

B How much time does it take to go from Forest Hills to Oak Grove?

C How many miles is it from one station to another?

How can one travel from Alewife to the Aquarium by public train?

Geography Content Area: Spatial Dynamics and Connections

Grade 8 Sample Question 5:

In this multiple-choice question students are asked to interpret a kind of map they may never

have seen to determine exactly what kind of information it provides and doesn’t provide. It

was a fairly easy task for students. The scale score point for this question on the eighth-grade

item map is 257.
94 C H A P T E R 6 • GEOGRAPHY REPORT CARD

Table 6.6 Sample Question 6 Results (Multiple-Choice)

Overall percentage correct and percentages correct within each achievement-level range: 2001

*NAEP geography composite scale range.

***Sample size is insufficient to permit a reliable estimate (see appendix A).

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education

Statistics, National Assessment of Educational Progress (NAEP), 2001 Geography Assessment.

Percentage correct within

achievement-level intervals

Overall percentage Below Basic Basic Proficient Advanced

correct 241 and below* 242–281* 282–314* 315 and above*

50 36 47 64 ***

Grade 8

How do the forces listed below affect the natural environment?

Gravity

Ice

Water

Wind

They are major causes of erosion.

B They are important influences on human settlement.

C They are responsible for seismic activity.

D They cause continental drift.

Geography Content Area: Space and Place

Grade 8 Sample Question 6:

Sample question 6 asks about an important aspect of physical geography. One-half of eighthgraders knew that the four forces contribute to erosion. This question maps at scale score

point 316.
CHAPTER 6 • G E O G R A P H Y R E P O R T C A R D 95

Table 6.7 Sample Question 7 Results (Multiple-Choice)

Overall percentage correct and percentages correct within each achievement-level range: 2001

*NAEP geography composite scale range.

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education

Statistics, National Assessment of Educational Progress (NAEP), 2001 Geography Assessment.

Percentage correct within

achievement-level intervals

Overall percentage Below Basic Basic Proficient Advanced

correct 241 and below* 242–281* 282–314* 315 and above*

74 40 80 93 100

Grade 8

Florida is an example of

A an isthmus

B an island

a peninsula

D a plateau

Geography Content Area: Space and Place

Grade 8 Sample Question 7:

Sample question 7 tests students’ knowledge of landforms as well as their skill with what

geographers call “mental mapping”—the ability to visualize spatial patterns in one’s mind.

Students had to create an image of Florida in their minds before they could identify it as a

peninsula. Nearly three-quarters of the students answered correctly. The question maps at a

scale score of 256.
96 C H A P T E R 6 • GEOGRAPHY REPORT CARD

Table 6.8 Sample Question 8 Results (Multiple-Choice)

Overall percentage correct and percentages correct within each achievement-level range: 2001

*NAEP geography composite scale range.

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education

Statistics, National Assessment of Educational Progress (NAEP), 2001 Geography Assessment.

Percentage correct within

achievement-level intervals

Overall percentage Below Basic Basic Proficient Advanced

correct 241 and below* 242–281* 282–314* 315 and above*

60 40 57 79 96

Grade 8

Geography Content Area: Spatial Dynamics and Connections

Grade 8 Sample Question 8:

Sample question 8 measures students’ understanding of why countries join trans-regional

organizations, a topic related to the larger theme of how people from different places work

together across space to address common issues. Sixty percent of students answered this

moderately difficult question correctly. The item map scale score for this question is 285.

What is an important reason that countries join international

organizations like the United Nations?

A Countries who do not join usually lose their independence.

Many of the world’s problems involve more than one country.

C Most citizens want their countries to join as many international

organizations as possible.

D Such organizations force countries to join.
CHAPTER 6 • G E O G R A P H Y R E P O R T C A R D 97

Tropical forests are being destroyed at the rate of at least eleven million

hectares each year, an area the size of Pennsylvania. About half of all

tropical forests are already gone.

Discuss two major reasons for this high rate of tropical deforestation.

Geography Content Area: Environment and Society

Grade 8 Sample Question 9:

Sample question 9 is a short-constructed-response question designed to measure students’

understanding of the interaction between human beings and the environment. Responses

were scored on a three-level rubric as “Complete,” “Partial,” or “Inappropriate.” The question was quite difficult for students, with only 22 percent giving a “Complete” response. On

the item map for eighth grade this question appears as scale score 328.
98 C H A P T E R 6 • GEOGRAPHY REPORT CARD

Responses scored “Complete” provided two reasons for the high rate of tropical deforestation. Reasons could relate to demand for land and resources or to the lack of regulation that

allows deforestation to occur.

Sample “Complete” Response:

Discuss two major reasons for this high rate of tropical deforestation.

Percentage “Complete” within

achievement-level intervals

Overall percentage Below Basic Basic Proficient Advanced

“Complete” 241 and below* 242–281* 282–314* 315 and above*

22 6 18 38 ***

Overall percentage “Complete” and percentages “Complete” within each achievement-level range:

2001

Table 6.9a Sample Question 9 Results (“Complete” Short-Constructed-Response)

*NAEP geography composite scale range.

***Sample size is insufficient to permit a reliable estimate (see appendix A).

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education

Statistics, National Assessment of Educational Progress (NAEP), 2001 Geography Assessment.

Grade 8
CHAPTER 6 • G E O G R A P H Y R E P O R T C A R D 99

Responses scored “Partial” provided only one reason for the high rate of tropical deforestation, thereby revealing a more limited knowledge of the subject.

Sample “Partial” Response:

Discuss two major reasons for this high rate of tropical deforestation.

Percentage “Partial” or better within

achievement-level intervals

Overall percentage Below Basic Basic Proficient Advanced

“Partial” or better 241 and below* 242–281* 282–314* 315 and above*

60 26 62 84 ***

Overall percentage “Partial” or better and percentages “Partial” or better within each achievementlevel range: 2001

Table 6.9b Sample Question 9 Results (“Partial” Short-Constructed-Response)

*NAEP geography composite scale range.

***Sample size is insufficient to permit a reliable estimate (see appendix A).

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education

Statistics, National Assessment of Educational Progress (NAEP), 2001 Geography Assessment.

Grade 8
100 C H A P T E R 6 • GEOGRAPHY REPORT CARD

Overall percentage correct and percentages correct within each achievement-level range: 2001

Table 6.10 Sample Question 10 Results (Multiple-Choice)

*NAEP geography composite scale range.

***Sample size is insufficient to permit a reliable estimate (see appendix A).

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education

Statistics, National Assessment of Educational Progress (NAEP), 2001 Geography Assessment.

Percentage correct within

achievement-level intervals

Overall percentage Below Basic Basic Proficient Advanced

correct 269 and below* 270–304* 305–338* 339 and above*

78 46 86 99 ***

Grade 12

The varying widths of the lines on the map most probably indicate the

A strength of ocean currents

B type of trade

volume of trade

D type of transportation used

Geography Content Area: Space and Place

Grade 12 Sample Question 10:

Grade 12 Sample Assessment

Questions and Results

The grade 12 assessment included higher

percentages of extended-constructedresponse questions and questions devoted

to non-U.S. geography than the assessments

at grades 4 and 8. It also contained the most

complex stimuli and challenging concepts.

Sample question 10 is a skills question designed to measure whether students understand

the conventions used in what is known as a flow map. A majority of students (78 percent)

successfully answered the question. This question appears on the twelfth-grade item map at

scale score 272.
CHAPTER 6 • G E O G R A P H Y R E P O R T C A R D 101

What religion is practiced by most people who live in India?

A Confucianism

B Buddhism

C Christianity

Hinduism

Overall percentage correct and percentages correct within each achievement-level range: 2001

Table 6.11 Sample Question 11 Results (Multiple-Choice)

*NAEP geography composite scale range.

***Sample size is insufficient to permit a reliable estimate (see appendix A).

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education

Statistics, National Assessment of Educational Progress (NAEP), 2001 Geography Assessment.

Percentage correct within

achievement-level intervals

Overall percentage Below Basic Basic Proficient Advanced

correct 269 and below* 270–304* 305–338* 339 and above*

61 46 62 76 ***

Grade 12

Geography Content Area: Spatial Dynamics and Connections

Grade 12 Sample Question 11:

This straightforward multiple-choice question helps measure students’ knowledge of the

distribution of world religions. Six out of ten students answered correctly. The item map scale

score point for this question is 318.
102 C H A P T E R 6 • GEOGRAPHY REPORT CARD

Sample question 12 is a map-based, short-constructed-response question dealing with the

interaction between humans and the natural environment. Although some students may have

been able to answer without referring to the map, others could use it to gain valuable information about the region. Responses were scored on a three-level rubric as “Complete,”

“Partial,” or “Inappropriate.” The question was moderately difficult, with 47 percent of

students providing a “Complete” response. This question “maps” at scale score 300 for

“Complete.”

Give two reasons why early civilizations flourished in the valley of

the Tigris and Euphrates rivers.

Geography Content Area: Environment and Society

Grade 12 Sample Question 12:

Eu phrates

0 600 mi

N

S

W E

SOUTHWEST ASIA

Egypt

Turkey

Syria Lebanon

Jordan

Israel Iraq

Saudi

Arabia

Iran

Persian

Gulf

Tigris

Qatar

United Arab Emirates

Oman

Afghanistan

0 600 km

Key

 = National Boundaries

 = Rivers

Kuwait
CHAPTER 6 • G E O G R A P H Y R E P O R T C A R D 103

Responses scored “Complete” gave two valid reasons why river valleys were important to the

early civilization of Iraq.

Sample “Complete” Response:

Give two reasons why early civilizations flourished in the valley of

the Tigris and Euphrates rivers.

Percentage “Complete”within

achievement-level intervals

Overall percentage Below Basic Basic Proficient Advanced

“Complete” 269 and below* 270–304* 305–338* 339 and above*

47 17 52 70 ***

Overall percentage “Complete” and percentages “Complete” within each achievement-level range:

2001

Table 6.12a Sample Question 12 Results (“Complete” Short-Constructed-Response)

*NAEP Geography composite scale range.

***Sample size is insufficient to permit a reliable estimate (see appendix A).

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education

Statistics, National Assessment of Educational Progress (NAEP), 2001 Geography Assessment.

Grade 12
104 C H A P T E R 6 • GEOGRAPHY REPORT CARD

Responses scored “Partial” gave only one valid reason for the importance of the river valley

to the early civilization of Iraq.

Sample “Partial” Response:

Give two reasons why early civilizations flourished in the valley of

the Tigris and Euphrates rivers.

Percentage “Partial” or better within

achievement-level intervals

Overall percentage Below Basic Basic Proficient Advanced

“Partial” or better 269 and below* 270–304* 305–338* 339 and above*

76 42 85 96 ***

Overall percentage “Partial” or better and percentages “Partial” or better within each achievement-level

range: 2001

Table 6.12b Sample Question 12 Results (“Partial” Short-Constructed-Response)

*NAEP Geography composite scale range.

***Sample size is insufficient to permit a reliable estimate (see appendix A).

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education

Statistics, National Assessment of Educational Progress (NAEP), 2001 Geography Assessment.

Grade 12
CHAPTER 6 • G E O G R A P H Y R E P O R T C A R D 105

Describe the difference in population patterns for people age 60

and over in countries 1 and 2. Give one possible explanation for the

difference you have identified.

Geography Content Area: Spatial Dynamics and Connections

Grade 12 Sample Question 13:

Sample question 13 is a short-constructed-response that measures students’ ability to read

and understand population pyramids. Responses were scored on a three-point rubric as

“Complete,” “Partial,” or “Inappropriate.” Students found this question to be very difficult.

Sixteen percent received a score of “Complete.” This question maps at scale score 347 for

“Complete.”

Age

% of

Total

Pop’n

70 + 1.0%

60-69 1.6%

50-59 2.6%

40-49 3.9%

30-39 5.6%

20-29 7.7%

10-19 10.4%

0-9 17.0%

Male

COUNTRY 1 

Age Distribution

Female

Age

% of

Total

Pop’n

1.2% 70 + 

1.8% 60-69

2.7% 50-59

4.0% 40-49

5.5% 30-39

7.7% 20-29

10.4% 10-19

16.9% 0-9

Age

% of

Total

Pop’n

70 + 2.9%

60-69 3.7%

50-59 4.7%

40-49 5.8%

30-39 8.2%

20-29 9.3%

10-19 7.5%

0-9 7.3%

Male

COUNTRY 2 

Age Distribution

Female

Age

% of

Total

Pop’n

4.2% 70 + 

4.3% 60-69

4.8% 50-59

5.7% 40-49

8.3% 30-39

9.2% 20-29

7.1% 10-19

7.0% 0-9
106 C H A P T E R 6 • GEOGRAPHY REPORT CARD

Percentage “Complete” within

achievement-level intervals

Overall percentage Below Basic Basic Proficient Advanced

“Complete” 269 and below* 270–304* 305–338* 339 and above*

16 2 15 33 ***

Overall percentage “Complete” and percentages “Complete” within each achievement-level range:

2001

Table 6.13a Sample Question 13 Results (“Complete” Short-Constructed-Response)

*NAEP geography composite scale range.

***Sample size is insufficient to permit a reliable estimate (see appendix A).

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education

Statistics, National Assessment of Educational Progress (NAEP), 2001 Geography Assessment.

Grade 12

Responses scored “Complete” had to accurately describe the difference between the population patterns for people age 60 and over in the two countries and give a plausible explanation

for the difference.

Sample “Complete” Response:

Describe the difference in population patterns for people age 60

and over in countries 1 and 2. Give one possible explanation for

the difference you have identified.
CHAPTER 6 • G E O G R A P H Y R E P O R T C A R D 107

Percentage “Partial” or better within

achievement-level intervals

Overall percentage Below Basic Basic Proficient Advanced

“Partial” or better 269 and below* 270–304* 305–338* 339 and above*

51 18 57 79 ***

Overall percentage “Partial” or better and percentages “Partial” or better within each achievement-level

range: 2001

Table 6.13b Sample Question 13 Results (“Partial” Short-Constructed-Response)

*NAEP geography composite scale range.

***Sample size is insufficient to permit a reliable estimate (see appendix A).

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education

Statistics, National Assessment of Educational Progress (NAEP), 2001 Geography Assessment.

Grade 12

Responses scored “Partial” either described the difference between the two population

pyramids but did not explain the difference or, as in the following example, incorrectly

described the difference as one of absolute numbers rather than percentages of the population and gave a plausible explanation.

Sample “Partial” Response:

Describe the difference in population patterns for people age 60

and over in countries 1 and 2. Give one possible explanation for

the difference you have identified.
108 C H A P T E R 6 • GEOGRAPHY REPORT CARD

Maps of Selected Item

Descriptions on the

NAEP Geography Scale –

Grades 4, 8, and 12

The geography performance of fourth-,

eighth-, and twelfth-graders can be illustrated by maps that position item descriptions along the NAEP geography scale

where they are likely to be answered

successfully by students.1 The descriptions

used on the item maps focus on the geography knowledge or skill needed to answer

the question. For multiple-choice questions, the description indicates the knowledge or skill demonstrated by selection of

the correct option; for constructed-response questions, the description takes into

account the knowledge or skill specified by

the different levels of scoring criteria for

that question. The questions described on

the item maps include the 12 sample

questions in the preceding section.

Figures 6.1 through 6.3 are item maps

for grades 4, 8, and 12, respectively. The

item map location of each question

identifies the scale score at which that

question was answered successfully by at

least 65 percent of the students for

constructed-response questions and 74

percent of the students for four-option,

multiple-choice questions. For each

question indicated on the item map,

students whose average score fell at or

above the scale point had a higher

probability of successfully answering the

question. Students whose average score

fell below that scale point had a lower

probability of successfully answering the

question.

As an example of how to interpret the

item maps, consider the multiple-choice

question in figure 6.1 that maps at score

point 271. This question appeared as

sample question 2 earlier in the chapter,

and was shown to have been a difficult

question answered correctly by 33 percent

of students. Students whose geography

ability corresponds to a score of 271 or

above on the scale had at least a 74 percent

probability of answering this question

correctly. Students whose ability is represented by a score below 271 had less than a

74 percent probability of answering correctly. This does not mean that all of the

former students answered the question

correctly or that all of the latter students

answered it incorrectly. Rather, the item

map indicates higher or lower probability

of answering correctly depending on

students’ overall geography ability as measured on the NAEP scale.

The three geography achievement levels

are indicated on the item map for each

grade. It is important to note that, although

the same 0-500 geography scale is used at

each grade, the achievement levels are

grade-specific, and each achievement level

begins at a different score point at each

grade. Returning to the example of the

question mapping at score point 271, the

item map is useful in showing how this

difficult question maps relatively high up

on the scale. In terms of achievement levels,

one sees that students with a 74 percent

probability of answering the question

correctly performed near the upper end of

the Proficient achievement-level range.

1 Details on the procedures used to develop item maps are provided in appendix A.
CHAPTER 6 • G E O G R A P H Y R E P O R T C A R D 109

NOTE: Regular type denotes a constructed-response question. Italic type denotes a multiple-choice question.

* Each grade 4 geography question in the 2001 assessment was mapped onto the NAEP 0–500 geography scale. The position of the question on the scale represents the scale score

attained by students who had a 65 percent probability of successfully answering a constructed-response question, or a 74 percent probability of correctly answering a four-option

multiple-choice question. Only selected questions are presented. Scale score ranges for geography achievement levels are referenced on the map. For constructed-response questions, the

question description represents students’ performance at the scoring criteria level being mapped.

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of Educational Progress (NAEP),

2001 Geography Assessment.

NAEP Geography Scale

320

310

300

290

280

270

260

250

240

230

220

210

200

190

180

170

160

Figure 6.1

Grade 4

Item Map

Map of selected item

descriptions on the

National Assessment

of Educational

Progress (NAEP)

geography scale for

grade 4

This map describes

the knowledge or skill

associated with

answering individual

geography questions.

The map identifies

the score point at

which students had a

high probability of

successfully

answering the

question.*

aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa aaaaaaaaaaaaaaaaaaaaa

Proficient

240

aaaa aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa

Advanced

276

aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa

Basic

187

320 Interpret resource map to determine likely location for large city to develop

295 Draw map based on written description of its features—Sample Question 4

288 Identify mountain range in which Switzerland is located

276 Use multiple maps to locate states where crops grow year round

271 Use map to determine which countries might have a conflict over resources—Sample Question 2

269 Use multiple maps to compare conditions for farming in two countries

259 Interpret information given in a transit map

255 Find and draw specified route on a transit system map

251 Identify a megalopolis on a population map

249 Determine elevation of a region on a physical map

244 Identify world’s largest ocean

242 Interpret a specialized map of irrigation

230 Compare climate and land use of two countries based on three maps

225 Locate bordering countries on a political map

221 Identify Mississippi River on map of North America

218 Locate place with specified features on physical map

218 Use resource map to explain where steel industry would develop

216 Recognize desert landscape in a photograph and need for irrigation to grow food

—Sample Question 1

207 Identify capital city on a political map

202 Distinguish activities associated with large cities and small towns

195 Recognize features of dry climate in a photograph

192 Identify exact or approximate location of home state on map of United States

—Sample Question 3

187 Use map to determine products traded between two countries

182 Identify some land forms on map

175 Recognize type of land use shown in photograph

161 Interpret a simple population pie chart
110 C H A P T E R 6 • GEOGRAPHY REPORT CARD

NOTE: Regular type denotes a constructed-response question. Italic type denotes a multiple-choice question.

* Each grade 8 geography question in the 2001 assessment was mapped onto the NAEP 0–500 geography scale. The position of the question on the scale represents the scale score

attained by students who had a 65 percent probability of successfully answering a constructed-response question, or a 74 percent probability of correctly answering a four-option

multiple-choice question. Only selected questions are presented. Scale score ranges for geography achievement levels are referenced on the map. For constructed-response questions, the

question description represents students’ performance at the scoring criteria level being mapped.

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of Educational Progress (NAEP),

2001 Geography Assessment.

NAEP Geography Scale

360

350

340

330

320

310

300

290

280

270

260

250

240

230

220

210

200

Figure 6.2

Grade 8

Item Map

Map of selected item

descriptions on the

National Assessment

of Educational

Progress (NAEP)

geography scale for

grade 8

This map describes

the knowledge or skill

associated with

answering individual

geography questions.

The map identifies

the score point at

which students had a

high probability of

successfully

answering the

question.*

aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa

Basic

242

aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa aaaaaaaaaaaaaaaaaaaaa

Proficient

282

aaaa aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa

Advanced

315

359 Describe and explain differences in population pyramids

348 Use a time zone map

338 Use map to explain international trade in oil

328 Explain two reasons for high rate of tropical deforestation—Sample Question 9

319 Use multiple maps to explain land use in Canada region

316 Recognize the natural forces that cause erosion—Sample Question 6

314 Use map to explain historical shift in center of U.S. population

309 Identify purpose of OPEC

309 Understand and compare different views on land ownership

303 Interpret resource map to determine likely location for large city to develop

301 Use map to help explain two reasons why early civilizations developed in Fertile Crescent

297 Use atlas to find some information about urbanization

295 Identify an economic impact of EL Niño on Peru

291 Apply concept of interior to locate capital city on political map

288 Use map to determine which countries might have a conflict over resources

285 Recognize why countries join international organizations—Sample Question 8

283 Use a political map and a land use map to locate an African city

278 Compare states using rainfall and growing season maps

271 Explain one reason for high rate of tropical deforestation

270 Understand how to read a population pyramid

267 Use map to explain one reason why early civilizations developed in Fertile Crescent

262 Recognize fault line on a map

257 Determine direct or nearly direct route between two points on a transit map—Sample Question 5

256 Identify Florida as a peninsula—Sample Question 7

250 Locate Lake Superior on map of North America

240 Identify city closest to earthquake epicenter on map

227 Use map to identify large U.S. trading partner

217 Use resource map to explain where steel industry would develop

207 Locate home state on map
CHAPTER 6 • G E O G R A P H Y R E P O R T C A R D 111

NOTE: Regular type denotes a constructed-response question. Italic type denotes a multiple-choice question.

* Each grade 12 geography question in the 2001 assessment was mapped onto the NAEP 0–500 geography scale. The position of the question on the scale represents the scale score

attained by students who had a 65 percent probability of successfully answering a constructed-response question, or a 74 percent probability of correctly answering a four-option

multiple-choice question. Only selected questions are presented. Scale score ranges for geography achievement levels are referenced on the map. For constructed-response questions, the

question description represents students’ performance at the scoring criteria level being mapped.

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of Educational Progress (NAEP),

2001 Geography Assessment.

NAEP Geography Scale

380

370

360

350

340

330

320

310

300

290

280

270

260

250

240

Figure 6.3

Grade 12

Item Map

Map of selected item

descriptions on the

National Assessment

of Educational

Progress (NAEP)

geography scale for

grade 12

This map describes

the knowledge or

skill associated with

answering individual

geography questions.

The map identifies

the score point at

which students had a

high probability of

successfully

answering the

question.*

aaaaaaaaaaaaaaaaaaaaaaa aaaaaaaaaaaaaaaaaaaaaaaaaaaaa

Advanced

339

aaaaaaaaaaaaaaaaaaaaa aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa

Proficient

305

aaaaaaaaaaaaaaaaaaaaaaaa aaaaaaaaaaaaaaaaaaaaaaaaaaaa

Basic

270

377 Use data and maps to explain Mongolia’s economic development

350 Explain reasons for international trade in oil

347 Explain differences between two countries using population pyramids—Sample Question 13

345 Use atlas to explain regional variations in land use

337 Use map to explain historical shift in center of U.S. population

333 Use map to explain economic impact of Mid-East War

331 Use multiple maps to describe regions where most Australians live

325 Explain high rate of tropical deforestation

321 Use map and charts to compare urbanization in two European countries

319 Use a time zone map

318 Identify most widely-practiced religion in India—Sample Question 11

314 Identify oil as product depicted on map of international trade

311 Use climate map to locate countries in tropical zone

305 Use map and charts to identify source of forest products for Japan

301 Identify purpose of OPEC

300 Explain reasons why Fertile Crescent was home to early civilizations—Sample Question 12

299 Use multiple maps to determine U.S. region with highest population density

295 Use map to explain geographic distribution of languages

287 Identify an economic impact of El Niño on Peru

284 Define the characteristic of a region

282 Locate natural hazards on map and explain their impact

280 Use map and data to evaluate an environmental threat

276 Explain siting of cities

272 Recognize how to read a flow map—Sample Question 10

271 Read a population pyramid

267 Draw partially accurate map based on written description

258 Identify an area of flat land on contour map

253 Recognize that Richter scale is used for measuring earthquake intensity

242 Use map to locate area likely to suffer earthquake damage

APPENDIX A • G E O G R A P H Y R E P O R T C A R D 113

A

Appendix

Contents

The Assessment

The Samples

Students with

Disabilities (SD)

and

Limited English

Proficient (LEP)

Students

Data Collection

Data Analysis

NAEP Reporting

Groups

Cautions in

Interpretations

1 National Assessment Governing Board (1994). Geography framework for the 1994 and 2001

National Assessment of Educational Progress. Washington, DC: Author.

Technical aspects of

the NAEP 2001

geography

assessment

Appendix

Focus

Appendix A

Overview of Procedures Used for the

NAEP 2001 Geography Assessment

This appendix provides an overview of the NAEP 2001

geography assessment’s primary components—framework,

development, administration, scoring, and analysis. A more

extensive review of the procedures and methods used in the

geography assessment will be included in the forthcoming

NAEP 2001 Technical Report.

The NAEP 2001 Geography Assessment

The National Assessment Governing Board (NAGB),

created by Congress in 1988, is responsible for

formulating policy for NAEP. The NAGB is

specifically charged with developing assessment

objectives and test specifications through a national

consensus approach. That consensus approach results

in the development of an assessment framework. The

design of the NAEP 2001 geography assessment

followed the guidelines provided in the framework

developed for the 1994 assessment.1

The framework underlying both the NAEP

1994 and 2001 assessments reflects consensus among

educators and researchers about the study of geography. Its

purpose is to present a comprehensive overview of the most

essential outcomes of students’ geography education.

Developing this framework and the specifications that

guided development of the assessment involved the critical
114 A P P E N D I X A • GEOGRAPHY REPORT CARD

Target and actual percentage distribution of questions by grade and geography content area, grades

4, 8, and 12: 1994 and 2001

Table A.1 Distribution of Questions

Space and Place 40 42 48 40 39 40 40 42 38

Environment and

Society 30 28 24 30 30 32 30 30 35

Spatial Dynamics

and Connections 30 31 28 30 32 28 30 29 27

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of

Educational Progress (NAEP), 1994 and 2001 Geography Assessments.

Grade 4 Grade 8 Grade 12

Actual Actual Actual Actual Actual Actual

Content Areas Target 1994 2001 Target 1994 2001 Target 1994 2001

input of hundreds of individuals across the

country, including representatives of

national education organizations, teachers,

parents, policymakers, business leaders, and

the interested general public. This consensus process was managed by the Council of

Chief State School Officers for NAGB.

The assessment framework specified not

only the particular content areas of geography to be measured (see chapter 1 for a

description of these dimensions), but also

the percentage of assessment questions that

should be devoted to each. The target

percentage distribution of content areas, as

specified in the framework, along with the

actual percentage distributions in the 1994

and 2001 assessments, are presented in table

A.1. The targeted content mix of 40

percent Space and Place, 30 percent Environment and Society, and 30 percent

Spatial Dynamics and Connections was

held constant across all three grades. The

actual content of the assessment in terms of

percentage of time spent by students was

generally within a few percentage points of

the targeted distribution in both assessment

years. Such variation across years in item

classification distribution does not affect

the reporting of trends in student performance. Trend reporting is based upon the

underlying scale, which uses the common

items (i.e., those used in both assessment

years), but maintains its stability even if

some items are dropped or replaced. Moreover, the weighting of subscales in deriving

the composite scale is based on the target

item classification distribution.

The Assessment Design

Each student who participated in the

geography assessment received a booklet

containing three or four sections: a set of

general background questions, a set of

subject-specific background questions

dealing largely with the student’s use of

technology, and one or two sets, or

“blocks,” of cognitive questions assessing

knowledge and skills in geography as

outlined in the framework. At grades 8

and 12, students were given either two 25-

minute blocks or one 50-minute block. At

grade 4, however, only 25-minute blocks

were used. At each grade, one of the 25-

minute blocks of questions required the use

of an atlas, which was provided.
APPENDIX A • G E O G R A P H Y R E P O R T C A R D 115

2 These blocks were distributed across the student booklets in a Balanced Incomplete Block (BIB) design that is

described later in this section.

At grade 4, a total of six 25-minute

blocks of cognitive questions were given,

while at grades 8 and 12, seven blocks (six

25-minute blocks and one 50-minute

block) were administered.2 Some of the

blocks at each grade (three at grade 4, and

four at grades 8 and 12) were carried

forward from the 1994 assessment to the

2001 assessment to allow for the measurement of changes across time. Each block

consisted of both multiple-choice and

constructed-response questions. Shortconstructed-response questions required a

few sentences for an answer, while

extended-constructed-response questions

generally required a paragraph or more.

Some of the constructed-response

questions required students to create

maps or graphics. It was expected that

students could adequately answer the

short-constructed-response questions in

about two to three minutes and the

extended-constructed-response questions

in about five minutes. The 50-minute

Distribution of questions administered by question type, grades 4, 8, and 12: 1994 and 2001

Table A.2 Distribution of Questions by Question Type

Multiple-choice 59 63 84 85 85 86

Short constructedresponse 23 21 32 30 25 24

Extended constructedresponse 8 7 9 9 13 13

Total 90 91 125 124 123 123

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of

Educational Progress (NAEP), 1994 and 2001 Geography Assessments.

Grade 4 Grade 8 Grade 12

1994 2001 1994 2001 1994 2001

blocks contained questions focusing on

a particular theme, and included two

extended-constructed-response questions.

Only one 50-minute block was administered at each of grades 8 and 12.

Table A.2 displays the number of questions by type and by grade level for the

1994 and 2001 assessments. Some of the

questions were used at more than one

grade level; thus, the sum of the questions

that appears at each grade level is greater

than the total number of unique questions.

The total number of questions at each

grade level varied little from 1994 to 2001,

despite the release to the public of several

blocks at each grade level and attendant

replacement with new blocks of questions.

It should be noted that any such variation

across years does not affect NAEP’s ability

to report on changes in students’ performance across years because this reporting is

based on the presence of blocks that were

common to the assessment in two years.
116 A P P E N D I X A • GEOGRAPHY REPORT CARD

3 For further details on the booklet design, see the forthcoming NAEP 2001 Technical Report.

The assessment design allowed for maximum coverage of geography content at

grades 4, 8, and 12, while minimizing the

time burden for any one student. This was

accomplished through the use of matrix

sampling of cognitive questions, in which

representative samples of students took

different portions of the entire pool of

assessment questions. The aggregate results

across the entire assessment allowed for

broad reporting of the geography performance of the targeted population. Matrix

sampling did not apply to background

questions; each student received all the

background questions appropriate for his

or her grade.

In addition to matrix sampling, the

assessment design utilized a procedure for

distributing test booklets that controlled for

position and context effects. Students

received different blocks of questions in

their booklets according to a procedure

called “Balanced Incomplete Block (BIB)

spiraling.” This procedure assigns blocks of

questions so that every block appears in the

first or second position within a booklet an

equal number of times. Every block of

questions is paired with every other block,

with the exception of the 50-minute

theme block, which appears on its own

without another block of cognitive questions. The spiraling aspect of this procedure cycles the booklets for administration,

so that typically only a few students in any

assessment session receive the same booklet.

This design allows for some balancing of

the impact of context and fatigue effects to

be measured and reported, but makes

allowance for the difficulties of administering the 50-minute blocks.3

In addition to the student assessment

booklets, three other instruments provided

data relating to the assessment: a teacher

questionnaire, a school questionnaire, and a

Students with Disabilities and/or Limited

English Proficiency (SD and/or LEP)

questionnaire. The teacher questionnaire

was administered to the geography or social

studies teachers of fourth- and eighthgrade students participating in the assessment. The questionnaire consisted of three

sections and took approximately 20 minutes to complete. The first section focused

on the teacher’s general background and

experience; the second section on computer resources available in the school; and

the third section on classroom information

about geography/social studies instruction.

The school characteristics and policy

questionnaire was given to the principal or

other administrator in each participating

school and took about 20 minutes to

complete. The questions asked about

school policies, programs, facilities, and

demographic composition and background

of the student body.

The SD and/or LEP student questionnaire was completed by a school staff

member knowledgeable about those
APPENDIX A • G E O G R A P H Y R E P O R T C A R D 117

students who were selected to participate

in the assessment and who were identified

as: 1) having an Individualized Education

Program (IEP) or equivalent program (for

reasons other than being gifted and talented) or 2) being limited English proficient (LEP). A questionnaire was completed for each SD and/or LEP student

sampled regardless of whether the student

participated in the assessment. Each questionnaire took approximately 3 minutes to

complete and asked about the student and

the special programs in which he or she

participated.

National Sample

The national results presented in this report

are based on nationally representative

probability samples of fourth-, eighth-, and

twelfth-grade students. The sample was

chosen using a multistage design that

involved sampling students from selected

schools within selected geographic areas

across the country. The sample design had

the following stages:

1) selection of geographic areas (a county,

group of counties, or metropolitan

statistical area);

2) selection of schools (public and

nonpublic) within the selected areas; and

3) selection of students within selected

schools.

4 Additional details regarding the design and structure of the national and state samples will be included in the

forthcoming NAEP 2001 Technical Report. In addition, the reader may consult the NAEP 2000 Technical Report for a

discussion of sampling procedures that are mostly common to all NAEP assessments.

Each selected school that participated in

the assessment and each student assessed

represents a portion of the population of

interest. Sampling weights are needed to

make valid inferences between the student

samples and the respective populations

from which they were drawn. Sampling

weights account for disproportionate

representation due to the oversampling of

students who attend schools with high

concentrations of Black and/or Hispanic

students and students who attend

nonpublic schools. Among other uses,

sampling weights also account for lower

sampling rates for very small schools and

are used to adjust for school and student

nonresponse.4

Unlike the 1994 national assessment, a

special feature of the 2001 national assessment was the collection of data from

samples of students where assessment

accommodations for special-needs students

were not permitted and from samples of

students where accommodations for

special-needs students were permitted.

NAEP inclusion rules were applied, and

accommodations were offered only when a

student had an Individualized Education

Program (IEP) because of a disability and/

or was identified as being a limited English

proficient student (LEP); all other students

were asked to participate in the assessment

under standard conditions.
118 A P P E N D I X A • GEOGRAPHY REPORT CARD

Table A.3 shows the number of students

included in the national samples for the

NAEP 1994 and 2001 geography assessments at each grade level. For the 2001

assessment, the table includes the number

of students in the sample where accommodations were not permitted and the number of students in the sample where

accommodations were permitted. The

table shows that the same non-SD and/or

LEP students were included in both

National student sample size by type of results, grades 4, 8, and 12: 1994 and 2001

Table A.3 National Student Sample Size

1994 2001

Accommodations Accommodations Accommodations

not permitted not permitted permitted

sample sample sample

Grade 4

Non SD/LEP students assessed 5,045 6,375

SD/LEP students assessed

without accommodations 462 551 476

SD/LEP students assessed

with accommodations NA NA 368

Total students assessed 5,507 6,926 7,219

Grade 8

Non SD/LEP students assessed 6,482 8,227

SD/LEP students assessed

without accommodations 396 721 675

SD/LEP students assessed

 with accommodations NA NA 397

Total students assessed 6,878 8,948 9,299

Grade 12

Non SD/LEP students assessed 5,944 8,477

SD/LEP students assessed

without accommodations 290 522 467

SD/LEP students assessed

with accommodations NA NA 188

Total students assessed 6,234 8,999 9,132

SD = Students with Disabilities.

LEP = Limited English Proficient students.

NA = Not applicable. No accommodations were permitted in this sample.

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of

Educational Progress (NAEP), 1994 and 2001 Geography Assessments.

samples in 2001; only the SD and/or LEP

students differed between the two samples.

The 1994 design differed somewhat in that

the SD and/or LEP students were assessed

in standard conditions and accommodations were not permitted.

Table A.4 provides a summary of the

national school and student participation

rates for the geography assessment samples

where accommodations were not permitted and where accommodations were
APPENDIX A • G E O G R A P H Y R E P O R T C A R D 119

5 The initial base sampling weights were used in weighting the percentages of participating schools and students. An

attempt was made to preselect (before field processes began) a maximum of two substitute schools for each sampled

public school (one in-district and one out-of-district) and each sampled Catholic school, and one for each sampled

nonpublic school (other than Catholic). To minimize bias, a substitute school resembled the original selection as

much as possible on affiliation, estimated number of grade-eligible students, and minority composition.

National school and student participation rates for public schools, nonpublic schools, and public

and nonpublic schools combined, grades 4, 8, and 12: 2001

Table A.4 Participation Rates

Samples where accommodations Samples where accommodations

Weighted school participation were not permitted were permitted

Student participation Overall participation rate Student participation Overall participation rate

Weighted Total Weighted Total

Percentage Percentage Total percentage number of percentage number of

before after number student students Before After student students Before After

substitution substitution of schools participation assessed substitution substitution participation assessed substitution substitution

Grade 4

Public 83 88 276 95 5,895 79 84 95 6,181 79 84

Nonpublic 83 91 89 96 1,031 80 87 96 1,038 80 88

Combined 83 88 365 95 6,926 79 84 95 7,219 79 84

Grade 8

Public 79 87 259 92 7,728 73 80 92 8,063 72 80

Nonpublic 84 88 110 96 1,232 81 84 96 1,245 80 84

Combined 79 87 369 93 8,960 74 81 92 9,308 73 80

Grade 12

Public 73 80 311 76 7,977 55 61 76 8,112 55 61

Nonpublic 67 77 63 98 1,022 66 76 91 1,021 61 70

Combined 72 80 374 77 8,999 56 62 77 9,133 56 62

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of

Educational Progress (NAEP), 2001 Geography Assessment.

permitted. Participation rates are presented

for public and nonpublic schools, individually and combined. The first rate is the

weighted percentage of schools participating in the assessment before substitution of

demographically similar schools.5 This rate

is based only on the number of schools that

were initially selected for the assessment.

The numerator of this rate is the sum of

the number of students represented by each

initially selected school that participated in

the assessment. The denominator is the

sum of the number of students represented

by each of the initially selected schools that

had eligible students enrolled.

The second school participation rate is

the weighted participation rate after substitution. The numerator of this rate is the

sum of the number of students represented

by each of the participating schools,

whether originally selected or selected as a

substitute for a school that chose not to

participate. The denominator is the same

as that for the weighted participation rate

for the initial sample. Because of the

common denominators, the weighted

participation rate after substitution is at

least as great as the weighted participation

rate before substitution.
120 A P P E N D I X A • GEOGRAPHY REPORT CARD

6 As described in the following section, a second sample in the 2001 national assessments was assessed that included

students who required and were provided with accommodations.

Also presented in table A.4 are weighted

student participation rates. The numerator

of this rate is the sum across all students

assessed (in either an initial session or a

makeup session) of the number of students

that each represents. The denominator of

this rate is the sum across all eligible

sampled students in participating schools of

the number of students that each represents. The overall participation rates take

into account the weighted percentage of

school participation before or after substitution and the weighted percentage of

student participation after makeup sessions.

For the grade 12 national sample, where

school and student response rates did not

meet NCES standards, an extensive analysis

was conducted that examined, among

other factors, the potential for nonresponse

bias at both the school and student level.

No evidence of any significant potential for

either school or student nonresponse bias

was found. Results of these analyses, as well

as nonresponse bias analyses for the grade 4

and grade 8 national samples, will be

included in the forthcoming NAEP 2001

Technical Report.

Students with Disabilities (SD)

and/or Limited English Proficient

(LEP) Students

It is NAEP’s intent to assess all selected

students from the target population.

Therefore, every effort is made to ensure

that all selected students who are capable of

participating in the assessment are assessed.

Some students sampled for participation in

NAEP can be excluded from the sample

according to carefully defined criteria.

These criteria were revised in 1996 to

communicate more clearly a presumption

of inclusion except under special circumstances. According to these criteria, students with Individualized Education

Programs (IEPs) were to be included in the

NAEP assessment except in the following

cases:

1) The school’s IEP team determined that

the student could not participate, OR,

2) The student’s cognitive functioning was

so severely impaired that she or he could

not participate, OR,

3) The student’s IEP required that the

student had to be tested with an accommodation or adaptation and that the

student could not demonstrate his or her

knowledge without that accommodation.6

All LEP students receiving academic

instruction in English for three years or

more were to be included in the assessment. Those LEP students receiving

instruction in English for fewer than three

years were to be included unless school

staff judged them to be incapable of participating in the assessment in English.

Participation of SD and/or LEP

Students in the NAEP Samples

Testing all sampled students is the best way

for NAEP to ensure that the statistics

generated by the assessment are as representative as possible of the performance of

the entire national population and the

populations of participating jurisdictions.

However, all groups of students include

certain proportions that cannot be tested in
APPENDIX A • G E O G R A P H Y R E P O R T C A R D 121

7 Office of Special Education Programs (1997). Nineteenth annual report to Congress on the implementation of the

individuals with disabilities education act. Washington, DC: U. S. Department of Education.

8 Section 504 of the Rehabilitation Act of 1973 is a civil rights law designed to prohibit discrimination on the basis of

disability in programs and activities, including education, that receive federal financial assistance.

large-scale assessments (such as students

who have profound mental disabilities), or

who can only be tested through the use of

“accommodations” such as extra time, oneon-one administration, or use of magnifying equipment.

Some students with disabilities and some

LEP students cannot show on a test what

they know and can do unless they are

provided accommodations. When such

accommodations are not allowed, students

requiring such adjustments are often

excluded from large-scale assessments such

as NAEP. This phenomenon has become

more common in the last decade and

gained momentum with the passage of the

Individuals with Disabilities Education Act

(IDEA), which led schools and states to

identify increasing proportions of students

as needing accommodations on assessments

to best show what they know and can do.7

Furthermore, Section 504 of the Rehabilitation Act of 1973 requires that, when

students with disabilities are tested, schools

must provide them with appropriate

accommodations so that the test results

accurately reflect students’ achievement.8

 In

addition, as the proportion of limited

English proficient students in the population has increased, some states have started

offering accommodations, such as translated versions of assessments or the use of

bilingual dictionaries as part of assessments.

Before 1996, NAEP did not allow any

testing under nonstandard conditions

(i.e., accommodations were not permitted).

At that time, NAEP samples were able to

include almost all sampled students in

“standard” assessment sessions. However, as

the influence of IDEA grew more widespread, the failure to provide accommodations led to increasing levels of exclusion in

the assessment. Such increases posed two

threats to the program: 1) they threatened

the stability of trend lines (because excluding more students in one year than the

next might lead to apparent rather than real

gains), and 2) they made NAEP samples

less than optimally representative of target

populations.

NAEP reacted to this challenge by

adopting a multipart strategy. It became

clear that, to ensure that NAEP samples

were as inclusive as possible, the program

had to move toward allowing the same

assessment accommodations that were

afforded students in state and district

testing programs. However, allowing

accommodations represents a change in

testing conditions that may affect measurement of changes over time. Therefore,

beginning with the 1996 national assessments and the 1998 state assessments,

NAEP has assessed a series of parallel

samples of students. In one set of samples,

testing accommodations were not permitted; this has allowed NAEP to maintain the
122 A P P E N D I X A • GEOGRAPHY REPORT CARD

9 The two samples are described as “overlapping” because in 2001 the same group of non-SD and/or LEP students

were included in both samples.

measurement of achievement trends. In

addition to the samples where accommodations were not permitted, parallel samples

in which accommodations were permitted

were also assessed. By having two overlapping samples and two sets of related data

points, NAEP could meet two core program goals.9

 First, data trends could be

maintained. Second, parallel trend lines

could be set in ways that ensure that in

future years the program will be able to use

the most inclusive practices possible and

mirror the procedures used by most state

and district assessments. Beginning in 2002,

NAEP will use only the more inclusive

samples in which assessment accommodations are permitted.

In geography, national data from 1994

and 2001 are reported for the sample in

which accommodations were not permitted. National data for the second sample,

in which accommodations were permitted,

are reported at all grades for 2001 only.

In order to make it possible to evaluate

the impact of increasing exclusion rates,

data on exclusion in both assessment years

are included in this appendix. Since the

exclusion rates may affect average scale

scores, readers should consider the magnitude of exclusion rate changes when

interpreting score changes.

Percentages of SD and/or LEP students

for the national sample where accommodations were not permitted are presented

in table A.5. The data in this table include

the percentages of students identified as SD

and/or LEP, the percentage of students

excluded, and the percentage of assessed SD

and/or LEP students. Percentages of these

students in the national sample where

accommodations were permitted are

presented in table A.6. The data in this

table include the percentages of students

identified as SD and/or LEP, the percentage

of students excluded, the percentage of

assessed SD and/or LEP students, the

percentage assessed without accommodations,

and the percentage assessed with accommodations.

In the 2001 accommodations-notpermitted national sample, 8 percent of

students at grades 4 and 8, and 5 percent of

students at grade 12 were excluded from

the assessment. The comparable percentages in the 2001 accommodations-permitted national sample were 4 percent at

grades 4 and 8, and 2 percent at grade 12.
APPENDIX A • G E O G R A P H Y R E P O R T C A R D 123

Percentage of students identified as SD and/or LEP where accommodations were not permitted,

grades 4, 8, and 12: 1994 and 2001

Table A.5 Students Identified as SD and/or LEP Where Accommodations Were Not Permitted

Grade 4

SD and/or LEP students

Identified 1,487 14 100 1,051 16 100

Excluded 1,025 5 41 500 8 48

Assessed 462 8 59 551 8 52

SD students only

Identified 974 10 100 611 11 100

Excluded 685 4 43 378 6 58

Assessed 289 6 57 233 4 42

LEP students only

Identified 546 4 100 489 6 100

Excluded 368 1 35 157 2 32

Assessed 178 3 65 332 4 68

Grade 8

SD and/or LEP students

Identified 1,674 10 100 1,379 16 100

Excluded 1,278 5 46 658 8 48

Assessed 396 5 54 721 8 52

SD students only

Identified 1,254 8 100 947 12 100

Excluded 979 4 49 546 7 54

Assessed 275 4 51 401 6 46

LEP students only

Identified 450 2 100 489 4 100

Excluded 323 1 38 153 1 31

Assessed 127 1 62 336 3 69

Grade 12

SD and/or LEP students

Identified 1,238 8 100 1,096 11 100

Excluded 948 3 43 574 5 44

Assessed 290 4 57 522 6 56

SD students only

Identified 967 6 100 772 8 100

Excluded 776 3 47 483 4 49

Assessed 191 3 53 289 4 51

LEP students only

Identified 285 2 100 373 3 100

Excluded 184 # 29 121 1 31

Assessed 101 1 71 252 2 69

# Percentage is between 0.0 and 0.5.

SD = Students with Disabilities. LEP = Limited English Proficient students.

NOTE: Within each grade level, the combined SD/LEP portion of the table is not a sum of the separate SD and LEP portions because some students were

identified as both SD and LEP. Such students would be counted separately in the bottom portions, but counted only once in the top portion.

Within each portion of the table, percentages may not sum properly due to rounding. In 1994, the geography assessment was conducted at the same time as

the 1994 U.S. history assessment. The identification and exclusion of special-needs students occurred after they were sampled, but before they could be

assigned either a history or geography session. As a consequence, the 1994 sample sizes for identified and excluded students appear larger than would be

expected given the weighted percentages that were calculations based on the geography sample only.

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of

Educational Progress (NAEP), 1994 and 2001 Geography Assessments.

1994 2001

Weighted Weighted

Number of Weighted percentage of Number of Weighted percentage of

students percentage students students percentage students

sampled of all students identified sampled of all students identified
124 A P P E N D I X A • GEOGRAPHY REPORT CARD

Grade 4

SD and/or LEP students Identified 1,137 17 100

Excluded 293 4 24

Assessed 844 13 76

Assessed without accommodations 476 6 36

Assessed with accommodations 368 7 41

SD students only Identified 641 13 100

Excluded 138 3 21

Assessed 503 10 79

Assessed without accommodations 172 3 26

Assessed with accommodations 331 7 53

LEP students only Identified 576 5 100

Excluded 175 2 31

Assessed 401 4 69

Assessed without accommodations 309 3 54

Assessed with accommodations 92 1 16

Grade 8

SD and/or LEP students Identified 1,453 16 100

Excluded 381 4 23

Assessed 1,072 12 77

Assessed without accommodations 675 7 43

Assessed with accommodations 397 5 34

SD students only Identified 996 12 100

Excluded 262 3 22

Assessed 734 10 78

Assessed without accommodations 344 4 35

Assessed with accommodations 390 5 43

LEP students only Identified 545 4 100

Excluded 140 1 27

Assessed 405 3 73

Assessed without accommodations 348 3 63

Assessed with accommodations 57 # 10

Grade 12

SD and/or LEP students Identified 956 10 100

Excluded 301 2 23

Assessed 655 8 77

Assessed without accommodations 467 5 50

Assessed with accommodations 188 3 27

SD students only Identified 652 8 100

Excluded 252 2 26

Assessed 400 6 74

Assessed without accommodations 233 3 42

Assessed with accommodations 167 3 32

LEP students only Identified 334 2 100

Excluded 63 # 17

Assessed 271 2 83

Assessed without accommodations 242 2 75

Assessed with accommodations 29 # 8

# Percentage is between 0.0 and 0.5.

SD = Students with Disabilities. LEP = Limited English Proficient students.

NOTE: Within each grade level, the combined SD/LEP portion of the table is not a sum of the separate SD and LEP portions because some students were

identified as both SD and LEP. Such students would be counted separately in the bottom portions, but counted only once in the top portion.

Within each portion of the table, percentages may not sum properly due to rounding.

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of

Educational Progress (NAEP), 2001 Geography Assessment.

Number of Weighted percentage Weighted percentage

students sampled of all students of students identified

Percentage of students identified as SD and/or LEP where accommodations were permitted,

grades 4, 8, and 12: 2001

Table A.6 Students Identified as SD and/or LEP Where Accommodations Were Permitted
APPENDIX A • G E O G R A P H Y R E P O R T C A R D 125

10 For information on DIF studies of items assessed with accommodations in the 1996 mathematics and science

assessments, see Mazzeo, J. M., Carlson, J. E., Voelkl, K. E., & Lutkus, A. D. (1999). Increasing the participation of special

needs students in NAEP; A report on 1996 NAEP research activities. Washington, DC: National Center for Education

Statistics.

Investigating the Effects of Exclusion

Rates on Assessment Results

As indicated by the data in the previous

section, exclusion rates have tended to

increase across assessment years in the

samples that did not permit accommodations. In considering the effects of exclusion rates on assessment results, at least one

major issue becomes evident. If exclusion

rates vary substantially across assessment

years, then the ability to report trends (i.e.,

compare results between years) may be

affected by the fact that the results from

different years are based on different

proportions of the population.

NCES has funded research into ways in

which excluded students might be included in the estimation of scores for total

populations and has also commissioned

studies of the impact of assessment accommodations on overall scores. Several

statistical adjustment approaches for estimating full populations (including estimates

for excluded students) have been proposed,

but none has yet been judged ready for

operational use. Regarding the impact of

assessment accommodations on overall

scores, ETS has conducted differential item

functioning (DIF) studies of items assessed

with accommodations in the 1996 assessment.10 In these studies, ETS researchers

found little evidence that accommodations

changed the functioning of test questions.

Types of Accommodations Permitted

Table A.7 displays the number and the

percentages of SD and/or LEP students

assessed with the variety of available

accommodations. It should be noted that

students assessed with accommodations

typically received some combination of

accommodations. The numbers and percentages presented in the table reflect only

the primary accommodation provided. For

example, students assessed in small groups

(as compared to standard NAEP sessions of

about 30 students) usually received

extended time. In one-on-one administrations, students often received assistance in

recording answers and were afforded extra

time. Extended time was considered the

primary accommodation only when it was

the sole accommodation provided.
126 A P P E N D I X A • GEOGRAPHY REPORT CARD

Percentage of students identified as SD and/or LEP by type of accommodation where accommodations

were permitted, grades 4, 8, and 12: 2001

Table A.7 Students Identified as SD and/or LEP by Type of Accommodation

Grade 4 Grade 8 Grade 12

Weighted Weighted Weighted

Number Weighted percentage Number Weighted percentage Number Weighted percentage

of students percentage of students of students percentage of students of students percentage of students

sampled of all students identified sampled of all students identified sampled of all students identified

SD and/or LEP students

Bilingual dictionary 41 0.38 2.2 6 0.04 0.3 21 0.14 1.4

Large-print book 3 0.02 0.1 3 0.02 0.2 3 0.05 0.5

Extended time 40 0.59 3.4 70 0.82 5.1 68 0.86 8.6

Read aloud 15 0.27 1.5 14 0.15 0.9 10 0.13 1.3

Small group 230 4.97 28.6 286 4.02 25.1 83 1.43 14.4

One-on-one 27 0.50 2.9 6 0.09 0.6 3 0.03 0.4

Scribe/computer 10 0.31 1.8 3 0.08 0.5 0 0.00 0.0

Other 2 0.03 0.2 9 0.16 1.0 0 0.00 0.0

SD students only

Bilingual dictionary 5 0.05 0.4 0 0.00 0.0 0 0.00 0.0

Large-print book 3 0.02 0.2 3 0.02 0.2 3 0.05 0.6

Extended time 40 0.59 4.7 70 0.82 6.6 68 0.86 11.1

Read aloud 15 0.27 2.1 14 0.15 1.2 10 0.13 1.7

Small group 230 4.97 39.3 286 4.02 32.4 83 1.43 18.5

One-on-one 27 0.50 3.9 6 0.09 0.7 3 0.03 0.5

Scribe/computer 10 0.31 2.5 3 0.08 0.7 0 0.00 0.0

Other 1 0.02 0.2 8 0.15 1.2 0 0.00 0.0

LEP students only

Bilingual dictionary 41 0.38 6.9 6 0.04 1.0 21 0.14 5.7

Large-print book 0 0.00 0.0 0 0.00 0.0 0 0.00 0.0

Extended time 20 0.20 3.6 20 0.13 3.0 4 0.02 0.9

Read aloud 5 0.04 0.7 0 0.00 0.0 0 0.00 0.0

Small group 21 0.21 3.8 30 0.27 6.1 4 0.04 1.7

One-on-one 3 0.03 0.5 0 0.00 0.0 0 0.00 0.0

Scribe/computer 1 0.01 0.2 0 0.00 0.0 0 0.00 0.0

Other 1 0.01 0.1 1 0.01 0.3 0 0.00 0.0

SD = Students with Disabilities. LEP = Limited English Proficient students.

NOTE: The combined SD/LEP portion of the table is not a sum of the separate SD and LEP portions because some students were identified as both SD

and LEP. Such students would be counted separately in the bottom portions, but counted only once in the top portion.

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of Educational

Progress (NAEP), 2001 Geography Assessment.
APPENDIX A • G E O G R A P H Y R E P O R T C A R D 127

11 These procedures are described more fully in the “Weighting and Variance Estimation” section later in this document. For additional information about the use of weighting procedures, see the forthcoming NAEP 2001 Technical

Report. In addition, the reader may consult the NAEP 2000 Technical Report for a discussion of weighting procedures

that are common to all NAEP assessments.

12 Lord, F. M. (1980). Applications of item response theory to practical testing problems. Hillsdale, NJ: Lawrence Erlbaum

Associates.

Data Collection and Scoring

The 2001 geography assessment was

conducted from January through March

2001, with some makeup sessions in early

April. As with all NAEP assessments, data

collection for the 2001 assessment was

conducted by a trained field staff. This was

accomplished by staff from Westat, Inc.

Materials from the 2001 assessment were

shipped to NCS Pearson, where trained

staff evaluated the responses to the constructed-response questions using scoring

rubrics or guides prepared by ETS. Each

constructed-response question had a unique

scoring rubric that defined the criteria

used to evaluate students’ responses. The

extended-constructed-response questions

were evaluated with four-level rubrics,

and almost all of the short-constructedresponse questions were rated according to

three-level rubrics that permitted partial

credit. Other short-constructed-response

questions were scored as either acceptable

or unacceptable.

For the 2001 geography assessment,

approximately 303,000 constructed

responses were scored. This number

includes rescoring to monitor inter-rater

reliability. The within-year average percentage of agreement for the 2001 national

reliability sample was 95 percent at grade 4,

94 percent at grade 8, and 93 percent at

grade 12.

Data Analysis and IRT Scaling

Subsequent to the professional scoring, all

information was transcribed to the NAEP

database at ETS. Each processing activity

was conducted with rigorous quality

control. After the assessment information

was compiled in the database, the data were

weighted according to the population

structure. The weighting for the national

sample reflected the probability of selection

for each student as a result of the sampling

design, adjusted for nonresponse. Through

post-stratification, the weighting assured

that the representation of certain subpopulations corresponded to figures from

the U.S. Census and the Current

Population Survey.11

Analyses were then conducted to determine the percentages of students who gave

various responses to each cognitive and

background question. In determining these

percentages for the cognitive questions, a

distinction was made between missing

responses at the end of a block (i.e., missing

responses subsequent to the last question

the student answered) and missing responses

prior to the last observed response. Missing

responses before the last observed response

were considered intentional omissions. In

analysis, omitted responses to multiplechoice items were scored as fractionally

correct.12 For constructed-response items,
128 A P P E N D I X A • GEOGRAPHY REPORT CARD

omitted responses were placed into the

lowest score category. Missing responses at

the end of the block were considered “not

reached” and treated as if the questions had

not been presented to the student. In

calculating response percentages for each

question, only students classified as having

been presented the question were included

in the denominator of the statistic.

It is standard NAEP practice to treat all

nonrespondents to the last question in a

block as if they had not reached the question. For multiple-choice and shortconstructed-response questions, this practice produces a reasonable pattern of results

in that the proportion reaching the last

question is not dramatically smaller than

the proportion reaching the next-to-last

question. However, for geography blocks

that ended with extended-constructedresponse questions, the standard practice

could result in extremely large drops in the

proportion of students attempting some of

the final questions. Therefore, for blocks

ending with an extended-constructedresponse question, students who answered

the next-to-last question but did not

respond to the extended-constructedresponse question were classified as having

intentionally omitted the last question.

Item Response Theory (IRT) was used

to estimate average geography scale scores

for the nation and for various subgroups of

interest within the nation. IRT models the

probability of answering a question in a

certain way as a mathematical function of

proficiency or skill. The main purpose of

IRT analysis is to provide a common scale

on which performance can be compared

across groups such as those defined by

characteristics, including gender and race/

ethnicity.

In producing the geography scales, three

distinct IRT models were used. Multiplechoice questions were scaled using the

three-parameter logistic (3PL) model;

short-constructed-response questions rated

as acceptable or unacceptable were scaled

using the two-parameter logistic (2PL)

model; and short-constructed-response

questions rated according to a three-level

rubric, as well as extended-constructedresponse questions rated on a four-level

rubric, were scaled using a Generalized

Partial-Credit (GPC) model.13 Developed

by ETS and first used in 1992, the GPC

model permits the scaling of questions

scored according to multipoint rating

schemes. The model takes full advantage of

the information available from each of the

student response categories used for these

more complex constructed-response

questions.14

The geography scale is composed of

three types of questions: multiple-choice,

short-constructed-response (scored either

dichotomously or allowing for partial

credit), and extended-constructed-response

(scored according to a partial-credit model).

Unfortunately, the question of how much

information different question-types

contribute to the geography scale has no

simple answer. The information provided

by a given question is determined by the

13 Muraki, E. (1992). A generalized partial credit model: Application of an EM algorithm. Applied Psychological

Measurement, (16)2, 159–176.

14 More detailed information regarding the IRT analyses used in NAEP assessments will be provided in the forthcoming NAEP 2001 Technical Report. In addition, the reader may consult the NAEP 2000 Technical Report for a

discussion of analysis procedures that are common to all NAEP assessments.
APPENDIX A • G E O G R A P H Y R E P O R T C A R D 129

15 Donoghue, J. R. (1994). An empirical examination of the IRT information of polytomously scored reading items

under the generalized partial credit model. Journal of Educational Measurement, (31)4, 295–311.

16 For theoretical and empirical justification of the procedures employed, see Mislevy, R. J. (1988). Randomizationbased inferences about latent variables from complex samples. Psychometrika, (56)2, 177–196.

For computational details, see the forthcoming NAEP 2001 Technical Report.

IRT model used to scale the question. It is

a function of the item parameters and

varies by level of geography proficiency.15

Thus, the answer to the query “How much

information do the different types of

questions provide?” will differ for each

level of geography performance. When

considering the composite geography scale,

the answer is even more complicated. The

geography data are scaled separately by the

three themes (space and place; environment

and society; and, spatial dynamics and

connections), resulting in three separate

subscales at each grade. The composite

scale is a weighted combination of these

subscales. IRT information functions are

only strictly comparable when the item

parameters are estimated together. Because

the composite scale is based on three

separate estimation runs, there is no direct

way to compare the information provided

by the questions on the composite scale.

Because of the BIB-spiraling design used

by NAEP, students do not receive enough

questions about a specific topic to provide

reliable information about individual

performance. (For more information on

BIB-spiraling, see “The Assessment Design”

section presented earlier in this appendix.)

Traditional test scores for individual students, even those based on IRT, would lead

to misleading estimates of population

characteristics, such as subgroup means and

percentages of students at or above a

certain scale-score level. Consequently,

NAEP constructs sets of plausible values

designed to represent the distribution of

performance in the population. A plausible

value for an individual is not a scale score

for that individual, but may be regarded as

a representative value from the distribution

of potential scale scores for all students in

the population with similar characteristics

and identical patterns of item response.

Statistics describing performance on the

NAEP geography scale are based on the

plausible values. Under the assumptions of

the scaling models, these population estimates will be consistent, in the sense that

the estimates approach the model-based

population values as the sample size

increases, which would not be the case for

population estimates obtained by aggregating optimal estimates of individual

performance.16

Item Mapping Procedures

The geography performance of fourth-,

eighth-, and twelfth-graders can be illustrated by “item maps,” which position

question or “item” descriptions along the

NAEP geography scale at each grade. Each

question shown is placed at the point on

the scale where questions are likely to be

answered successfully by students. The

descriptions used on these item maps focus

on the geography knowledge or skill

needed to answer the question. For multiplechoice questions, the description indicates
130 A P P E N D I X A • GEOGRAPHY REPORT CARD

17 Huynh, H. (1994, October). Some technical aspects of standard setting. Paper presented at the Joint Conference on Standard

Setting for Large-Scale Assessment, Washington, DC.

the knowledge or skill demonstrated by

selection of the correct option; for constructed-response questions, the description

takes into account the knowledge or skill

specified by the different levels of scoring

criteria for that question.

To map questions to particular points on

the NAEP geography scale, a response

probability convention was adopted that

would divide those who had a higher

probability of success from those who had

a lower probability. Establishing a response

probability convention has an impact on

the mapping of the test questions onto the

geography scale. A lower boundary convention maps the geography questions at

lower points along the scale, and a higher

boundary convention maps the same

questions at higher points on the scale.

The underlying distribution of geography

skills in the population does not change,

but the choice of a response probability

convention does have an impact on the

proportion of the student population that is

reported as “able to do” the questions on

the geography scales.

There is no obvious choice of a point

along the probability scale that is clearly

superior to any other point. If the convention were set with a boundary at 50 percent, those above the boundary would be

more likely to get a question right than get

it wrong, while those below the boundary

would be more likely to get the question

wrong than right. Although this convention has some intuitive appeal, it was

rejected on the grounds that having a 

50/50 chance of getting the question right

shows an insufficient degree of mastery. If

the convention were set with a boundary at

80 percent, students above the criterion

would have a high probability of success

with a question. However, many students

below this criterion show some level of

geography ability that would be ignored by

such a stringent criterion. In particular,

those in the range between 50 and 80

percent correct would be more likely to

get the question right than wrong, yet

would not be in the group described as

“able to do” the question.

In a compromise between the 50 percent and the 80 percent conventions,

NAEP has adopted two related response

probability conventions for all its subjects:

65 percent for constructed-response questions (where guessing is not a factor) and

74 percent for multiple-choice questions

(to correct for the possibility of answering

correctly by guessing). These probability

conventions were established, in part, based

on an intuitive judgment that they would

provide the best picture of students’

geography skills.

Some additional support for the dual

conventions adopted by NAEP was provided by Huynh.17 He examined the IRT

information provided by items, according

to the IRT model used in scaling NAEP

questions. (“Information” is used here in a

technical sense. See the forthcoming

NAEP 2001 Technical Report for details.)

Following Bock, Huynh decomposed the

item information into that provided by a
APPENDIX A • G E O G R A P H Y R E P O R T C A R D 131

18 Bock, R. D. (1972). Estimating item parameters and latent ability when responses are scored in two or more latent

categories. Psychometrika, 37, 29–51.

19 Donoghue, J. R. (1997, March). Item mapping to a weighted composite scale. Paper presented at the annual meeting of

the American Educational Research Association, Chicago, IL.

correct response [P(q) I(q)] and that provided by an incorrect response [(1– P(q))

I(q)].18 Huynh showed that the item

information provided by a correct response

to a constructed-response item is maximized at the point along the geography

scale at which the probability of a correct

response is 0.65 (for multiple-choice items,

the information provided by a correct

response is maximized at the point at

which the probability of getting the item

correct is 0.74). It should be noted, however, that maximizing the item information

I(q), rather than the information provided

by a correct response [P(q) I(q)], would

imply an item mapping criterion closer to

50 percent.

Results are presented in terms of the

composite geography scale. However, the

geography assessment was scaled separately

for the three themes in geography at grades

4, 8, and 12. The composite scale is a

weighted combination of the three

subscales for the three themes in geography.

To obtain item map information, a procedure developed by Donoghue was used.19

This method models the relationship

between the item response function for the

subscale and the subscale structure to

derive the relationship between the item

score and the composite scale (i.e., an item

response function for the composite scale).

This item response function is then used to

derive the probability used in the mapping.

Weighting and

Variance Estimation

A multistage sampling design was used to

select the students who were assessed.

The properties of a sample selected

through such a design could be very

different from those of a simple random

sample, in which every student in the

target population has an equal chance of

selection and in which the observations

from different sampled students can be

considered to be statistically independent

of one another. Therefore, the properties

of the sample for the data collection design

were taken into account during the analysis

of the assessment data.

One way that the properties of the

sample design were addressed was by using

sampling weights to account for the fact

that the probabilities of selection were not

identical for all students. All population

and subpopulation characteristics based on

the assessment data were estimated using

sampling weights. These weights included

adjustments for school and student

nonresponse.

Not only must appropriate estimates of

population characteristics be derived, but

appropriate measures of the degree of

uncertainty must be obtained for those

statistics. Two components of uncertainty

are accounted for in the variability of

statistics based on student ability: 1) the

uncertainty due to sampling only a rela-
132 A P P E N D I X A • GEOGRAPHY REPORT CARD

20 For further details, see Johnson, E. G. & Rust, K. F. (1992). Population inferences and variance estimation for NAEP

data. Journal of Educational Statistics, (17)2, 175–190.

tively small number of students, and 2) the

uncertainty due to sampling only a portion

of the cognitive domain of interest. The

first component accounts for the variability

associated with the estimated percentages

of students who had certain background

characteristics or who answered a certain

cognitive question correctly.

Because NAEP uses multistage sampling

procedures, conventional formulas for

estimating sampling variability that assume

simple random sampling are inappropriate.

NAEP uses a jackknife replication procedure to estimate standard errors. The

jackknife standard error provides a reasonable measure of uncertainty for any student

information that can be observed without

error. However, because each student

typically responds to only a few questions

within any theme of geography, the scale

score for any single student would be

imprecise. In this case, plausible values

methodology can be used to describe the

performance of groups and subgroups of

students. Multiple plausible values (5) are

drawn for each student in order to estimate

the variance of the posterior scale score

distribution. This component of variability

is included in the standard errors of NAEP

scale scores.20

Typically, when the standard error is

based on a small number of students or

when the group of students is enrolled in a

small number of schools, the amount of

uncertainty associated with the estimation

of standard errors may be quite large.

Estimates of standard errors subject to a

large degree of uncertainty are followed by

the “!” symbol to indicate that the nature

of the sample does not allow accurate

determination of the variability of the

statistic. In such cases, the standard errors—

and any confidence intervals or significance

tests involving these standard errors—

should be interpreted cautiously. Additional details concerning procedures for

identifying such standard errors are discussed in the forthcoming NAEP 2001

Technical Report.

Drawing Inferences

from the Results

The reported statistics are estimates and are

therefore subject to a measure of uncertainty. There are two sources of such

uncertainty. First, NAEP uses a sample of

students rather than testing all students.

Second, all assessments have some amount

of uncertainty related to the fact that they

cannot ask all questions that might be

asked in a content area. The magnitude of

this uncertainty is reflected in the standard

error of each of the estimates. When the

percentages or average scale scores of

certain groups are compared, the standard

error should be taken into account, and

observed similarities or differences should

not be relied on solely. Therefore, the

comparisons are based on statistical tests

that consider the standard errors of those

statistics and the magnitude of the difference among the averages or percentages.
APPENDIX A • G E O G R A P H Y R E P O R T C A R D 133

Using confidence intervals based on the

standard errors provides a way to take into

account the uncertainty associated with

sample estimates and to make inferences

about the population averages and percentages in a manner that reflects that uncertainty. An estimated sample average scale

score plus or minus 1.96 standard errors

approximates a 95 percent confidence

interval for the corresponding population

quantity. This statement means that one

can conclude with approximately a 95

percent level of confidence that the average

performance of the entire population of

interest (e.g., all fourth-grade students in

public and nonpublic schools) is within

plus or minus 1.96 standard errors of the

sample average.

As an example, suppose that the average

geography scale score of the students in a

particular group was 256 with a standard

error of 1.2. An approximate 95 percent

confidence interval for the population

quantity would be as follows:

Average  1.96 standard errors

256  1.96 1.2

256  2.35

(253.65, 258.35)

Thus, one can conclude with a 95

percent level of confidence that the average

scale score for the entire population of

students in that group is between 253.65

and 258.35. It should be noted that this

example, and the examples in the following

sections are illustrative. More precise

estimates carried out to one or more

decimal places are used in the actual

analyses.

Similar confidence intervals can be

constructed for percentages, if the percentages are not extremely large or extremely

small. Extreme percentages should be

interpreted with caution. Adding or

subtracting the standard errors associated

with extreme percentages could cause the

confidence interval to exceed 100 percent

or go below 0 percent, resulting in numbers that are not meaningful. The forthcoming NAEP 2001 Technical Report will

contain a more complete discussion of

extreme percentages.

Analyzing Group Differences in

Averages and Percentages

Statistical tests determine whether the

evidence, based on the data from the

groups in the sample, is strong enough to

conclude that the averages or percentages

are actually different for those groups in

the population. If the evidence is strong

(i.e., the difference is statistically significant), the report describes the group

averages or percentages as being different

(e.g., one group performed higher than or

lower than another group), regardless of

whether the sample averages or percentages

appear to be approximately the same.

The reader is cautioned to rely on the

results of the statistical tests rather than on

the apparent magnitude of the difference

between sample averages or percentages

when determining whether the sample

differences are likely to represent actual

differences among the groups in the

population.
134 A P P E N D I X A • GEOGRAPHY REPORT CARD

To determine whether a real difference

exists between the average scale scores (or

percentages of a certain attribute) for two

groups in the population, one needs to

obtain an estimate of the degree of uncertainty associated with the difference

between the averages (or percentages) of

these groups for the sample. This estimate

of the degree of uncertainty, called the

“standard error of the difference” between

the groups, is obtained by taking the square

of each group’s standard error, summing

the squared standard errors, and taking the

square root of that sum.

Standard Error of the Difference 

SEA-B  √(SEA

2  SEB2

)

Similar to how the standard error for an

individual group average or percentage is

used, the standard error of the difference

can be used to help determine whether

differences among groups in the population

are real. The difference between the

averages or percentages of the two groups

plus or minus two standard errors of the

difference represents an approximate 95

percent confidence interval. If the resulting

interval includes zero, there is insufficient

evidence to claim a real difference between

the groups in the population. If the interval

does not contain zero, the difference

between the groups is statistically significant (different) at the 0.05 level.

As an example of comparing groups,

consider the problem of determining

whether the average geography scale score

of group A is higher than that of group B.

Suppose that the sample estimates of the

average scale scores and standard errors

were as follows:

The difference between the estimates of

the average scale scores of groups A and B

is two points (218  216). The standard

error of this difference is

√(0.92  1.12)  1.4

Thus, an approximate 95 percent confidence interval for this difference is plus or

minus two standard errors of the difference

2  1.96 1.4

2  2.74

(0.74, 4.74)

The value zero is within the confidence

interval; therefore, there is insufficient

evidence to claim that group A outperformed group B.

Conducting Multiple Tests

The procedures in the previous section and

the certainty ascribed to intervals (e.g., a 95

percent confidence interval) are based on

statistical theory that assumes that only one

confidence interval or test of statistical

significance is being performed. However,

many different groups are being compared

(i.e., multiple sets of confidence intervals

are being analyzed). In sets of confidence

intervals, statistical theory indicates that the

Average

Group Scale Score Standard Error

A 218 0.9

B 216 1.1
APPENDIX A • G E O G R A P H Y R E P O R T C A R D 135

21 Miller, R. G. (1966). Simultaneous statistical inference. New York: Wiley.

22 Benjamini, Y. & Hochberg, Y. (1995). Controlling the false discovery rate: A practical and powerful approach to

multiple testing. Journal of the Royal Statistical Society, Series B, No. 1., pp 289–300.

23 Williams, V. S. L., Jones, L. V., & Tukey, J. W. (1999). Controlling error in multiple comparisons with examples from state-tostate differences in educational achievement. Journal of Educational and Behavioral Statistics, 24(1), 42–69.

certainty associated with the entire set of

intervals is less than that attributable to

each individual comparison from the set.

To hold the significance level for the set of

comparisons at a particular level (e.g., 0.05),

adjustments (called “multiple comparison

procedures”21) must be made to the methods described in the previous section. One

such procedure, the False Discovery Rate

(FDR) procedure22 was used to control the

certainty level.

Unlike the other multiple comparison

procedures (e.g., the Bonferroni procedure)

that control the familywise error rate

(i.e., the probability of making even one

false rejection in the set of comparisons),

the FDR procedure controls the expected

proportion of falsely rejected hypotheses.

Furthermore, familywise procedures are

considered conservative for large families of

comparisons.23 Therefore, the FDR procedure is more suitable for multiple comparisons in NAEP than other procedures. A

detailed description of the FDR procedure

appears in the forthcoming NAEP 2001

Technical Report.

To illustrate how the FDR procedure is

used, consider the comparisons of current

and previous years’ average geography scale

scores for the five groups presented in table

A.8. Note that the difference in average

scale scores and the standard error of the

difference are calculated in a way comparable with that of the example in the

previous section. The test statistic shown is

the difference in average scale scores

divided by the standard error of the

difference.

Example of FDR comparisons of average scale scores for different groups of students

Table A.8 FDR Comparisons of Average Scale Scores

Previous year Current year Previous year and current year

Standard

Average Standard Average Standard Difference error of Test Percent

scale score error scale score error in averages difference statistic confidence*

Group 1 224 1.3 226 1.0 2.08 1.62 1.29 20

Group 2 187 1.7 193 1.7 6.31 2.36 2.68 1

Group 3 191 2.6 197 1.7 6.63 3.08 2.15 4

Group 4 229 4.4 232 4.6 3.24 6.35 .51 62

Group 5 201 3.4 196 4.7 –5.51 5.81 –.95 35

* The percent confidence is 2(1–F(x)) where F(x) is the cumulative distribution of the t-distribution with the degrees of freedom adjusted to reflect the

complexities of the sample design.
136 A P P E N D I X A • GEOGRAPHY REPORT CARD

24 The level of confidence times the number of comparisons minus one divided by the number of comparisons is

0.05(51)/5  0.04 100  4 percent.

The difference in average scale scores

and its standard error can be used to find

an approximate 95 percent confidence

interval as in the example in the previous

section or they can be used to identify a

confidence percentage. In the example in

the previous section, because an approximate 95 percent confidence interval was

desired, the number 1.96 was used to

multiply the standard error of the difference to create the approximate confidence

interval. In the current example, the confidence interval for the test statistics is

identified from statistical tables. Instead of

checking to see if zero is within the 95

percent confidence interval about the

mean, the significance level from the

statistical tables can be directly compared

to 10095  5 percent.

If the comparison of average scale scores

across two years were made for only one of

the five groups, there would be a significant

difference between the average scale scores

for the two years if the significance level

were less than 5 percent. However, because

we are interested in the difference in

average scale scores across the two years for

all five of the groups, comparing each of

the significance levels to 5 percent is not

adequate. Groups of students defined by

shared characteristics, such as race/ethnicity

groups, are treated as sets or families when

making comparisons. However, comparisons of average scale scores for each pair of

years were treated separately. So the steps

described in this example would be replicated for the comparison of other current

and previous year average scale scores.

To use the FDR procedure to take into

account that all comparisons are of interest

to us, the percents of confidence in the

example are ordered from largest to smallest: 62, 35, 20, 4, and 1. In the FDR procedure, 62 percent confidence for the Group

4 comparison would be compared to 5

percent, 35 percent for the Group 5

comparison would be compared to

0.05(51)/5  0.04100  4 percent,24

20 percent for the Group 1 comparison

would be compared to 0.05(52)/5 

0.03100  3 percent, 4 percent for the

Group 3 comparison would be compared

to 0.05(53)/5  0.02 100  2 percent,

and 1 percent for the Group 2 comparison

(actually slightly smaller than 1 prior to

rounding) would be compared to

0.05(54)/5  0.01100  1 percent.

The last of these comparisons is the only

one for which the percent confidence is

smaller than the FDR procedure value.

The difference in the current year and

previous years’ average scale scores for the

Group 2 students is significant; for all of the

other groups, average scale scores for

current and previous year are not significantly different from one another. In

practice, a very small number of counterintuitive results occur when using the FDR

procedures to examine between-year

differences in subgroup results by jurisdiction. In those cases, results were not included in this report. NCES is continuing

to evaluate the use of FDR and multiplecomparison procedures for future reporting.
APPENDIX A • G E O G R A P H Y R E P O R T C A R D 137

25 For the national assessment, a PSU is a selected geographic region (a county, group of counties, or metropolitan

statistical area). Further details about the procedure for determining minimum sample size appear in the NAEP 2000

Technical Report and the forthcoming NAEP 2001 Technical Report.

NAEP Reporting Groups

Results are provided for groups of students

defined by shared characteristics—region

of the country, gender, race or ethnicity,

school’s type of location, eligibility for the

free/reduced-price school lunch program,

and type of school. Based on participation

rate criteria, results are reported for subpopulations only when sufficient numbers

of students and adequate school representation are present. The minimum requirement is at least 62 students in a particular

subgroup from at least five primary sampling units (PSUs).25 However, the data for

all students, regardless of whether their

subgroup was reported separately, were

included in computing overall results.

Definitions of the subpopulations are

presented below.

Region

Results in NAEP are reported for four

regions of the nation: Northeast, Southeast,

Central, and West. Figure A.1 shows how

states are subdivided into these NAEP

regions. All 50 states and the District of

Columbia are listed. Other jurisdictions,

including territories and the two Department of Defense Educational Activities

jurisdictions are not assigned to any region.

Gender

Results are reported separately for males

and females.

Connecticut

Delaware

District of Columbia

Maine

Maryland

Massachusetts

New Hampshire

New Jersey

New York

Pennsylvania

Rhode Island

Vermont

* Virginia

Alabama

Arkansas

Florida

Georgia

Kentucky

Louisiana

Mississippi

North Carolina

South Carolina

Tennessee

*Virginia

West Virginia

Illinois

Indiana

Iowa

Kansas

Michigan

Minnesota

Missouri

Nebraska

North Dakota

Ohio

South Dakota

Wisconsin

Alaska

Arizona

California

Colorado

Hawaii

Idaho

Montana

Nevada

New Mexico

Oklahoma

Oregon

Texas

Utah

Washington

Wyoming

* The part of Virginia that is included in the Northeast region is the Washington, DC metropolitan area; the remainder of the state is included in the Southeast

region.

Northeast Southeast Central West

Figure A.1

States by

Region

States included in the four NAEP regions: 2001
138 A P P E N D I X A • GEOGRAPHY REPORT CARD

Race/Ethnicity

The race/ethnicity variable is derived from

two questions asked of students and from

school records, and it is used for race/

ethnicity subgroup comparisons. Two

questions from the set of general student

background questions were used to determine race/ethnicity:

If you are Hispanic, what is your Hispanic

background?

❏ I am not Hispanic

❏ Mexican, Mexican American, or Chicano

❏ Puerto Rican

❏ Cuban

❏ Other Spanish or Hispanic background

Students who responded to this question

by filling in the second, third, fourth, or

fifth oval were considered Hispanic. For

students who filled in the first oval, did not

respond to the question, or provided

information that was illegible or could not

be classified, responses to the following

question were examined to determine their

race/ethnicity.

Which best describes you?

❏ White (not Hispanic)

❏ Black (not Hispanic)

❏ Hispanic (“Hispanic” means someone

who is Mexican, Mexican American,

Chicano, Puerto Rican, Cuban, or other

Spanish or Hispanic background)

❏ Asian or Pacific Islander (“Asian or

Pacific Islander” means someone who is

from a Chinese, Japanese, Korean,

Filipino, Vietnamese, Asian American or

some other Asian or Pacific Islander

background.)

❏ American Indian or Alaskan Native

(“American Indian or Alaskan Native”

means someone who is from one of the

American Indian tribes or one of the

original people of Alaska.)

❏ Other (specify)

Students’ race/ethnicity was then assigned

on the basis of their responses. For students

who filled in the sixth oval (“Other”),

provided illegible information or information that could not be classified, or did not

respond at all, race/ethnicity was assigned

as determined by school records.

Race/ethnicity could not be determined

for students who did not respond to either

of the demographic questions and whose

schools did not provide information about

race/ethnicity.

Also, some students indicated that they

were from a Hispanic background

(e.g., Puerto Rican or Cuban) and that a

racial/ethnic category other than Hispanic

best described them. These students were

classified as Hispanic based on the rules

described above.

Type of Location

Results from the 2001 assessment are

reported for students attending schools in

three mutually exclusive location types:

central city, urban fringe/large town, and

rural/small town:

Central City: This category includes central

cities of all Standard Metropolitan Statistical Areas (SMSA) as defined by the Office

of Management and Budget. Central City

is a geographical term and is not synonymous with “inner city.”

Urban Fringe/Large Town: The urban fringe

category includes all densely settled places

and areas within SMSA’s that are classified

as urban by the Bureau of the Census, but

which do not qualify as Central City. A

Large Town is defined as a place outside a

SMSA with a population greater than or

equal to 25,000.
APPENDIX A • G E O G R A P H Y R E P O R T C A R D 139

Rural/Small Town: Rural includes all places

and areas with populations of less than

2,500 that are classified as rural by the

Bureau of the Census. A Small Town is

defined as a place outside a SMSA with a

population of less than 25,000, but greater

than or equal to 2,500.

Results for each type of location are not

compared across years. This was due to

new methods used by NCES to identify

the type of location assigned to each school

in the Common Core of Data (CCD).

The new methods were put into place by

NCES in order to improve the quality of

the assignments and they take into account

more information about the exact physical

location of the school. The variable was

revised in NAEP beginning with the 2000

assessments.

Eligibility for the Free/Reduced-Price

School Lunch Program

Based on available school records, students

were classified as either currently eligible

for the free/reduced-price school lunch

component of the Department of Agriculture’s National School Lunch Program

or not eligible. Eligibility for the program

is determined by students’ family income

in relation to the federally established

poverty level. Free lunch qualification is set

at 130 percent of the poverty level, and

reduced-price lunch qualification is set at

170 percent of the poverty level. The

classification applies only to the school year

when the assessment was administered

(i.e., the 2000–2001 school year) and is not

based on eligibility in previous years. If

school records were not available, the

student was classified as “Information not

available.” If the school did not participate

in the program, all students in that school

were classified as “Information not

available.”

Type of School

Results are reported by the type of school

that the student attends—public or nonpublic. Nonpublic schools include Catholic

and other private schools.26 Because they

are funded by federal authorities, not state/

local governments, Bureau of Indian Affairs

(BIA) schools and Department of Defense

Domestic Dependent Elementary and

Secondary Schools (DDESS) are not

included in either the public or nonpublic

categories; they are included in the overall

national results.

Grade 12 Participation Rates

NAEP has been described as a “low-stakes”

assessment. That is, students receive no

individual scores, and their NAEP performance has no effect on their grades, promotions, or graduation. There has been

continued concern that this lack of consequences affects participation rates of students and schools, as well as the motivation

of students to perform well on NAEP. Of

particular concern has been the performance of twelfth-graders, who typically

have lower student participation rates than

fourth- and eighth-graders, and who are

more likely to omit responses compared to

the younger cohorts.

26 Through a pilot study, more detailed breakdowns of nonpublic school results are available on the NAEP Web Site

(http://nces.ed.gov/nationsreportcard/geography/results/index.asp).
140 A P P E N D I X A • GEOGRAPHY REPORT CARD

In NAEP, there has been a consistent

pattern of lower participation rates for

older students. In the 2001 NAEP assessments, for example, the student participation rates were 95 percent and 93 percent at grades 4 and 8, respectively. At

grade 12, however, the participation rate

was 77 percent. School participation rates

(the percentage of sampled schools that

participated in the assessment) have also

typically decreased with grade level. Again

citing the 2001 assessments, the school

participation rate was 88 percent for the

fourth grade, 87 percent for the eighth

grade, and 80 percent for the twelfth grade.

The effect of participation rates on

student performance, however, is unclear.

Students may choose not to participate in

NAEP for many reasons, such as desire to

attend regular classes so as not to miss

important instruction or conflict with

other school-based activities. Similarly,

there are a variety of reasons for which

various schools do not participate. The

sampling weights and nonresponse adjustments, described earlier in this document,

provide an approximate statistical adjustment for nonparticipation. However, the

effect of some school and student nonparticipation may have some undetermined

effect on results.

More research is needed to delineate the

factors that contribute to nonparticipation

and lack of motivation. To that end, NCES

is currently investigating how various

types of incentives can be effectively used

to increase participation in NAEP. One

report that examines the impact of monetary incentives on student effort and

performance is available on the NCES Web

Site at http://nces.ed.gov/pubsearch/.

Enter NCES#: 2001024.

Cautions in Interpretations

As described earlier, the NAEP geography

scale makes it possible to examine relationships between students’ performance and

various background factors measured by

NAEP. However, a relationship that exists

between achievement and another variable

does not reveal its underlying cause, which

may be influenced by a number of other

variables. Similarly, the assessments do not

capture the influence of unmeasured

variables. The results are most useful when

they are considered in combination with

other knowledge about the student population and the educational system, such as

trends in instruction, changes in the

school-age population, and societal

demands and expectations.
APPENDIX B • G E O G R A P H Y R E P O R T C A R D 141

Appendix B

Data Appendix

This appendix contains complete data for all the tables and

figures presented in this report, including average scores,

achievement-level results, and percentages of students. In

addition, standard errors appear in parentheses next to each

scale score and percentage. The comparisons presented in

this report are based on statistical tests that consider the

magnitude of the difference between group averages

or percentages and the standard errors of those

statistics. Because NAEP scores and percentages are

based on samples rather than the entire population(s),

the results are subject to a measure of uncertainty

reflected in the standard errors of the estimates. It can

be said with 95 percent certainty that for each

population of interest, the value for the whole

population is within plus or minus two standard

errors of the estimate for the sample. As with the

figures and tables in the chapters, significant

differences between results of previous assessments

and the 2001 assessment are highlighted.

B

Appendix

Contents

Average Scores

AchievementLevel Results

Percentages of

Students

Standard Errors

Complete data

for all tables

and figures.

Appendix

Focus
142 A P P E N D I X B • GEOGRAPHY REPORT CARD

Average geography scale scores, grades 4, 8, and 12: 1994 and 2001

Grade 4 Grade 8 Grade 12

1994 206 (1.2) 260 (0.7) 285 (0.7)

2001 209 (1.0) * 262 (0.9) * 285 (0.8)

Standard errors of the estimated scale scores appear in parentheses.

* Significantly different from 1994.

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of

Educational Progress (NAEP), 1994 and 2001 Geography Assessments.

Table B.1: Data for Figure 2.1 National Scale Score Results

National geography scale score percentiles, grades 4, 8, and 12: 1994 and 2001

Table B.2: Data for Figure 2.2: National Performance Distribution

10th 25th 50th 75th 90th

Grade 4 1994 146 (1.9) 179 (1.5) 211 (1.1) 237 (1.3) 257 (2.0)

2001 158 (1.7) * 185 (1.9) * 212 (1.1) 236 (1.0) 254 (0.9)

Grade 8 1994 213 (1.3) 237 (1.0) 263 (1.1) 285 (0.9) 302 (1.9)

2001 217 (1.0) * 241 (0.9) * 265 (1.1) 286 (0.9) 303 (1.2)

Grade 12 1994 244 (0.9) 265 (1.1) 287 (0.9) 306 (1.0) 321 (1.0)

2001 247 (1.7) 267 (1.2) 287 (0.9) 305 (0.9) 319 (1.0)

Standard errors of the estimated scale scores appear in parentheses.

* Significantly different from 1994.

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of

Educational Progress (NAEP), 1994 and 2001 Geography Assessments.
APPENDIX B • G E O G R A P H Y R E P O R T C A R D 143

Table B.3: Data for Figure 2.3: National Achievement-Level Results

Percentage of students within and at or above geography achievement levels, grades 4, 8, and 12:

1994 and 2001

At or above At or above

Below Basic At Basic At Proficient At Advanced Basic Proficient

Grade 4 1994 30 (1.1) 48 (1.0) 19 (1.1) 3 (0.4) 70 (1.1) 22 (1.2)

2001 26 (1.2) * 53 (1.4) * 19 (1.1) 2 (0.3) 74 (1.2) * 21 (1.0)

Grade 8 1994 29 (1.0) 43 (1.1) 24 (0.9) 4 (0.4) 71 (1.0) 28 (1.0)

2001 26 (0.9) * 44 (0.9) 26 (1.1) 4 (0.6) 74 (0.9) * 30 (1.2)

Grade 12 1994 30 (0.9) 43 (1.0) 25 (1.0) 2 (0.5) 70 (0.9) 27 (1.2)

2001 29 (0.9) 47 (0.9) * 23 (1.0) 1 (0.3) 71 (0.9) 25 (1.1)

Standard errors of the estimated percentages appear in parentheses.

* Significantly different from 1994.

NOTE: Percentages within each geography achievement-level range may not add to 100, or to the exact percentages at or above achievement levels, due to rounding.

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of

Educational Progress (NAEP), 1994 and 2001 Geography Assessments.
144 A P P E N D I X B • GEOGRAPHY REPORT CARD

Table B.5: Data for Figure 3.2 National Scale Score Differences by Gender

Differences in average geography scale scores by gender, grades 4, 8, and 12: 1994 and 2001

Standard errors of the estimated difference in scale scores appear in parentheses.

Score differences are calculated based on differences between unrounded average scale scores.

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of

Educational Progress (NAEP), 1994 and 2001 Geography Assessments.

Male-Female

Grade 4 1994 5 (2.0)

2001 5 (1.7)

Grade 8 1994 4 (1.2)

2001 4 (1.4)

Grade 12 1994 7 (1.2)

2001 4 (1.2)

Table B.4: Data for Figure 3.1 National Scale Score Results by Gender

Percentage of students and average geography scale scores by gender, grades 4, 8, and 12:

1994 and 2001

Male Female

Grade 4 1994 51 (1.0) 49 (1.0)

208 (1.4) 203 (1.4)

2001 51 (0.8) 49 (0.8)

212 (1.1) 207 (1.2)

Grade 8 1994 51 (0.7) 49 (0.7)

262 (0.9) 258 (0.8)

2001 51 (0.6) 49 (0.6)

264 (1.0) 260 (1.1)

Grade 12 1994 50 (1.0) 50 (1.0)

288 (0.8) 281 (0.9)

2001 48 (0.8) 52 (0.8)

287 (0.9) 282 (0.8)

The percentage of students is listed first with the corresponding average scale score presented below.

Standard errors of the estimated percentages and scale scores appear in parentheses.

NOTE: Percentages may not add to 100 due to rounding.

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of

Educational Progress (NAEP), 1994 and 2001 Geography Assessments.
APPENDIX B • G E O G R A P H Y R E P O R T C A R D 145

Table B.6: Data for Figure 3.3 National Achievement-Level Results by Gender

Percentage of students within and at or above geography achievement levels by gender,

grades 4, 8, and 12: 1994 and 2001

At or above At or above

Below Basic At Basic At Proficient At Advanced Basic Proficient

Grade 4 Male 1994 29 (1.3) 46 (1.4) 22 (1.6) 4 (0.7) 71 (1.3) 26 (1.7)

2001 25 (1.3) 51 (1.6) * 21 (1.4) 3 (0.5) 75 (1.3) 24 (1.4)

Female 1994 32 (1.4) 49 (1.3) 17 (1.2) 2 (0.5) 68 (1.4) 19 (1.3)

2001 28 (1.6) 54 (1.7) * 17 (1.2) 1 (0.4) 72 (1.6) 18 (1.1)

Grade 8 Male 1994 28 (1.3) 42 (1.4) 25 (1.2) 5 (0.6) 72 (1.3) 30 (1.2)

2001 25 (1.0) 42 (1.3) 29 (1.7) 5 (0.7) 75 (1.0) 33 (1.5)

Female 1994 31 (1.1) 44 (1.2) 22 (1.2) 3 (0.4) 69 (1.1) 25 (1.1)

2001 27 (1.2) 47 (1.1) 24 (1.3) 3 (0.6) 73 (1.2) 26 (1.4)

Grade 12 Male 1994 27 (1.1) 41 (1.1) 29 (1.1) 2 (0.7) 73 (1.1) 32 (1.4)

2001 27 (1.1) 45 (1.3) * 26 (1.4) 2 (0.4) 73 (1.1) 28 (1.5)

Female 1994 33 (1.2) 45 (1.5) 21 (1.2) 1 (0.4) 67 (1.2) 22 (1.4)

2001 30 (1.0) 48 (1.0) * 20 (0.9) 1 (0.3) 70 (1.0) 21 (1.0)

Standard errors of the estimated percentages appear in parentheses.

* Significantly different from 1994.

NOTE: Percentages within each geography achievement-level range may not add to 100, or to the exact percentages at or above achievement levels, due to rounding.

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of

Educational Progress (NAEP), 1994 and 2001 Geography Assessments.
146 A P P E N D I X B • GEOGRAPHY REPORT CARD

Table B.8: Data for Figure 3.5 National Scale Score Differences by Race/Ethnicity

Differences in average geography scale scores by race/ethnicity, grades 4, 8, and 12: 1994 and 2001

White-Black White-Hispanic

Grade 4 1994 50 (2.9) 35 (2.9)

2001 40 (2.0) * 38 (3.0)

Grade 8 1994 40 (1.9) 31 (2.0)

2001 38 (2.0) 33 (2.0)

Grade 12 1994 33 (1.6) 23 (1.7)

2001 32 (1.7) 21 (1.8)

Standard errors of the estimated difference in scale scores appear in parentheses.

*Significantly different from 1994.

Score differences are calculated based on differences between unrounded average scale scores.

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of

Educational Progress (NAEP), 1994 and 2001 Geography Assessments.

Table B.7: Data for Figure 3.4 National Scale Score Results by Race/Ethnicity

Percentage of students and average geography scale scores by race/ethnicity, grades 4, 8, and 12:

1994 and 2001

Asian/ American

White Black Hispanic Pacific Islander Indian

Grade 4 1994 69 (0.2) 15 (0.1) 12 (0.2) 3 (0.2) 1 (0.2)

218 (1.5) 168 (2.5) 183 (2.5) 214 (3.8) 193 (3.6)

2001 64 (0.3) 14 (0.2) 16 (0.3) 3 (0.2) 2 (0.2)

222 (1.0) 181 (1.8) * 184 (2.8) 212 (2.7) 199 (3.6)

Grade 8 1994 69 (0.2) 15 (0.1) 11 (0.1) 3 (0.4) 2 (0.4)

270 (0.8) 229 (1.7) 239 (1.9) 264 (5.2) 248 (3.4) !

2001 66 (0.3) 14 (0.2) 14 (0.2) 4 (0.2) 1 (0.2)

273 (1.0) 234 (1.7) 240 (1.7) 266 (2.5) 261 (5.8)

Grade 12 1994 74 (0.3) 12 (0.4) 8 (0.2) 4 (0.2) 1 (0.2)

291 (0.8) 258 (1.4) 268 (1.5) 285 (2.7) *** (***)

2001 70 (0.3) 13 (0.3) 12 (0.2) 5 (0.2) 1 (0.2)

291 (0.9) 260 (1.4) 270 (1.5) 286 (2.9) 288 (3.6) !

The percentage of students is listed first with the corresponding average scale score presented below.

Standard errors of the estimated percentages and scale scores appear in parentheses.

* Significantly different from 1994.

! The nature of the sample does not allow accurate determination of the variability of the statistic.

***(***) Sample size is insufficient to permit a reliable estimate.

NOTE: Percentages may not add to 100 due to rounding.

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of

Educational Progress (NAEP), 1994 and 2001 Geography Assessments.
APPENDIX B • G E O G R A P H Y R E P O R T C A R D 147

At or above At or above

Below Basic At Basic At Proficient At Advanced Basic Proficient

Table B.9: Data for Figure 3.6a, b, c National Achievement-Level Results by Race/Ethnicity

Percentage of students within and at or above geography achievement levels by race/ethnicity,

grades 4, 8, and 12: 1994 and 2001

Grade 4 White 1994 19 (1.3) 53 (1.3) 25 (1.5) 4 (0.6) 81 (1.3) 29 (1.6)

2001 13 (1.3) * 58 (1.8) 26 (1.6) 3 (0.5) 87 (1.3) * 29 (1.5)

Black 1994 66 (2.4) 32 (2.4) 2 (0.6) # (***) 34 (2.4) 3 (0.6)

2001 56 (2.1) * 39 (2.1) 5 (0.8) # (***) 44 (2.1) * 5 (0.9)

Hispanic 1994 51 (2.7) 39 (2.0) 9 (1.7) 1 (0.4) 49 (2.7) 10 (1.7)

2001 51 (3.0) 43 (2.5) 6 (1.0) # (***) 49 (3.0) 6 (1.0)

Asian/Pacific Islander 1994 24 (4.0) 49 (4.3) 23 (3.9) 4 (2.2) 76 (4.0) 27 (4.4)

2001 23 (3.4) 52 (4.4) 23 (3.1) 1 (0.9) 77 (3.4) 25 (3.0)

American Indian 1994 38 (5.7) 53 (5.8) 9 (3.6) # (***) 62 (5.7) 9 (3.9)

2001 34 (4.9) 53 (6.3) 13 (4.2) # (***) 66 (4.9) 13 (4.1)

Grade 8 White 1994 18 (0.9) 47 (1.2) 30 (1.2) 5 (0.5) 82 (0.9) 36 (1.3)

2001 14 (0.9) * 48 (1.2) 34 (1.5) 5 (0.8) 86 (0.9) * 39 (1.7)

Black 1994 66 (2.9) 30 (2.8) 4 (0.7) # (0.3) 34 (2.9) 5 (0.7)

2001 60 (2.3) 34 (1.9) 6 (0.8) # (***) 40 (2.3) 6 (0.8)

Hispanic 1994 50 (3.6) 39 (3.1) 10 (1.2) 1 (0.5) 50 (3.6) 10 (1.2)

2001 52 (1.9) 38 (1.6) 9 (1.1) 1 (0.2) 48 (1.9) 10 (1.0)

Asian/Pacific Islander 1994 27 (6.3) 43 (4.4) 25 (3.1) 6 (3.1) 73 (6.3) 30 (4.2)

2001 21 (3.4) 47 (4.8) 28 (3.5) 4 (1.8) 79 (3.4) 32 (3.2)

American Indian 1994 41 (5.1) ! 43 (4.9) ! 13 (3.5) ! 2 (1.3) ! 59 (5.1) ! 15 (3.6) !

2001 28 (6.8) 41 (11.1) 29 (8.9) 3 (***) 72 (6.8) 31 (11.2)

Grade 12 White 1994 22 (0.9) 46 (1.3) 31 (1.2) 2 (0.6) 78 (0.9) 33 (1.5)

2001 19 (0.9) 51 (1.1) * 29 (1.2) 2 (0.4) 81 (0.9) 31 (1.4)

Black 1994 68 (2.3) 27 (2.1) 5 (1.0) # (***) 32 (2.3) 5 (1.0)

2001 65 (2.3) 31 (2.1) 4 (0.7) # (***) 35 (2.3) 4 (0.7)

Hispanic 1994 52 (2.8) 38 (2.4) 10 (1.7) # (***) 48 (2.8) 10 (1.8)

2001 48 (2.6) 42 (2.5) 10 (1.4) # (0.1) 52 (2.6) 10 (1.4)

Asian/Pacific Islander 1994 31 (3.1) 41 (3.4) 25 (4.0) 3 (1.3) 69 (3.1) 28 (4.4)

2001 28 (4.3) 45 (3.0) 25 (4.6) 1 (0.7) 72 (4.3) 26 (4.7)

American Indian 1994 *** (***) *** (***) *** (***) *** (***) *** (***) *** (***)

2001 26 (6.0) ! 41 (7.0) ! 31 (5.3) ! 1 (***) ! 74 (6.0) ! 32 (4.9) !

Standard errors of the estimated percentages appear in parentheses.

* Significantly different from 1994.

# Percentage is between 0.0 and 0.5.

! The nature of the sample does not allow accurate determination of the variability of the statistic.

(***) Standard error estimates cannot be accurately determined.

***(***) Sample size is insufficient to permit a reliable estimate.

NOTE: Percentages within each geography achievement-level range may not add to 100, or to the exact percentages at or above achievement levels, due to rounding.

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of

Educational Progress (NAEP), 1994 and 2001 Geography Assessments.
148 A P P E N D I X B • GEOGRAPHY REPORT CARD

Table B.10: Data for Figure 3.7 National Scale Score Results by Region of the Country

Percentage of students and average geography scale scores by region of the country,

grades 4, 8, and 12: 1994 and 2001

Northeast Southeast Central West

Grade 4 1994 22 (0.8) 23 (1.0) 25 (0.8) 30 (0.7)

203 (2.7) 200 (2.5) 215 (3.2) 205 (1.7)

2001 21 (0.8) 24 (1.3) 24 (0.4) 31 (1.3)

214 (2.8) * 207 (2.1) 219 (1.8) 200 (2.5)

Grade 8 1994 20 (0.8) 25 (1.0) 24 (0.6) 31 (0.7)

266 (1.9) 252 (1.6) 268 (1.6) 255 (1.8)

2001 21 (0.8) 22 (1.0) 25 (0.6) 32 (1.2)

266 (2.4) 260 (2.0) * 270 (2.5) 255 (1.5)

Grade 12 1994 21 (0.5) 23 (0.8) 28 (0.7) 29 (0.7)

284 (1.6) 278 (1.1) 289 (1.8) 286 (1.9)

2001 20 (0.9) 21 (1.2) 27 (0.6) 31 (1.4)

286 (2.8) 281 (1.0) 287 (1.3) 283 (1.3)

The percentage of students is listed first with the corresponding average scale score presented below.

Standard errors of the estimated percentages and scale scores appear in parentheses.

* Significantly different from 1994.

NOTE: Percentages may not add to 100 due to rounding.

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of

Educational Progress (NAEP), 1994 and 2001 Geography Assessments.
APPENDIX B • G E O G R A P H Y R E P O R T C A R D 149

At or above At or above

Below Basic At Basic At Proficient At Advanced Basic Proficient

Table B.11: Data for Figure 3.8a, b, c National Achievement-Level Results by Region of the Country

Percentage of students within and at or above geography achievement levels by region of the

country, grades 4, 8, and 12: 1994 and 2001

Grade 4 Northeast 1994 33 (2.7) 45 (2.5) 19 (2.1) 3 (0.8) 67 (2.7) 22 (2.5)

2001 22 (3.7) 54 (3.7) 22 (2.1) 3 (0.9) 78 (3.7) 24 (2.2)

Southeast 1994 36 (2.6) 48 (2.2) 14 (1.9) 2 (0.5) 64 (2.6) 17 (2.0)

2001 28 (2.5) 54 (2.7) 17 (1.8) 1 (0.6) 72 (2.5) 18 (1.9)

Central 1994 22 (2.6) 49 (2.3) 25 (3.2) 4 (1.3) 78 (2.6) 28 (3.3)

2001 18 (1.7) 51 (1.8) 27 (2.3) 3 (0.7) 82 (1.7) 30 (2.5)

West 1994 30 (1.7) 48 (1.9) 19 (2.0) 3 (0.6) 70 (1.7) 21 (1.7)

2001 34 (2.7) 52 (2.4) 13 (1.5) 1 (0.3) 66 (2.7) 14 (1.7) *

Grade 8 Northeast 1994 24 (2.2) 43 (1.6) 28 (1.8) 6 (1.0) 76 (2.2) 33 (2.0)

2001 22 (2.5) 44 (2.1) 29 (3.2) 4 (1.3) 78 (2.5) 34 (3.3)

Southeast 1994 38 (2.1) 40 (1.8) 19 (1.4) 3 (0.5) 62 (2.1) 21 (1.6)

2001 27 (2.4) * 46 (1.7) 24 (1.5) 3 (0.6) 73 (2.4) * 26 (1.6)

Central 1994 20 (1.7) 44 (1.9) 30 (1.9) 6 (0.9) 80 (1.7) 36 (2.1)

2001 18 (2.3) 43 (2.4) 32 (3.1) 6 (1.3) 82 (2.3) 38 (3.7)

West 1994 33 (2.4) 45 (1.8) 20 (1.8) 3 (0.7) 67 (2.4) 23 (2.0)

2001 34 (1.7) 44 (1.7) 21 (1.6) 2 (0.6) 66 (1.7) 23 (1.7)

Grade 12 Northeast 1994 31 (2.3) 44 (2.1) 23 (1.8) 2 (0.6) 69 (2.3) 25 (2.1)

2001 29 (2.3) 46 (2.4) 24 (3.2) 2 (1.1) 71 (2.3) 26 (4.1)

Southeast 1994 40 (1.5) 41 (1.5) 19 (1.4) 1 (0.6) 60 (1.5) 20 (1.3)

2001 33 (1.6) * 46 (1.4) 20 (1.2) 1 (0.3) 67 (1.6) * 21 (1.3)

Central 1994 25 (2.1) 43 (2.3) 30 (2.2) 2 (1.0) 75 (2.1) 32 (2.9)

2001 24 (1.8) 48 (1.8) 27 (1.8) 1 (0.5) 76 (1.8) 28 (1.9)

West 1994 28 (2.1) 43 (1.6) 27 (2.3) 2 (0.7) 72 (2.1) 29 (2.6)

2001 30 (1.9) 47 (1.7) 22 (1.7) 1 (0.4) 70 (1.9) 23 (1.8)

Standard errors of the estimated percentages appear in parentheses.

* Significantly different from 1994.

NOTE: Percentages within each geography achievement-level range may not add to 100, or to the exact percentages at or above achievement levels, due to rounding.

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of

Educational Progress (NAEP), 1994 and 2001 Geography Assessments.
150 A P P E N D I X B • GEOGRAPHY REPORT CARD

Table B.12: Data for Figure 3.9 National Scale Score Results by Parents’ Education

Percentage of students and average geography scale scores by parents’ highest level of education,

grades 8 and 12: 1994 and 2001

Some education

Less than Graduated after Graduated

high school high school high school college Unknown

Grade 8 1994 7 (0.5) 22 (0.9) 19 (0.7) 42 (1.2) 10 (0.5)

238 (1.7) 250 (1.2) 265 (1.0) 272 (1.0) 234 (1.5)

2001 6 (0.4) 18 (0.5) 19 (0.6) 48 (1.2) 9 (0.6)

241 (1.7) 253 (1.2) 266 (1.0) 274 (0.9) 245 (1.5) *

Grade 12 1994 7 (0.4) 22 (0.8) 25 (0.7) 44 (1.2) 3 (0.2)

263 (1.2) 274 (1.1) 286 (1.0) 294 (0.9) 257 (2.8)

2001 7 (0.4) 19 (0.7) 25 (0.7) 46 (1.1) 3 (0.3)

269 (1.7) * 276 (0.9) 284 (0.9) 293 (1.1) 257 (2.9)

The percentage of students is listed first with the corresponding average scale score presented below.

Standard errors of the estimated percentages and scale scores appear in parentheses.

* Significantly different from 1994.

NOTE: Percentages may not add to 100 due to rounding.

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of

Educational Progress (NAEP), 1994 and 2001 Geography Assessments.
APPENDIX B • G E O G R A P H Y R E P O R T C A R D 151

Table B.13: Data for Figure 3.10a, b National Achievement-Level Results by Parents’ Education

Percentage of students within and at or above geography achievement levels by parents’ highest

level of education, grades 8 and 12: 1994 and 2001

At or above At or above

Below Basic At Basic At Proficient At Advanced Basic Proficient

Standard errors of the estimated percentages appear in parentheses.

* Significantly different from 1994.

# Percentage is between 0.0 and 0.5.

(***) Standard error estimates cannot be accurately determined.

NOTE: Percentages within each geography achievement-level range may not add to 100, or to the exact percentages at or above achievement levels, due to rounding.

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of

Educational Progress (NAEP), 1994 and 2001 Geography Assessments.

Grade 8

Less than high school 1994 53 (3.4) 39 (3.4) 8 (1.4) 1 (***) 47 (3.4) 8 (1.6)

2001 52 (2.6) 41 (3.6) 8 (2.6) # (***) 48 (2.6) 8 (2.6)

Graduated high school 1994 38 (2.0) 47 (2.1) 14 (1.6) 1 (0.6) 62 (2.0) 15 (1.5)

2001 34 (2.1) 48 (1.5) 16 (2.1) 1 (0.6) 66 (2.1) 18 (1.9)

Some education after high school 1994 21 (1.3) 50 (2.7) 26 (2.5) 3 (0.8) 79 (1.3) 29 (2.3)

2001 20 (1.6) 51 (1.8) 27 (1.7) 2 (0.8) 80 (1.6) 30 (1.8)

Graduated College 1994 18 (1.2) 41 (1.4) 34 (1.3) 7 (0.7) 82 (1.2) 41 (1.4)

2001 14 (0.9) 42 (1.5) 37 (1.3) 6 (0.9) 86 (0.9) 43 (1.5)

Unknown 1994 56 (2.9) 35 (3.2) 8 (1.6) 1 (***) 44 (2.9) 8 (1.5)

2001 44 (2.6) * 44 (3.1) 11 (1.9) 1 (0.5) 56 (2.6) * 12 (1.9)

Grade 12

Less than high school 1994 59 (2.9) 34 (3.5) 7 (2.0) 0 (***) 41 (2.9) 7 (2.0)

2001 52 (3.5) 38 (3.2) 10 (1.5) # (***) 48 (3.5) 10 (1.5)

Graduated high school 1994 44 (2.0) 42 (2.5) 13 (1.6) # (0.3) 56 (2.0) 14 (1.6)

2001 38 (2.0) 50 (1.8) * 12 (1.6) # (***) 62 (2.0) 12 (1.6)

Some education after high school 1994 25 (1.5) 51 (1.7) 23 (1.4) 1 (***) 75 (1.5) 24 (1.8)

2001 27 (1.2) 52 (1.7) 20 (1.6) 1 (0.3) 73 (1.2) 21 (1.7)

Graduated College 1994 19 (1.1) 41 (1.3) 37 (1.5) 3 (0.8) 81 (1.1) 40 (1.6)

2001 18 (1.1) 46 (1.4) * 34 (1.5) 3 (0.6) 82 (1.1) 36 (1.8)

Unknown 1994 64 (4.4) 29 (4.1) 7 (1.6) 0 (***) 36 (4.4) 7 (1.6)

2001 67 (4.2) 28 (4.3) 5 (2.0) 0 (***) 33 (4.2) 5 (2.0)
152 A P P E N D I X B • GEOGRAPHY REPORT CARD

Table B.14: Data for Figure 3.11 National Scale Score Results by Type of School

Percentage of students and average geography scale scores by type of school, grades 4, 8, and 12:

1994 and 2001

Public Nonpublic Nonpublic: Catholic Nonpublic: Other

Grade 4 1994 90 (0.8) 10 (0.8) 6 (0.7) 4 (0.5)

204 (1.4) 221 (2.2) 222 (2.6) 220 (3.8)

2001 89 (1.2) 11 (1.2) 6 (0.8) 5 (0.9)

207 (1.1) 226 (2.2) 230 (1.7) 221 (4.4)

Grade 8 1994 90 (0.8) 10 (0.8) 6 (0.6) 4 (0.6)

258 (0.8) 276 (1.3) 276 (1.6) 276 (2.6)

2001 90 (0.9) 10 (0.9) 5 (0.6) 5 (0.7)

261 (1.0) * 274 (2.5) 277 (2.0) 271 (4.7)

Grade 12 1994 89 (1.0) 11 (1.0) 6 (0.9) 4 (0.6)

283 (0.8) 294 (1.6) 291 (3.0) 298 (2.0)

2001 92 (0.8) 8 (0.8) 4 (0.6) 3 (0.6)

284 (0.8) 291 (2.3) 294 (2.0) 287 (4.3)

The percentage of students is listed first with the corresponding average scale score presented below.

Standard errors of the estimated percentages and scale scores appear in parentheses.

* Significantly different from 1994.

NOTE: Percentages may not add to 100 due to rounding.

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of

Educational Progress (NAEP), 1994 and 2001 Geography Assessments.
APPENDIX B • G E O G R A P H Y R E P O R T C A R D 153

Table B.15: Data for Figure 3.12a, b, c National Achievement-Level Results by Type of School

Percentage of students within and at or above geography achievement levels by type of school,

grades 4, 8, and 12: 1994 and 2001

At or above At or above

Below Basic At Basic At Proficient At Advanced Basic Proficient

Standard errors of the estimated percentages appear in parentheses.

* Significantly different from 1994.

NOTE: Percentages within each geography achievement-level range may not add to 100, or to the exact percentages at or above achievement levels, due to rounding.

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of

Educational Progress (NAEP), 1994 and 2001 Geography Assessments.

Grade 4 Public 1994 32 (1.2) 47 (1.1) 19 (1.2) 3 (0.5) 68 (1.2) 21 (1.3)

2001 28 (1.3) 52 (1.6) * 18 (1.1) 2 (0.3) 72 (1.3) 20 (1.1)

Nonpublic 1994 16 (2.2) 53 (2.3) 26 (2.3) 5 (1.2) 84 (2.2) 30 (2.5)

2001 9 (2.2) 59 (2.1) 28 (2.6) 3 (0.9) 91 (2.2) 31 (2.8)

Nonpublic: Catholic 1994 15 (2.8) 54 (3.2) 25 (2.4) 5 (1.9) 85 (2.8) 30 (3.0)

2001 7 (1.2) * 56 (2.3) 32 (2.0) 4 (1.6) 93 (1.2) * 36 (2.5)

Nonpublic: other 1994 18 (3.5) 52 (3.6) 26 (3.7) 4 (1.2) 82 (3.5) 30 (4.1)

2001 12 (5.1) 63 (4.1) 23 (4.9) 2 (1.1) 88 (5.1) 25 (5.5)

Grade 8 Public 1994 31 (1.0) 43 (1.1) 22 (1.0) 4 (0.4) 69 (1.0) 26 (1.0)

2001 28 (0.9) * 44 (1.0) 25 (1.2) 3 (0.6) 72 (0.9) * 28 (1.2)

Nonpublic 1994 13 (1.7) 43 (2.3) 36 (2.3) 8 (1.4) 87 (1.7) 44 (2.2)

2001 13 (2.6) 46 (3.6) 36 (3.5) 5 (1.3) 87 (2.6) 41 (4.2)

Nonpublic: Catholic 1994 11 (1.8) 45 (2.2) 35 (2.8) 8 (1.6) 89 (1.8) 44 (2.6)

2001 11 (2.5) 44 (3.4) 40 (2.9) 6 (1.3) 89 (2.5) 46 (3.4)

Nonpublic: other 1994 14 (3.2) 41 (4.7) 38 (4.6) 7 (2.0) 86 (3.2) 45 (4.8)

2001 15 (4.7) 48 (5.1) 33 (6.2) 4 (1.8) 85 (4.7) 37 (7.2)

Grade 12 Public 1994 32 (1.0) 42 (1.1) 24 (1.0) 1 (0.5) 68 (1.0) 26 (1.3)

2001 29 (1.0) 47 (1.0) * 23 (1.1) 1 (0.3) 71 (1.0) 24 (1.2)

Nonpublic 1994 17 (2.2) 47 (1.7) 33 (2.3) 3 (0.7) 83 (2.2) 36 (2.3)

2001 20 (3.0) 48 (2.0) 30 (2.6) 2 (0.7) 80 (3.0) 32 (3.0)

Nonpublic: Catholic 1994 20 (3.9) 47 (2.0) 32 (3.6) 1 (0.5) 80 (3.9) 33 (3.8)

2001 15 (2.5) 51 (2.5) 32 (3.4) 2 (0.6) 85 (2.5) 34 (3.4)

Nonpublic: other 1994 13 (2.4) 47 (2.9) 35 (2.9) 5 (1.5) 87 (2.4) 40 (3.1)

2001 27 (5.5) 44 (3.1) 26 (4.7) 2 (1.3) 73 (5.5) 29 (5.6)
154 A P P E N D I X B • GEOGRAPHY REPORT CARD

Table B.17: Data for Figure 3.13 National Achievement-Level Results by Type of Location

Percentage of students within and at or above geography achievement levels

by type of school location, grades 4, 8, and 12: 2001

At or above At or above

Below Basic At Basic At Proficient At Advanced Basic Proficient

Percentage of students and average geography scale scores by type of school location, grades 4, 8,

and 12: 2001

Central city Urban fringe/large town Rural/small town

Table B.16: Data for Table 3.1 National Scale Score Results by Type of Location

Grade 4 27 (1.6) 44 (2.9) 29 (2.8)

199 (2.3) 212 (2.1) 215 (2.0)

Grade 8 27 (2.0) 45 (2.8) 28 (2.5)

255 (2.0) 265 (1.6) 265 (2.0)

Grade 12 26 (2.0) 40 (2.8) 34 (2.5)

279 (1.6) 288 (1.6) 284 (1.0)

The percentage of students is listed first with the corresponding average scale score presented below.

Standard errors of the estimated percentages and scale scores appear in parentheses.

NOTE: Percentages may not add to 100 due to rounding.

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of

Educational Progress (NAEP), 2001 Geography Assessment.

Standard errors of the estimated percentages appear in parentheses.

NOTE: Percentages within each geography achievement-level range may not add to 100, or to the exact percentages at or above achievement levels, due to rounding.

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of

Educational Progress (NAEP), 2001 Geography Assessment.

Grade 4 Central city 38 (2.4) 46 (1.8) 14 (1.6) 2 (0.4) 62 (2.4) 16 (1.8)

Urban fringe/large town 24 (2.1) 53 (1.7) 21 (1.5) 2 (0.6) 76 (2.1) 23 (1.7)

Rural/small town 19 (2.1) 58 (3.0) 21 (2.4) 2 (0.5) 81 (2.1) 23 (2.4)

Grade 8 Central city 36 (2.2) 39 (1.7) 22 (1.8) 3 (0.7) 64 (2.2) 25 (2.1)

Urban fringe/large town 22 (1.7) 45 (1.3) 29 (1.5) 4 (0.9) 78 (1.7) 32 (1.8)

Rural/small town 22 (2.2) 48 (2.0) 27 (2.4) 3 (0.7) 78 (2.2) 30 (2.7)

Grade 12 Central city 37 (2.4) 42 (2.0) 19 (1.8) 1 (0.3) 63 (2.4) 20 (1.9)

Urban fringe/large town 25 (1.6) 45 (1.6) 28 (1.9) 2 (0.6) 75 (1.6) 30 (2.3)

Rural/small town 26 (1.5) 52 (1.5) 21 (1.4) 1 (0.3) 74 (1.5) 22 (1.4)
APPENDIX B • G E O G R A P H Y R E P O R T C A R D 155

Table B.19: Data for Figure 3.14 National Achievement-Level Results by Free/Reduced-Price School

Lunch Program Eligibility

Percentage of students within and at or above geography achievement levels by student eligibility

for the Free/Reduced-Price School Lunch program, grades 4, 8, and 12: 2001

At or above At or above

Below Basic At Basic At Proficient At Advanced Basic Proficient

Standard errors of the estimated percentages appear in parentheses.

# Percentage is between 0.0 and 0.5.

(***) Standard error estimates cannot be accurately determined.

NOTE: Percentages within each geography achievement-level range may not add to 100, or to the exact percentages at or above achievement levels, due to rounding.

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of

Educational Progress (NAEP), 2001 Geography Assessment.

Grade 4 Eligible 49 (2.2) 45 (2.1) 6 (0.9) # (***) 51 (2.2) 6 (0.9)

Not eligible 14 (1.1) 56 (1.6) 27 (1.4) 3 (0.6) 86 (1.1) 29 (1.5)

Info not available 16 (2.5) 57 (2.9) 24 (3.1) 3 (0.8) 84 (2.5) 27 (3.2)

Grade 8 Eligible 50 (1.8) 39 (1.6) 10 (1.1) 1 (0.3) 50 (1.8) 11 (1.2)

Not eligible 17 (0.9) 46 (1.3) 32 (1.5) 5 (0.8) 83 (0.9) 37 (1.7)

Info not available 21 (2.1) 46 (2.5) 29 (2.2) 4 (0.9) 79 (2.1) 33 (2.5)

Grade 12 Eligible 49 (2.3) 40 (1.7) 10 (1.5) # (***) 51 (2.3) 11 (1.6)

Not eligible 25 (1.2) 49 (1.3) 25 (1.4) 1 (0.4) 75 (1.2) 26 (1.6)

Info not available 24 (2.0) 45 (1.6) 29 (2.0) 2 (0.4) 76 (2.0) 31 (2.1)

Table B.18: Data for Table 3.2 National Scale Score Results by Free/Reduced-Price School Lunch

Program Eligibility

Percentage of students and average geography scale scores by student eligibility for

Free/Reduced-Price School Lunch program, grades 4, 8, and 12: 2001

Eligible Not eligible Info not available

Grade 4 33 (1.4) 48 (2.3) 18 (2.4)

186 (1.7) 221 (1.2) 218 (2.5)

Grade 8 25 (1.1) 53 (2.1) 22 (2.3)

242 (1.4) 270 (1.1) 266 (1.8)

Grade 12 16 (1.0) 64 (2.2) 21 (2.4)

269 (1.6) 287 (1.0) 289 (1.5)

The percentage of students is listed first with the corresponding average scale score presented below.

Standard errors of the estimated percentages and scale scores appear in parentheses.

NOTE: Percentages may not add to 100 due to rounding.

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of

Educational Progress (NAEP), 2001 Geography Assessment.
156 A P P E N D I X B • GEOGRAPHY REPORT CARD

Percentage of students and average geography scale scores by teachers’ reported undergraduate/

graduate major and minor/special emphasis, grades 4 and 8: 2001

Table B.20: Data for Table 4.1 Teachers’ Majors/Minors

Grade 4

Geography or Geography Education 7 (1.0) 93 (1.0)

204 (5.2) 210 (1.1)

History or History Education 15 (1.5) 85 (1.5)

206 (3.6) 211 (1.1)

Social Science or Social Studies Education 20 (1.7) 80 (1.7)

208 (2.6) 210 (1.3)

Other Social Science 57 (2.5) 43 (2.5)

210 (1.6) 209 (1.4)

Elementary Education 93 (1.0) 7 (1.0)

211 (1.1) 197 (4.3)

Grade 8

Geography or Geography Education 28 (2.5) 72 (2.5)

263 (2.1) 263 (1.2)

History or History Education 71 (2.7) 29 (2.7)

263 (1.3) 261 (1.9)

Social Science or Social Studies Education 55 (3.0) 45 (3.0)

263 (1.3) 262 (1.5)

Other Social Science 51 (2.1) 49 (2.1)

261 (1.5) 264 (1.4)

The percentage of students is listed first with the corresponding average scale score presented below.

Standard errors of the estimated percentages and scale scores appear in parentheses.

NOTE: Percentages may not add to 100 due to rounding.

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of

Educational Progress (NAEP), 2001 Geography Assessment.

Yes No Did you have a major, minor, or special emphasis in any

of the following subjects as part of your undergraduate

or graduate course work?
APPENDIX B • G E O G R A P H Y R E P O R T C A R D 157

Percentage of students and average geography scale scores by teachers’ reports on how well prepared they felt they were to teach geography, grades 4 and 8: 1994 and 2001

Table B.21: Data for Table 4.2 Teachers’ Preparedness to Teach Geography

Grade 4

Very prepared 23 (2.0) 31 (2.1) *

209 (2.1) 211 (2.1)

Adequately prepared 57 (2.0) 53 (2.3)

206 (1.8) 210 (1.3)

Somewhat prepared 18 (1.8) 15 (1.5)

207 (2.8) 206 (2.4)

Unprepared 2 (0.5) 1 (0.3)

200 (8.9) ! 209 (8.6) !

Grade 8

Very prepared 36 (2.9) 44 (2.5)

260 (2.2) 263 (1.4)

Adequately prepared 48 (3.6) 43 (2.4)

262 (1.4) 262 (1.3)

Somewhat prepared 13 (2.3) 11 (1.7)

265 (2.9) 261 (2.5)

Unprepared 2 (***) 2 (0.6)

260 (3.7) ! 264 (8.9) !

The percentage of students is listed first with the corresponding average scale score presented below.

Standard errors of the estimated percentages and scale scores appear in parentheses.

! The nature of the sample does not allow accurate determination of the variability of the statistic.

* Significantly different from 1994.

(***) Standard error estimates cannot be accurately determined.

NOTE: Percentages may not add to 100 due to rounding.

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of

Educational Progress (NAEP), 1994 and 2001 Geography Assessments.

Regardless or whether you are currently teaching the

topic, how well prepared do you feel you are to teach

geography at the elementary/middle school level?

1994 2001
158 A P P E N D I X B • GEOGRAPHY REPORT CARD

Using maps and globes

Almost every day 29 (2.3) 28 (2.3)

210 (2.1) 213 (1.9)

Once or twice a week 54 (2.5) 47 (2.2)

208 (1.7) 209 (1.5)

Once or twice a month 17 (2.1) 22 (1.7)

199 (3.2) 206 (2.4)

Never or hardly ever 1 (0.3) 3 (0.7)

*** (***) 209 (8.9) !

Natural resources

Almost every day 9 (1.8) 9 (1.4)

201 (4.1) 217 (4.6)

Once or twice a week 38 (2.5) 31 (1.6) *

209 (2.3) 208 (1.9)

Once or twice a month 44 (2.5) 46 (1.9)

208 (2.2) 210 (1.7)

Never or hardly ever 9 (1.4) 14 (1.4) *

198 (4.8) 208 (3.1)

Foreign countries and cultures

Almost every day 6 (1.2) 3 (0.8)

206 (5.7) 206 (6.3) !

Once or twice a week 19 (1.9) 23 (1.7)

203 (2.7) 208 (2.3)

Once or twice a month 43 (2.1) 45 (2.7)

208 (2.0) 209 (1.4)

Never or hardly ever 32 (2.3) 29 (2.4)

209 (1.9) 212 (2.2)

Environmental issues

Almost every day 4 (0.9) 7 (1.5)

201 (5.6) ! 212 (3.6) !

Once or twice a week 27 (2.1) 21 (1.7)

206 (2.5) 205 (2.6)

Once or twice a month 56 (2.0) 56 (2.2)

208 (2.0) 211 (1.4)

Never or hardly ever 13 (1.5) 16 (1.9)

208 (3.9) 211 (3.0)

The percentage of students is listed first with the corresponding average scale score presented below.

Standard errors of the estimated percentages and scale scores appear in parentheses.

! The nature of the sample does not allow accurate determination of the variability of the statistic.

* Significantly different from 1994.

*** (***) Sample size is insufficient to permit a reliable estimate.

NOTE: Percentages may not add to 100 due to rounding.

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of

Educational Progress (NAEP), 1994 and 2001 Geography Assessments.

Percentage of students and average geography scale scores by teachers’ reports on frequency of

instruction of selected skills and topics, grade 4:1994 and 2001

Table B.22: Data for Table 4.3 Frequency of Instruction in Fourth-Grade Geography Skills and Topics

1994 2001 How often do you teach the following skills and topics as a

part of geography instruction with this class?
APPENDIX B • G E O G R A P H Y R E P O R T C A R D 159

Using maps and globes

Almost every day 9 (0.6) 12 (0.7) *

261 (1.7) 259 (1.7)

Once or twice a week 30 (0.9) 34 (0.7) *

264 (1.1) 264 (1.1)

Once or twice a month 33 (0.9) 33 (0.7)

263 (1.1) 268 (1.2)

Never or hardly ever 28 (1.0) 21 (0.7) *

253 (1.0) 258 (1.2)

Natural resources

Almost every day 9 (0.4) 9 (0.5)

251 (1.7) 249 (1.5)

Once or twice a week 21 (0.8) 24 (0.6) *

259 (1.2) 262 (1.3)

Once or twice a month 36 (0.8) 35 (0.6)

265 (1.0) 269 (1.1)

Never or hardly ever 34 (1.1) 32 (0.8)

260 (0.9) 263 (1.2)

Countries and cultures

Almost every day 23 (0.8) 31 (1.0) *

260 (1.2) 264 (1.1)

Once or twice a week 29 (1.0) 32 (0.7) *

261 (1.1) 266 (1.2)

Once or twice a month 28 (0.9) 24 (0.6) *

264 (1.2) 263 (1.2)

Never or hardly ever 20 (0.7) 13 (0.6) *

256 (1.3) 254 (1.6)

Environmental issues

Almost every day 12 (0.7) 11 (0.5)

258 (1.7) 254 (1.7)

Once or twice a week 21 (0.6) 24 (0.8) *

260 (1.2) 265 (0.9)

Once or twice a month 33 (0.8) 33 (0.7)

263 (1.1) 267 (1.2)

Never or hardly ever 34 (1.0) 32 (1.0)

260 (0.9) 262 (1.3)

Percentage of students and average geography scale scores by students’ reports on frequency of

instruction of selected skills and topics, grade 8 : 1994 and 2001

Table B.23a: Data for Table 4.4a Frequency of Instruction in Eighth-Grade Geography Skills and Topics

The percentage of students is listed first with the corresponding average scale score presented below.

Standard errors of the estimated percentages and scale scores appear in parentheses.

* Significantly different from 1994.

NOTE: Percentages may not add to 100 due to rounding.

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of

Educational Progress (NAEP), 1994 and 2001 Geography Assessments.

1994 2001 How often have you studied the following geography skills

and topics in school?
160 A P P E N D I X B • GEOGRAPHY REPORT CARD

The percentage of students is listed first with the corresponding average scale score presented below.

Standard errors of the estimated percentages and scale scores appear in parentheses.

* Significantly different from 1994.

NOTE: Percentages may not add to 100 due to rounding.

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of

Educational Progress (NAEP), 1994 and 2001 Geography Assessments.

Percentage of students and average geography scale scores by students’ reports on frequency of

instruction of selected skills and topics, grade 12: 1994 and 2001

Table B.23b: Data for Table 4.4b Frequency of Instruction in Twelfth-Grade Geography Skills and Topics

Using maps and globes

Almost every day 7 (0.4) 6 (0.4)

284 (2.0) 277 (1.5)

Once or twice a week 22 (0.7) 24 (0.5)

288 (0.9) 285 (1.1)

Once or twice a month 31 (0.7) 34 (0.6) *

286 (0.8) 287 (0.9)

Never or hardly ever 40 (0.9) 36 (0.8) *

283 (1.1) 284 (1.0)

Natural resources

Almost every day 7 (0.4) 7 (0.4)

282 (2.1) 275 (1.7)

Once or twice a week 18 (0.6) 22 (0.8) *

286 (1.2) 283 (1.2)

Once or twice a month 31 (0.7) 32 (0.7)

288 (1.0) 288 (0.9)

Never or hardly ever 45 (0.9) 39 (0.8) *

284 (0.9) 285 (1.0)

Countries and cultures

Almost every day 16 (0.6) 20 (0.5) *

287 (1.3) 286 (0.9)

Once or twice a week 26 (0.5) 32 (0.6) *

288 (1.0) 288 (1.0)

Once or twice a month 30 (0.7) 29 (0.6)

286 (0.8) 286 (1.2)

Never or hardly ever 28 (0.8) 19 (0.5) *

280 (1.0) 277 (1.0)

Environmental issues

Almost every day 11 (0.5) 11 (0.5)

284 (1.6) 279 (1.3)

Once or twice a week 22 (0.7) 26 (0.7) *

288 (1.2) 286 (1.1)

Once or twice a month 30 (0.7) 33 (0.6) *

288 (0.9) 289 (1.1)

Never or hardly ever 37 (0.8) 30 (0.6) *

282 (0.9) 282 (0.9)

1994 2001 How often have you studied the following geography skills

and topics in school?
APPENDIX B • G E O G R A P H Y R E P O R T C A R D 161

Percentage of students and average geography scale scores by students’ reports on grades in which

geography was taken since the 6th grade, grade 8: 1994 and 2001

Table B.24: Data for Table 4.5 Eighth-Grade Frequency of Geography Course Taking

The percentage of students is listed first with the corresponding average scale score presented below.

Standard errors of the estimated percentages and scale scores appear in parentheses.

* Significantly different from 1994.

NOTE: Percentages may not add to 100 due to rounding.

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of

Educational Progress (NAEP), 1994 and 2001 Geography Assessments.

Number of grades selected

None 18 (1.0) 12 (0.7)*

250 (1.6) 255 (1.7)

One 30 (0.7) 20 (0.6)*

257 (1.1) 256 (1.5)

Two 14 (0.9) 16 (0.6)

269 (1.4) 263 (1.3)

Three 26 (0.9) 43 (1.1)*

274 (0.9) 272 (1.1)

Don’t know 13 (0.6) 9 (0.5)

243 (1.5) 246 (1.5)

1994 2001 Did you take or do you expect to take a geography course

in 6th, 7th, or 8th grade?
162 A P P E N D I X B • GEOGRAPHY REPORT CARD

Percentage of students and average geography scale scores by students’ reports on grades in which

geography was taken since 9th grade, grade 12: 1994 and 2001

Table B.25: Data for Table 4.6 Twelfth-Grade Frequency of Geography Course Taking

The percentage of students is listed first with the corresponding average scale score presented below.

Standard errors of the estimated percentages and scale scores appear in parentheses.

* Significantly different from 1994.

NOTE: Percentages may not add to 100 due to rounding.

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of

Educational Progress (NAEP), 1994 and 2001 Geography Assessments.

Number of grades selected

None 31 (1.6) 21 (1.2) *

286 (1.4) 289 (1.3)

One 35 (1.4) 32 (1.4)

288 (0.9) 288 (1.2)

Two 16 (0.8) 15 (0.7)

286 (1.5) 285 (1.3)

Three 10 (0.6) 18 (0.9)*

281 (1.7) 280 (1.1)

Four 5 (0.5) 10 (0.6) *

277 (2.5) 281 (1.3)

Don’t know 3 (0.4) 3 (0.3)

268 (2.1) 265 (2.6)

Did you take or do you expect to take a geography course

in 9th, 10th, 11th, or 12th grade?

1994 2001
APPENDIX B • G E O G R A P H Y R E P O R T C A R D 163

Percentage of students and average geography scale scores by teachers’ reports on computer use for

social studies instruction, grade 4: 2001

Table B.26a: Data for Table 4.7a Fourth-Grade Computer Use

Grade 4

Use CD-ROM to look up reference works

Not at all 37 (2.5)

205 (2.0)

Small extent 47 (2.4)

211 (1.6)

Moderate extent 14 (1.8)

216 (2.4)

Large extent 2 (0.9)

214 (7.9) !

Retrieve information through the Internet

Not at all 34 (2.3)

203 (2.3)

Small extent 45 (2.3)

212 (1.6)

Moderate extent 17 (2.2)

216 (3.0)

Large extent 4 (1.1)

211 (6.3) !

Use exploration/simulation software

Not at all 54 (2.4)

207 (1.4)

Small extent 37 (2.4)

213 (1.7)

Moderate extent 8 (1.2)

211 (3.7)

Large extent 1 (0.2)

*** (***)

Organize information using spreadsheets/databases

Not at all 89 (1.4)

209 (1.2)

Small extent 9 (1.3)

213 (3.7)

Moderate extent 1 (0.4)

213 (8.5) !

Large extent # (0.2)

*** (***)

When students in this class work on social studies, to what 2001

extent do they use computers to do each of the following?

The percentage of students is listed first with the corresponding average scale score presented below.

Standard errors of the estimated percentages and scale scores appear in parentheses.

# Percentage is between 0.0 and 0.5.

! The nature of the sample does not allow accurate determination of the variability of the statistic.

*** (***) Sample size is insufficient to permit a reliable estimate.

NOTE: Percentages may not add to 100 due to rounding.

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of

Educational Progress (NAEP), 2001 Geography Assessment.
164 A P P E N D I X B • GEOGRAPHY REPORT CARD

Percentage of students and average geography scale scores by teachers’ reports on computer use for

social studies instruction, grade 8: 2001

Table B.26b: Data for Table 4.7b Eighth-Grade Computer Use

When students in this class work on social studies, to what 2001

extent do they use computers to do each of the following?

Grade 8

Use CD-ROM to look up reference works

Not at all 31 (2.0)

258 (1.3)

Small extent 48 (2.3)

263 (1.4)

Moderate extent 17 (2.4)

266 (2.4)

Large extent 4 (0.9)

268 (4.7) !

Retrieve information through the Internet

Not at all 20 (1.9)

255 (2.0)

Small extent 47 (2.4)

261 (1.3)

Moderate extent 29 (2.6)

266 (1.9)

Large extent 4 (0.8)

273 (3.8)

Use exploration/simulation software

Not at all 62 (2.2)

261 (1.2)

Small extent 32 (2.3)

265 (1.9)

Moderate extent 5 (1.1)

259 (3.4) !

Large extent 1 (0.4)

257 (11.0) !

Organize information using spreadsheets/databases

Not at all 74 (2.7)

261 (1.2)

Small extent 22 (2.7)

266 (2.4)

Moderate extent 2 (0.8)

262 (6.2) !

Large extent 1 (0.5)

249 (5.5) !

The percentage of students is listed first with the corresponding average scale score presented below.

Standard errors of the estimated percentages and scale scores appear in parentheses.

! The nature of the sample does not allow accurate determination of the variability of the statistic.

NOTE: Percentages may not add to 100 due to rounding.

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of

Educational Progress (NAEP), 2001 Geography Assessment.
APPENDIX B • G E O G R A P H Y R E P O R T C A R D 165

Percentage of students and average geography scale scores by students’ reports on computer use for

history and geography, grade 12: 2001

Table B.27: Data for Table 4.8 Twelfth-Grade Computer Use

Research projects using a CD or the Internet

Not at all 26 (0.8)

274 (1.0)

Small extent 32 (0.5)

285 (1.0)

Moderate extent 29 (0.7)

290 (1.1)

Large extent 13 (0.6)

292 (1.3)

Use exploration/simulation software

Not at all 66 (0.7)

287 (0.8)

Small extent 23 (0.5)

281 (1.1)

Moderate extent 9 (0.4)

276 (1.4)

Large extent 2 (0.2)

278 (3.3)

Tables, charts or graphs on the computer

Not at all 55 (0.9)

284 (0.7)

Small extent 30 (0.7)

288 (1.2)

Moderate extent 12 (0.5)

281 (1.6)

Large extent 4 (0.3)

277 (2.7)

The percentage of students is listed first with the corresponding average scale score presented below.

Standard errors of the estimated percentages and scale scores appear in parentheses.

NOTE: Percentages may not add to 100 due to rounding.

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of

Educational Progress (NAEP), 2001 Geography Assessment.

2001 Think about all the courses since the 9th grade in which you

have studied history or geography. To what extent have

you used computers to do the following? For this question

include both work in class and homework assignments.
166 A P P E N D I X B • GEOGRAPHY REPORT CARD

Percentage of students and average geography scale scores by students’ reports on how much they

like studying geography, grades 8 and 12: 1994 and 2001

Table B.28: Data for Table 4.9 How Much Eighth- and Twelfth-Grade Students Like Geography

Grade 8

One of my favorite subjects 19 (0.8) 20 (0.6)

274 (1.2) 275 (1.3)

Like other subjects better 67 (0.9) 69 (0.6)

260 (0.7) 263 (1.0)

Never studied geography 14 (0.6) 11 (0.5) *

241 (1.9) 247 (1.7)

Grade 12

One of my favorite subjects 14 (0.6) 15 (0.6)

297 (1.3) 293 (1.2)

Like other subjects better 63 (1.1) 72 (0.8) *

285 (0.8) 285 (0.8)

Never studied geography 23 (1.2) 13 (0.8) *

277 (1.3) 278 (1.8)

The percentage of students is listed first with the corresponding average scale score presented below.

Standard errors of the estimated percentages and scale scores appear in parentheses.

* Significantly different from 1994.

NOTE: Percentages may not add to 100 due to rounding.

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of

Educational Progress (NAEP), 1994 and 2001 Geography Assessments.

1994 2001 How much do you like studying geography?
APPENDIX B • G E O G R A P H Y R E P O R T C A R D 167

Table B.29: Data for Table 5.1 Comparison of Two Sets of National Scale Score Results

National average geography scale scores by type of results, grades 4, 8, and 12: 2001

Accommodations not permitted Accommodations permitted

Grade 4 209 (1.0) 208 (0.9)

Grade 8 262 (0.9) 260 (1.0) 

Grade 12 285 (0.8) 284 (0.8)

Standard errors of the estimated scale scores appear in parentheses.

 Significantly different from the sample where accommodations were not permitted.

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of

Educational Progress (NAEP), 2001 Geography Assessment.

Percentage of students within and at or above geography achievement levels by type of results,

grades 4, 8, and 12: 2001

Table B.30: Data for Table 5.2 Comparison of Two Sets of National Achievement-Level Results

At or above At or above

Below Basic At Basic At Proficient At Advanced Basic Proficient

Grade 4

Accommodations were not permitted 26 (1.2) 53 (1.4) 19 (1.1) 2 (0.3) 74 (1.2) 21 (1.0)

Accommodations were permitted 27 (1.0) 52 (1.1) 19 (0.9) 2 (0.3) 73 (1.0) 20 (0.9)

Grade 8

Accommodations were not permitted 26 (0.9) 44 (0.9) 26 (1.1) 4 (0.6) 74 (0.9) 30 (1.2)

Accommodations were permitted 28 (1.2)  43 (0.9) 25 (1.0) 4 (0.5) 72 (1.2)  29 (1.3)

Grade 12

Accommodations were not permitted 29 (0.9) 47 (0.9) 23 (1.0) 1 (0.3) 71 (0.9) 25 (1.1)

Accommodations were permitted 29 (1.0) 47 (0.9) 23 (1.0) 1 (0.3) 71 (1.0) 24 (1.2)

Standard errors of the estimated percentages appear in parentheses.

 Significantly different from the sample where accommodations were not permitted.

NOTE: Percentages within each geography achievement-level range may not add to 100, or to the exact percentages at or above achievement levels, due to rounding.

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of

Educational Progress (NAEP), 2001 Geography Assessment.
168 A P P E N D I X B • GEOGRAPHY REPORT CARD

National average geography scale scores by gender and type of results, grades 4, 8, and 12: 2001

Table B.31: Data for Table 5.3 Comparison of Two Sets of National Scale Score Results by Gender

Male Female

Grade 4

Accommodations were not permitted 212 (1.1) 207 (1.2)

Accommodations were permitted 210 (1.0) 206 (1.3)

Grade 8

Accommodations were not permitted 264 (1.0) 260 (1.1)

Accommodations were permitted 262 (1.2)  258 (1.0)

Grade 12

Accommodations were not permitted 287 (0.9) 282 (0.8)

Accommodations were permitted 287 (1.0) 281 (0.8)

Standard errors of the estimated scale scores appear in parentheses.

 Significantly different from the sample where accommodations were not permitted.

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of

Educational Progress (NAEP), 2001 Geography Assessment.

Table B.32: Table 5.4 Comparison of Two Sets of National Achievement-Level Results by Gender

At or above At or above

Below Basic At Basic At Proficient At Advanced Basic Proficient

Percentage of students within and at or above geography achievement levels by gender and

type of results, grades 4, 8, and 12: 2001

Grade 4

Male

Accommodations were not permitted 25 (1.3) 51 (1.6) 21 (1.4) 3 (0.5) 75 (1.3) 24 (1.4)

Accommodations were permitted 26 (0.9) 51 (1.3) 21 (1.1) 3 (0.5) 74 (0.9) 23 (1.2)

Female

Accommodations were not permitted 28 (1.6) 54 (1.7) 17 (1.2) 1 (0.4) 72 (1.6) 18 (1.1)

Accommodations were permitted 29 (1.5) 54 (1.5) 16 (1.3) 1 (0.3) 71 (1.5) 18 (1.3)

Grade 8

Male

Accommodations were not permitted 25 (1.0) 42 (1.3) 29 (1.7) 5 (0.7) 75 (1.0) 33 (1.5)

Accommodations were permitted 27 (1.5) 41 (1.0) 27 (1.2) 4 (0.7) 73 (1.5) 32 (1.5)

Female

Accommodations were not permitted 27 (1.2) 47 (1.1) 24 (1.3) 3 (0.6) 73 (1.2) 26 (1.4)

Accommodations were permitted 29 (1.1) 45 (1.5) 23 (1.4) 3 (0.5) 71 (1.1) 26 (1.5)

Grade 12

Male

Accommodations were not permitted 27 (1.1) 45 (1.3) 26 (1.4) 2 (0.4) 73 (1.1) 28 (1.5)

Accommodations were permitted 26 (1.3) 45 (1.4) 26 (1.4) 2 (0.4) 74 (1.3) 28 (1.6)

Female

Accommodations were not permitted 30 (1.0) 48 (1.0) 20 (0.9) 1 (0.3) 70 (1.0) 21 (1.0)

Accommodations were permitted 32 (1.2) 48 (1.2) 19 (1.2) 1 (0.3) 68 (1.2) 20 (1.2)

Standard errors of the estimated percentages appear in parentheses.

NOTE: Percentages within each geography achievement-level range may not add to 100, or to the exact percentages at or above achievement levels, due to rounding.

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of

Educational Progress (NAEP), 2001 Geography Assessment.
APPENDIX B • G E O G R A P H Y R E P O R T C A R D 169

National average geography scale scores by race/ethnicity and type of results, grades 4, 8, and 12:

2001

Table B.33: Data for Table 5.5 Comparison of Two Sets of National Scale Score Results by Race/Ethnicity

Asian/Pacific American

White Black Hispanic Islander Indian

Grade 4

Accommodations were not permitted 222 (1.0) 181 (1.8) 184 (2.8) 212 (2.7) 199 (3.6)

Accommodations were permitted 220 (1.0) 181 (1.9) 185 (2.1) 216 (2.6) 199 (3.4)

Grade 8

Accommodations were not permitted 273 (1.0) 234 (1.7) 240 (1.7) 266 (2.5) 261 (5.8)

Accommodations were permitted 271 (1.4) 232 (1.6) 238 (1.8) 267 (2.2) 259 (4.9)

Grade 12

Accommodations were not permitted 291 (0.9) 260 (1.4) 270 (1.5) 286 (2.9) 288 (3.6) !

Accommodations were permitted 292 (0.8) 258 (1.5) 269 (1.4) 285 (5.0) 286 (3.5) !

Standard errors of the estimated scale scores appear in parentheses.

! The nature of the sample does not allow accurate determination of the variability of the statistic.

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of

Educational Progress (NAEP), 2001 Geography Assessment.
170 A P P E N D I X B • GEOGRAPHY REPORT CARD

Table B.34: Data for Table 5.6 Comparison of Two Sets of National Achievement-Level Results by Race/Ethnicity

Percentage of students within and at or above geography achievement levels by race/ethnicity and type

of results, grades 4, 8, and 12: 2001

At or above At or above

Below Basic At Basic At Proficient At Advanced Basic Proficient

Standard errors of the estimated percentages appear in parentheses.

# Percentage is between 0.0 and 0.5.

(***) Standard error estimates cannot be accurately determined.

! The nature of the sample does not allow accurate determination of the variability of the statistic.

NOTE: Percentages within each geography achievement-level range may not add to 100, or to the exact percentages at or above achievement levels, due to rounding.

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of

Educational Progress (NAEP), 2001 Geography Assessment.

Grade 4

White

Accommodations were not permitted 13 (1.3) 58 (1.8) 26 (1.6) 3 (0.5) 87 (1.3) 29 (1.5)

Accommodations were permitted 15 (1.0) 57 (1.4) 25 (1.3) 3 (0.5) 85 (1.0) 28 (1.3)

Black

Accommodations were not permitted 56 (2.1) 39 (2.1) 5 (0.8) # (***) 44 (2.1) 5 (0.9)

Accommodations were permitted 56 (2.7) 40 (2.6) 4 (0.5) # (***) 44 (2.7) 4 (0.6)

Hispanic

Accommodations were not permitted 51 (3.0) 43 (2.5) 6 (1.0) # (***) 49 (3.0) 6 (1.0)

Accommodations were permitted 49 (2.5) 45 (2.1) 5 (0.9) # (***) 51 (2.5) 6 (0.9)

Asian/Pacific Islander

Accommodations were not permitted 23 (3.4) 52 (4.4) 23 (3.1) 1 (0.9) 77 (3.4) 25 (3.0)

Accommodations were permitted 18 (3.4) 57 (4.0) 24 (3.9) 2 (0.8) 82 (3.4) 25 (3.7)

American Indian

Accommodations were not permitted 34 (4.9) 53 (6.3) 13 (4.2) # (***) 66 (4.9) 13 (4.1)

Accommodations were permitted 37 (5.7) 51 (5.7) 12 (3.1) # (***) 63 (5.7) 12 (3.3)

Grade 8

White

Accommodations were not permitted 14 (0.9) 48 (1.2) 34 (1.5) 5 (0.8) 86 (0.9) 39 (1.7)

Accommodations were permitted 16 (1.5) 46 (1.2) 33 (1.5) 5 (0.7) 84 (1.5) 38 (1.9)

Black

Accommodations were not permitted 60 (2.3) 34 (1.9) 6 (0.8) # (***) 40 (2.3) 6 (0.8)

Accommodations were permitted 62 (2.5) 32 (2.2) 6 (0.9) # (***) 38 (2.5) 6 (1.1)

Hispanic

Accommodations were not permitted 52 (1.9) 38 (1.6) 9 (1.1) 1 (0.2) 48 (1.9) 10 (1.0)

Accommodations were permitted 54 (2.3) 37 (1.9) 9 (0.8) 1 (0.2) 46 (2.3) 9 (0.8)

Asian/Pacific Islander

Accommodations were not permitted 21 (3.4) 47 (4.8) 28 (3.5) 4 (1.8) 79 (3.4) 32 (3.2)

Accommodations were permitted 20 (2.7) 49 (2.9) 28 (3.1) 4 (1.4) 80 (2.7) 32 (3.0)

American Indian

Accommodations were not permitted 28 (6.8) 41 (11.1) 29 (8.9) 3 (***) 72 (6.8) 31 (11.2)

Accommodations were permitted 30 (5.2) 46 (5.9) 21 (6.0) 3 (***) 70 (5.2) 24 (7.2)

Grade 12

White

Accommodations were not permitted 19 (0.9) 51 (1.1) 29 (1.2) 2 (0.4) 81 (0.9) 31 (1.4)

Accommodations were permitted 19 (0.9) 51 (1.1) 29 (1.3) 2 (0.4) 81 (0.9) 31 (1.4)

Black

Accommodations were not permitted 65 (2.3) 31 (2.1) 4 (0.7) # (***) 35 (2.3) 4 (0.7)

Accommodations were permitted 67 (2.0) 30 (1.8) 3 (0.9) # (***) 33 (2.0) 3 (0.9)

Hispanic

Accommodations were not permitted 48 (2.6) 42 (2.5) 10 (1.4) # (0.1) 52 (2.6) 10 (1.4)

Accommodations were permitted 50 (2.4) 42 (2.2) 9 (1.1) # (***) 50 (2.4) 9 (1.1)

Asian/Pacific Islander

Accommodations were not permitted 28 (4.3) 45 (3.0) 25 (4.6) 1 (0.7) 72 (4.3) 26 (4.7)

Accommodations were permitted 29 (6.1) 46 (2.6) 23 (5.0) 1 (0.9) 71 (6.1) 25 (5.6)

American Indian

Accommodations were not permitted 26 (6.0) ! 41 (7.0) ! 31 (5.3) ! 1 (***) 74 (6.0) ! 32 (4.9) !

Accommodations were permitted 29 (7.6) ! 41 (7.6) ! 29 (6.9) ! 1 (***) 71 (7.6) ! 30 (6.9) !
APPENDIX B • G E O G R A P H Y R E P O R T C A R D 171

Table B.36: Data for Table 6.2 Grade 4 Sample Question 2 Results (Multiple-Choice)

Table B.35: Data for Table 6.1 Grade 4 Sample Question 1 Results (Multiple-Choice)

Overall percentage correct and percentages correct within each achievement-level range: 2001

Standard errors of the estimated percentages appear in parentheses.

*NAEP geography composite scale range.

***(***)Sample size is insufficient to permit a reliable estimate (see appendix A).

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education

Statistics, National Assessment of Educational Progress (NAEP), 2001 Geography Assessment.

Percentage correct within

achievement-level intervals

Overall percentage Below Basic Basic Proficient Advanced

correct 186 and below* 187–239* 240–275* 276 and above*

70 (1.4) 50 (2.8) 74 (1.7) 84 (2.5) *** (***)

Grade 4

Overall percentage correct and percentages correct within each achievement-level range: 2001

Standard errors of the estimated percentages appear in parentheses.

*NAEP geography composite scale range.

***(***)Sample size is insufficient to permit a reliable estimate (see appendix A).

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education

Statistics, National Assessment of Educational Progress (NAEP), 2001 Geography Assessment.

Percentage correct within

achievement-level intervals

Overall percentage Below Basic Basic Proficient Advanced

correct 186 and below* 187–239* 240–275* 276 and above*

33 (1.1) 22 (1.8) 28 (1.8) 56 (3.2) *** (***)

Grade 4
172 A P P E N D I X B • GEOGRAPHY REPORT CARD

Table B.37: Data for Table 6.3a Grade 4 Sample Question 3 Results (“Complete” Short Constructed-Response)

Overall percentage “Complete” and percentages “Complete” within each achievement-level range:

2001

Standard errors of the estimated percentages appear in parentheses.

*NAEP geography composite scale range.

***(***) Sample size is insufficient to permit a reliable estimate (see appendix A).

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education

Statistics, National Assessment of Educational Progress (NAEP), 2001 Geography Assessment.

Percentage “Complete” within

achievement-level intervals

Overall percentage Below Basic Basic Proficient Advanced

“Complete” 186 and below* 187–239* 240–275* 276 and above*

66 (1.4) 38 (2.3) 71 (2.0) 88 (2.3) *** (***)

Grade 4

Overall percentage “Partial” or better and percentages “Partial” or better within each achievementlevel range: 2001

Standard errors of the estimated percentages appear in parentheses.

*NAEP geography composite scale range.

*** (***) Sample size is insufficient to permit a reliable estimate (see appendix A).

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education

Statistics, National Assessment of Educational Progress (NAEP), 2001 Geography Assessment.

Percentage “Partial” or better within

achievement-level intervals

Overall percentage Below Basic Basic Proficient Advanced

“Partial” or better 186 and below* 187–239* 240–275* 276 and above*

72 (1.4) 43 (2.5) 78 (1.8) 93 (2.3) *** (***)

Grade 4

Table B.38: Data for Table 6.3b Grade 4 Sample Question 3 Results (“Partial” Short Constructed-Response)
APPENDIX B • G E O G R A P H Y R E P O R T C A R D 173

Overall percentage “Complete” and percentages “Complete” within each achievement-level range: 2001

Percentage “Complete” within

achievement-level intervals

Overall percentage Below Basic Basic Proficient Advanced

“Complete” or better 186 and below* 187–239* 240–275* 276 and above*

11 (0.8) 0 (***) 6 (1.2) 32 (3.4) *** (***)

Standard errors of the estimated percentages appear in parentheses.

*NAEP geography composite scale range.

*** (***) Sample size is insufficient to permit a reliable estimate (see appendix A).

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education

Statistics, National Assessment of Educational Progress (NAEP), 2001 Geography Assessment.

Grade 4

Table B.39: Data for Table 6.4a Grade 4 Sample Question 4 Results (“Complete” Extended Constructed-Response)

Table B.40: Data for Table 6.4b Grade 4 Sample Question 4 Results (“Essential” Extended Constructed-Response)

Overall percentage “Essential” or better and percentages “Essential” or better within each achievementlevel range: 2001

Percentage “Essential” or better within

achievement-level intervals

Overall percentage Below Basic Basic Proficient Advanced

“Essential” or better 186 and below* 187–239* 240–275* 276 and above*

28 (1.3) 1 (0.6) 25 (2.1) 65 (3.8) *** (***)

Standard errors of the estimated percentages appear in parentheses.

*NAEP geography composite scale range.

***(***) Sample size is insufficient to permit a reliable estimate (see appendix A).

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education

Statistics, National Assessment of Educational Progress (NAEP), 2001 Geography Assessment.

Grade 4

Table B.41: Data for Table 6.4c Grade 4 Sample Question 4 Results (“Partial” Extended Constructed-Response)

Overall percentage “Partial” or better and percentages “Partial” or better within each achievement-level

range: 2001

Percentage “Partial” or better within

achievement-level intervals

Overall percentage Below Basic Basic Proficient Advanced

“Partial” or better 186 and below* 187–239* 240–275* 276 and above*

38 (1.3) 4 (1.5) 36 (2.2) 78 (2.5) *** (***)

Standard errors of the estimated percentages appear in parentheses.

*NAEP geography composite scale range.

*** (***) Sample size is insufficient to permit a reliable estimate (see appendix A).

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education

Statistics, National Assessment of Educational Progress (NAEP), 2001 Geography Assessment.

Grade 4
174 A P P E N D I X B • GEOGRAPHY REPORT CARD

Table B.42: Data for Table 6.5 Grade 8 Sample Question 5 Results (Multiple-Choice)

Overall percentage correct and percentages correct within each achievement-level range: 2001

Standard errors of the estimated percentages appear in parentheses.

*NAEP geography composite scale range.

(***) Standard error estimates cannot be accurately determined.

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education

Statistics, National Assessment of Educational Progress (NAEP), 2001 Geography Assessment.

Percentage correct within

achievement-level intervals

Overall percentage Below Basic Basic Proficient Advanced

correct 241 and below* 242–281* 282–314* 315 and above*

70 (1.2) 37 (2.3) 74 (1.7) 91 (1.5) 97 (***)

Grade 8

Table B.43: Data for Table 6.6 Grade 8 Sample Question 6 Results (Multiple-Choice)

Overall percentage correct and percentages correct within each achievement-level range: 2001

Standard errors of the estimated percentages appear in parentheses.

*NAEP geography composite scale range.

***(***) Sample size is insufficient to permit a reliable estimate (see appendix A).

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education

Statistics, National Assessment of Educational Progress (NAEP), 2001 Geography Assessment.

Percentage correct within

achievement-level intervals

Overall percentage Below Basic Basic Proficient Advanced

correct 241 and below* 242–281* 282–314* 315 and above*

50 (1.3) 36 (2.2) 47 (2.3) 64 (3.1) *** (***)

Grade 8
APPENDIX B • G E O G R A P H Y R E P O R T C A R D 175

Table B.45: Data for Table 6.8 Grade 8 Sample Question 8 Results (Multiple-Choice)

Overall percentage correct and percentages correct within each achievement-level range: 2001

Standard errors of the estimated percentages appear in parentheses.

*NAEP geography composite scale range.

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education

Statistics, National Assessment of Educational Progress (NAEP), 2001 Geography Assessment.

Percentage correct within

achievement-level intervals

Overall percentage Below Basic Basic Proficient Advanced

correct 241 and below* 242–281* 282–314* 315 and above*

60 (1.4) 40 (2.6) 57 (2.0) 79 (2.7) 96 (1.7)

Grade 8

Table B.44: Data for Table 6.7 Grade 8 Sample Question 7 Results (Multiple-Choice)

Overall percentage correct and percentages correct within each achievement-level range: 2001

Standard errors of the estimated percentages appear in parentheses.

*NAEP geography composite scale range.

(***) Standard error estimates cannot be accurately determined.

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education

Statistics, National Assessment of Educational Progress (NAEP), 2001 Geography Assessment.

Percentage correct within

achievement-level intervals

Overall percentage Below Basic Basic Proficient Advanced

correct 241 and below* 242–281* 282–314* 315 and above*

74 (1.4) 40 (2.3) 80 (2.0) 93 (1.9) 100 (***)

Grade 8
176 A P P E N D I X B • GEOGRAPHY REPORT CARD

Table B.46: Data for Table 6.9a Grade 8 Sample Question 9 Results (“Complete” Short Constructed-Response)

Percentage “Complete” within

achievement-level intervals

Overall percentage Below Basic Basic Proficient Advanced

“Complete” 241 and below* 242–281* 282–314* 315 and above*

22 (1.4) 6 (2.1) 18 (1.9) 38 (2.7) *** (***)

Overall percentage “Complete” and percentages “Complete” within each achievement-level range:

2001

Standard errors of the estimated percentages appear in parentheses.

*NAEP geography composite scale range.

***(***) Sample size is insufficient to permit a reliable estimate (see appendix A).

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education

Statistics, National Assessment of Educational Progress (NAEP), 2001 Geography Assessment.

Grade 8

Table B.47: Data for Table 6.9b Grade 8 Sample Question 9 Results (“Partial” Short Constructed-Response)

Percentage “Partial” or better within

achievement-level intervals

Overall percentage Below Basic Basic Proficient Advanced

“Partial” or better 241 and below* 242–281* 282–314* 315 and above*

60 (1.3) 26 (2.5) 62 (2.1) 84 (2.3) *** (***)

Overall percentage “Partial” or better and percentages “Partial” or better within each achievementlevel range: 2001

Standard errors of the estimated percentages appear in parentheses.

*NAEP geography composite scale range.

*** (***) Sample size is insufficient to permit a reliable estimate (see appendix A).

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education

Statistics, National Assessment of Educational Progress (NAEP), 2001 Geography Assessment.

Grade 8
APPENDIX B • G E O G R A P H Y R E P O R T C A R D 177

Table B.49: Data for Table 6.11 Grade 12 Sample Question 11 Results (Multiple-Choice)

Overall percentage correct and percentages correct within each achievement-level range: 2001

Standard errors of the estimated percentages appear in parentheses.

*NAEP geography composite scale range.

***(***) Sample size is insufficient to permit a reliable estimate (see appendix A).

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education

Statistics, National Assessment of Educational Progress (NAEP), 2001 Geography Assessment.

Percentage correct within

achievement-level intervals

Overall percentage Below Basic Basic Proficient Advanced

correct 269 and below* 270–304* 305–338* 339 and above*

61 (1.4) 46 (2.3) 62 (2.2) 76 (3.3) *** (***)

Grade 12

Table B.48: Data for Table 6.10 Grade 12 Sample Question 10 Results (Multiple-Choice)

Overall percentage correct and percentages correct within each achievement-level range: 2001

Standard errors of the estimated percentages appear in parentheses.

*NAEP geography composite scale range.

(***) Standard error estimates cannot be accurately determined.

***(***)Sample size is insufficient to permit a reliable estimate (see appendix A).

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education

Statistics, National Assessment of Educational Progress (NAEP), 2001 Geography Assessment.

Percentage correct within

achievement-level intervals

Overall percentage Below Basic Basic Proficient Advanced

correct 269 and below* 270–304* 305–338* 339 and above*

78 (1.2) 46 (2.3) 86 (1.6) 99 (***) *** (***)

Grade 12
178 A P P E N D I X B • GEOGRAPHY REPORT CARD

Table B.50: Data for Table 6.12a Grade 12 Sample Question 12 Results (“Complete” Short Constructed-Response)

Overall percentage “Complete” and percentages “Complete” within each achievement-level range: 2001

Percentage “Complete” within

achievement-level intervals

Overall percentage Below Basic Basic Proficient Advanced

“Complete” 269 and below* 270–304* 305–338* 339 and above*

47 (1.3) 17 (1.9) 52 (2.1) 70 (3.2) *** (***)

Standard errors of the estimated percentages appear in parentheses.

*NAEP Geography composite scale range.

***(***) Sample size is insufficient to permit a reliable estimate (see appendix A).

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education

Statistics, National Assessment of Educational Progress (NAEP), 2001 Geography Assessment.

Grade 12

Percentage “Partial” or better within

achievement-level intervals

Overall percentage Below Basic Basic Proficient Advanced

“Partial” or better 269 and below* 270–304* 305–338* 339 and above*

76 (1.2) 42 (2.8) 85 (1.4) 96 (1.4) *** (***)

Overall percentage “Partial” or better and percentages “Partial” or better within each achievement-level

range: 2001

Standard errors of the estimated percentages appear in parentheses.

*NAEP Geography composite scale range.

***(***) Sample size is insufficient to permit a reliable estimate (see appendix A).

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education

Statistics, National Assessment of Educational Progress (NAEP), 2001 Geography Assessment.

Grade 12

Table B.51 Data for Table 6.12b Grade 12 Sample Question 12 Results (“Partial” Short Constructed-Response)
APPENDIX B • G E O G R A P H Y R E P O R T C A R D 179

Percentage “Complete” within

achievement-level intervals

Overall percentage Below Basic Basic Proficient Advanced

“Complete” 269 and below* 270–304* 305–338* 339 and above*

16 (0.9) 2 (0.9) 15 (1.3) 33 (3.0) *** (***)

Overall percentage “Complete” and percentages “Complete” within each achievement-level range:

2001

Standard errors of the estimated percentages appear in parentheses.

*NAEP geography composite scale range.

*** (***) Sample size is insufficient to permit a reliable estimate (see appendix A).

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education

Statistics, National Assessment of Educational Progress (NAEP), 2001 Geography Assessment.

Grade 12

Table B.52: Data for Table 6.13a Grade 12 Sample Question 13 Results (“Complete” Short Constructed-Response)

Percentage “Partial” or better within

achievement-level intervals

Overall percentage Below Basic Basic Proficient Advanced

“Partial” or better 269 and below* 270–304* 305–338* 339 and above*

51 (1.7) 18 (2.1) 57 (2.0) 79 (2.7) *** (***)

Overall percentage “Partial” or better and percentages “Partial” or better within each achievement-level

range: 2001

Standard errors of the estimated percentages appear in parentheses.

*NAEP geography composite scale range.

*** (***) Sample size is insufficient to permit a reliable estimate (see appendix A).

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education

Statistics, National Assessment of Educational Progress (NAEP), 2001 Geography Assessment.

Grade 12

Table B.53: Data for Table 6.13b Grade 12 Sample Question 13 Results (“Partial” Short Constructed-Response)

APPENDIX C • G E O G R A P H Y R E P O R T C A R D 181

Appendix C

Members of the C NAEP Geography Standing Committee

Sarah Bednarz

Department of Geography

Texas A&M University

College Station, TX

Kristi Desaulniers

Tea Primary School

Tea, SD

Norman Bettis

Dept. of Curriculum & Instruction

Illinois State University

Normal, IL

Richard Boehm

Department of Geography and Planning

Southwest Texas State University

San Marcos, TX

Leah Bug-Townsend

Clair E. Gale Jr. High School

Idaho Falls, ID

Present affiliation:

Idaho/Oregon NASA Representative

NASA Ames Research Center

Idaho Falls, ID

Karen Coney

Stone Scholastic Academy

Chicago, IL

Roger Downs

Chairman, Department of Geography

The Pennsylvania State University

University Park, PA

Briavel Holcomb

Department of Urban Studies and Community Health

E. J. Bloustein School of Planning

Rutgers University

New Brunswick, NJ

Marianne Kenney

Colorado Department of Education

Denver, CO

Present affiliation:

McREL

Aurora, CO

R. Keith Lucero

East High School

Denver, CO

James Marran

(Retired) New Trier Twp. High School

Winnetka, IL

Ines Miyares

Department of Geography

Hunter College

New York, NY

Barbara Patty

Arkansas Department of Education

Little Rock, AR

Tom Wissinger

Harrington Middle School

Mt. Laurel, NJ
182 A C K N O W L E D G M E N T S • GEOGRAPHY REPORT CARD

Acknowledgments

This report is the culmination of the effort of many individuals who contributed their considerable

knowledge, experience, and creativity to the NAEP 2001 geography assessment. The assessment

was a collaborative effort among staff from the National Center for Education Statistics (NCES),

the National Assessment Governing Board (NAGB), Educational Testing Service (ETS), Westat, and

National Computer Systems (NCS). Most importantly, NAEP is grateful to the students and

school staff who made the assessment possible.

The 2001 NAEP geography assessment was funded through NCES, in the Office of Educational

Research and Improvement of the U.S. Department of Education. The Deputy Commissioner of

Education Statistics, Gary W. Phillips, and the NCES staff—Peggy Carr, Arnold Goldstein, Carol

Johnson, and Andrew Kolstad—worked closely and collegially with the authors to produce this

report.

The NAEP project at ETS is directed by Stephen Lazer and John Mazzeo, with assistance from

John Barone. Sampling and data collection activities were conducted by Westat under the direction

of Rene Slobasky, Nancy Caldwell, Keith Rust, and Dianne Walsh. Printing, distribution, scoring,

and processing activities were conducted by NCS under the direction of Brad Thayer, Connie

Smith, and William Buckles.

Test development activities took place at ETS under the direction of Andy Weiss with assistance

from Barbara Hildebrant and John Kasbarian.

The complex statistical and psychometric activities necessary to report results for the NAEP

2001 geography assessment were directed by Catherine Hombo and Matthew Johnson, with

assistance from Hui Deng. Frank Jenkins, advised by Matthew Johnson, supervised the analyses that

produced results for the data for which student accommodations were permitted.

The extensive data processing and computer programming activities underlying the statistical

and psychometric analyses conducted at ETS are under the direction of David Freund, Edward

Kulick, Bruce Kaplan, and Steven Isham. Data Analyses presented in this report were managed by

Scott Davis with assistance from Mei-jang Lin, Christina Tang, Norma Norris, and Alfred Rogers.

The complex database work for this assessment was managed by Katherine Pashley with assistance

from Gerry Kokolis.

The design and production of this report was overseen by Loretta Casalaina. Joseph Kolodey and

Rick Hasney contributed invaluable design and production expertise to the effort. Wendy Grigg

coordinated the documentation and data checking procedures with assistance from Janice Goodis,

Andrea Bergen, and Alice Kass. Shari Santapau coordinated the editorial and proofreading procedures with assistance from Valerie Mukuna and Julia McGuire. The Web version of this report was

coordinated by Pat O’Reilly with assistance from Rick Hasney.

Many thanks are due to the numerous reviewers, both internal and external to NCES and ETS.

The comments and critical feedback of the following reviewers are reflected in the final version of

this report: Ghedam Bairu, Teresa Calahan, Young Chun, Douglas Cohrane, Mary Crovo, Roger

Downs, Steven Gorman, Carol Johnson, Dana Kelly, Andrew Kolstad, Marilyn McMillen, Barbara

Patty, Taslima Rahman, Patrick Rooney and Sharif Shakrani.

United States

Department of Education

ED Pubs

8242-B Sandy Court

Jessup, MD 20794-1398

Official Business Only

Penalty for Private Use, $300

Postage and Fees Paid

U.S. Department of Education

Permit No. G-17

Parcel