NAEP CIVICS ASSESSMENTS–1988 AND 1998

he Next

 Generation

of Citizens

U.S. Department of Education

Office of Educational Research and Improvement NCES 2001-452

NATIONAL CENTER FOR EDUCATION STATISTICS
Adam Urbanski

President

Rochester Teachers Association

Rochester, New York

Migdania D. Vega

Principal

Coral Way Elementary Bilingual School

Miami, Florida

Deborah Voltz

Assistant Professor

Department of Special Education

University of Louisville

Louisville, Kentucky

Honorable Michael E. Ward

State Superintendent of Public Instruction

North Carolina Public Schools

Raleigh, North Carolina

Marilyn A. Whirry

Twelfth-Grade English Teacher

Manhattan Beach, California

Dennie Palmer Wolf

Senior Research Associate

Harvard University

Graduate School of Education

Cambridge, Massachusetts

(Ex-Officio)

Assistant Secretary of Education

Office of Educational Research and

Improvement

U.S. Department of Education

Washington, DC

Roy Truby

Executive Director, NAGB

Washington, DC

What is The Nation’s Report Card?

THE NATION’S REPORT CARD, the National Assessment of Educational Progress (NAEP), is the only nationally

representative and continuing assessment of what America’s students know and can do in various subject areas. Since 1969,

assessments have been conducted periodically in reading, mathematics, science, writing, history, geography, and other

fields. By making objective information on student performance available to policymakers at the national, state, and local

levels, NAEP is an integral part of our nation’s evaluation of the condition and progress of education. Only information

related to academic achievement is collected under this program. NAEP guarantees the privacy of individual students and

their families.

NAEP is a congressionally mandated project of the National Center for Education Statistics, the U.S. Department of

Education. The Commissioner of Education Statistics is responsible, by law, for carrying out the NAEP project through

competitive awards to qualified organizations. NAEP reports directly to the Commissioner, who is also responsible

for providing continuing reviews, including validation studies and solicitation of public comment, on NAEP’s conduct

and usefulness.

In 1988, Congress established the National Assessment Governing Board (NAGB) to formulate policy guidelines for

NAEP. The Board is responsible for selecting the subject areas to be assessed from among those included in the National

Education Goals; for setting appropriate student performance levels; for developing assessment objectives and test

specifications through a national consensus approach; for designing the assessment methodology; for developing guidelines

for reporting and disseminating NAEP results; for developing standards and procedures for interstate, regional, and national

comparisons; for determining the appropriateness of test items and ensuring they are free from bias; and for taking actions

to improve the form and use of the National Assessment.

The National Assessment Governing Board

Mark D. Musick, Chair

President

Southern Regional Education Board

Atlanta, Georgia

Michael T. Nettles, Vice Chair

Professor of Education

University of Michigan

Ann Arbor, Michigan

Moses Barnes

Secondary School Principal

Fort Lauderdale, Florida

Melanie A. Campbell

Fourth-Grade Teacher

Topeka, Kansas

Honorable Wilmer S. Cody

Former Commissioner of Education

State of Kentucky

Frankfort, Kentucky

Daniel A. Domenech

Superintendent of Schools

Fairfax County Public Schools

Fairfax, Virginia

Edward Donley

Former Chairman

Air Products & Chemicals, Inc.

Allentown, Pennsylvania

Thomas H. Fisher

Director

Student Assessment Services

Florida Department of Education

Tallahassee, Florida

Edward H. Haertel

Professor, School of Education

Stanford University

Stanford, California

Juanita Haugen

Local School Board Member

Pleasanton, California

Honorable Nancy Kopp

State Legislator

Annapolis, Maryland

Honorable Ronnie Musgrove

Governor of Mississippi

Jackson, Mississippi

Roy M. Nageak, Sr.

First Vice-Chair

Alaska Board of Education and

Early Development

Barrow, Alaska

Debra Paulson

Eighth-Grade Mathematics Teacher

El Paso, Texas

Honorable Jo Ann Pottorff

State Legislator

Wichita, Kansas

Diane Ravitch

Research Professor

New York University

New York, New York

Sister Lourdes Sheehan, R.S.M.

Secretary for Education

United States Catholic Conference

Washington, DC

John H. Stevens

Executive Director

Texas Business and Education Coalition

Austin, Texas
Andrew R. Weiss

Anthony D. Lutkus

Wendy S. Grigg

Richard G. Niemi

In collaboration with

Edward Kulick

Spencer Swinton

Laura Jerry

June 2001

U.S. Department of Education

Office of Educational Research and Improvement NCES 2001–452

NAEP CIVICS ASSESSMENTS–1988 AND 1998

he Next

 Generation

of Citizens

NATIONAL CENTER FOR EDUCATION STATISTICS
U.S. Department of Education

Rod Paige

Secretary

National Center for Education Statistics

Gary W. Phillips

Acting Commissioner

June 2001

SUGGESTED CITATION

U.S. Department of Education. Office of Educational Research and Improvement. National Center

for Education Statistics. The Next Generation of Citizens: NAEP Civics Assessments–1988 and 1998,

NCES 2001–452, by A.R. Weiss, A.D. Lutkus, W.S. Grigg, and R.G. Niemi. Washington, DC: 2001.

FOR MORE INFORMATION

Content contact:

Patricia Dabbs

202–502–7332

To obtain single copies of this report, while supplies last, or ordering information on other U.S. Department

of Education products, call toll free 1–877– 4ED PUBS (877–433–7827), or write:

Education Publications Center (ED Pubs)

U.S. Department of Education

P.O. Box 1398

Jessup, MD 20794–1398

TTY/TDD 1–877–576–7734

FAX 301–470–1244

Online ordering via the Internet: http://www.ed.gov/pubs/edpubs.html

Copies also are available in alternate formats upon request.

This report also is available on the World Wide Web: http://nces.ed.gov/nationsreportcard

Due to the confidential nature of NAEP surveys, the photographs throughout this report do not portray

actual students who participated in the NAEP civics assessment. All photographs used are from Comstock,

EyeWire Images, The Stock Market, and PhotoDisc stock libraries.

The work upon which this publication is based was performed for the

National Center for Education Statistics by Educational Testing Service.
NAEP CIVICS ASSESSMENTS–1988 AND 1998 • TABLE OF CONTENTS iii

TABLE OF CONTENTS

vii EXECUTIVE SUMMARY

1 INTRODUCTION

1 Civic Engagement

2 Political Knowledge Among Youths

3 Origins of the 1988 and 1998 Civics Special Study

4 The Assessment Instrument

6 Description of School and Student Samples

6 Reporting the Civics Special Study Results

7 Cautions in Interpretation

9 CHAPTER 1. 1988 and 1998 Civics Assessment Results for the Nation

and Sample Questions

9 Results for Selected Subgroups

9 Gender

10 Race/Ethnicity

12 Sample Questions by Civics Content Area

12 Democratic Principles and the Purpose of Government

15 Political Institutions (Structures and Functions)

17 Political Processes

21 Rights, Responsibilities, and the Law

24 Summary

25 CHAPTER 2. Trends in Civics Topics Studied

25 Overview

25 Trends in the Frequency of Social Studies/Civics Classes and Years of Study

27 Trends in Amount of Study of Selected Topics in American Government or Civics

30 Trends in the Amount of Civics Coursework Completed

31 Summary

33 CHAPTER 3. Instructional Practices and Out-of-School Contexts

33 Overview

33 Civics Instructional Practices

36 Out-of-School Contexts

38 Summary

39 APPENDIX A. Overview of Procedures Used in the 1988 and 1998 NAEP Civics

Special Study

51 APPENDIX B. Standard Errors

65 APPENDIX C. Average Percentage Correct

71 APPENDIX D. Members of the NAEP Civics Standing Committee

73 ACKNOWLEDGMENTS
iv NAEP CIVICS ASSESSMENTS–1988 AND 1998 • TABLE OF CONTENTS

TABLES AND FIGURES

5 Table i.1: Distribution of questions and percentage of total assessment by content area,

grades 4, 8, and 12: 1988 and 1998

6 Table i.2: Number of students assessed and participating schools,

grades 4, 8, and 12, civics trend: 1998

9 Table 1.1: Average percentage of correct responses to civics questions by grade: 1988 and 1998

10 Table 1.2: Average percentage of correct responses to civics questions by gender: 1988 and 1998

11 Table 1.3: Average percentage of correct responses to civics questions by race/ethnicity:

1988 and 1998

12 Figure 1.1: Grade 4 sample questions, 1988 and 1998, Content area: Democratic principles and

the purpose of government

13 Figure 1.2: Grade 8 sample questions, 1988 and 1998, Content area: Democratic principles and

the purpose of government

14 Figure 1.3: Grade 12 sample questions, 1988 and 1998, Content area: Democratic principles and

the purpose of government

15 Figure 1.4: Grade 4 sample questions, 1988 and 1998, Content area:

Political institutions (structures and functions)

16 Figure 1.5: Grade 8 sample questions, 1988 and 1998, Content area:

Political institutions (structures and functions)

17 Figure 1.6: Grade 12 sample questions, 1988 and 1998, Content area:

Political institutions (structures and functions)

18 Figure 1.7: Grade 4 sample questions, 1988 and 1998, Content area: Political processes

19 Figure 1.8: Grade 8 sample questions, 1988 and 1998, Content area: Political processes

20 Figure 1.9: Grade 12 sample questions, 1988 and 1998, Content area: Political processes

21 Figure 1.10: Grade 4 sample questions, 1988 and 1998, Content area:

Rights, responsibilities, and the law

22 Figure 1.11: Grade 8 sample questions, 1988 and 1998, Content area:

Rights, responsibilities, and the law

23 Figure 1.12: Grade 12 sample questions, 1988 and 1998, Content area:

Rights, responsibilities, and the law

25 Table 2.1: Percentage of students reporting the frequency of social studies class by type of school,

grade 4: 1988 and 1998

26 Table 2.2: Percentage of students reporting they studied American government or civics by

grade levels: grades 8 and 12: 1988 and 1998

28 Table 2.3: Percentage of students who reported how much they studied selected civics topics,

grade 4: 1988 and 1998

29 Table 2.4: Percentage of students who reported how much they studied selected civics topics,

grade 8: 1988 and 1998

29 Table 2.5: Percentage of students who reported how much they studied selected civics topics,

grade 12: 1988 and 1998

30 Table 2.6: Percentage of students by number of years of civics studied, grade 12: 1988 and 1998

34 Figure 3.1: Percentage of students doing group projects at least once a week

34 Table 3.1: Percentage of students reporting how often their teachers use various classroom practices,

grade 8: 1988 and 1998

35 Figure 3.2: Percentage of students reading extra material

35 Table 3.2: Percentage of students reporting how often their teachers use various classroom practices,

grade 12: 1988 and 1998

36 Table 3.3: Percentage of students reporting how often they discuss current events,

grade 4: 1988 and 1998
NAEP CIVICS ASSESSMENTS–1988 AND 1998 • TABLE OF CONTENTS v

37 Table 3.4: Percentage of students reporting the amount of time they spend weekly on civics or

government homework, grade 12: 1988 and 1998

37 Table 3.5: Percentage of students reporting receiving newspapers in the home,

grades 4, 8, and 12: 1988 and 1998

40 Table A.1: NAEP civics special study sample sizes, grades 4, 8 and 12: 1988 and 1998

41 Table A.2: Weighted participation rates (percentages) by grade and subject type, 1998 NAEP civics

special study reporting samples

42 Table A.3: Students with disabilities and limited English proficient students excluded from the NAEP

civics special study: National sample, public and nonpublic schools combined: 1998

51 Table B1.1: Standard errors for average percentage of correct responses to civics questions by grade:

1988 and 1998

51 Table B1.2: Standard errors for average percentage of correct responses to civics questions

by gender: 1988 and 1998

52 Table B1.3: Standard errors for average percentage of correct responses to civics questions

by race/ethnicity: 1988 and 1998

52 Figure B1.1: Standard errors for grade 4 sample questions, 1988 and 1998, Content area:

Democratic principles and the purpose of government

53 Figure B1.2: Standard errors for grade 8 sample questions, 1988 and 1998, Content area:

Democratic principles and the purpose of government

53 Figure B1.3: Standard errors for grade 12 sample questions, 1988 and 1998, Content area:

Democratic principles and the purpose of government

54 Figure B1.4: Standard errors for grade 4 sample questions, 1988 and 1998, Content area:

Political institutions (structures and functions)

54 Figure B1.5: Standard errors for grade 8 sample questions, 1988 and 1998, Content area:

Political institutions (structures and functions)

55 Figure B1.6: Standard errors for grade 12 sample questions, 1988 and 1998, Content area:

Political institutions (structures and functions)

55 Figure B1.7: Standard errors for grade 4 sample questions, 1988 and 1998, Content area:

Political processes

56 Figure B1.8: Standard errors for grade 8 sample questions, 1988 and 1998, Content area:

Political processes

56 Figure B1.9: Standard errors for grade 12 sample questions, 1988 and 1998, Content area:

Political processes

57 Figure B1.10: Standard errors for grade 4 sample questions: 1988 and 1998, Content area:

Rights, responsibilities, and the law

57 Figure B1.11: Standard errors for grade 8 sample questions, 1988 and 1998, Content area:

Rights, responsibilities, and the law

58 Figure B1.12: Standard errors for grade 12 sample questions, 1988 and 1998, Content area:

Rights, responsibilities, and the law

58 Table B2.1: Standard errors for percentage of students reporting frequency of social studies class by

type of school, grade 4: 1988 and 1998

59 Table B2.2: Standard errors for percentage of students reporting they studied American government or

civics by grade levels, grades 8 and 12: 1988 and 1998

59 Table B2.3: Standard errors for percentage of students who reported how much they studied selected

civics topics, grade 4: 1988 and 1998

60 Table B2.4: Standard errors for percentage of students who reported how much they studied selected

civics topics, grade 8: 1988 and 1998
vi NAEP CIVICS ASSESSMENTS–1988 AND 1998 • TABLE OF CONTENTS

60 Table B2.5: Standard errors for percentage of students who reported how much they studied selected

civics topics, grade 12: 1988 and 1998

61 Table B2.6: Standard errors for percentage of students by number of years of civics studied,

grade 12: 1988 and 1998

61 Table B3.1: Standard errors for percentage of students reporting how often their teachers

use various classroom practices, grade 8: 1988 and 1998

62 Table B3.2: Standard errors for percentage of students reporting how often their teachers

use various classroom practices, grade 12: 1988 and 1998

62 Table B3.3: Standard errors for percentage of students reporting how often they discuss current

events, grade 4: 1988 and 1998

63 Table B3.4: Standard errors for percentage of students reporting the amount of time they spend

weekly on civics or government homework, grade 12: 1988 and 1998

63 Table B3.5: Standard errors for percentage of students reporting receiving newspapers in the home,

grades 4, 8, and 12: 1988 and 1998

65 Table C.1: Average percentage of correct responses to each test question in the civics special study,

grade 4: 1988 and 1998

67 Table C.2: Average percentage of correct responses to each test question in the civics special study,

grade 8: 1988 and 1998

69 Table C.3: Average percentage of correct responses to each test question in the civics special study,

grade 12: 1988 and 1998
NAEP CIVICS ASSESSMENTS–1988 AND 1998 • EXECUTIVE SUMMARY vii

EXECUTIVE SUMMARY

As we move into the twenty-first century,

our nation looks to its youth for

confirmation that the government established

over 200 years ago will remain relevant, vital, and

strong. We expect that today’s students are being

prepared to understand and maintain the values

of our democratic society. Civics education in our

nation’s schools informs students about the

structures, functions, and processes of

government and about the meaningful ways in

which citizens can make decisions about public

issues and participate in governance. This report,

based on findings from the 1998 National

Assessment of Educational Progress (NAEP),

provides a view of students’ achievement in civics

over the ten-year period from 1988 to 1998.

The data and information provided give some

indication as to whether there have been changes

in students’ understanding of civics and whether

civics education has changed during the last

decade of the twentieth century.

The National Assessment of Educational

Progress is the nation’s only ongoing survey of

what students know and can do in various

academic subject areas. Authorized by Congress

and administered by the National Center for

Education Statistics in the U.S. Department of

Education, NAEP regularly reports to the

public on the educational progress of students

in grades 4, 8, and 12. In 1998, NAEP conducted

two national assessments of students’ civics

knowledge in each of these grades. One

1 Lutkus, A.D., Weiss, A.R., Campbell, J.R., Mazzeo, J., &

Lazer, S. (1999). NAEP 1998 civics report card for the nation.

Washington, DC: National Center for Education Statistics.

assessment employed a set of new test

CIVICS SUMMARY

specifications (or “framework”).

The other civics assessment (based on the

1988 civics objectives) was a special study that

repeated a number of the multiple-choice test

questions used in 1988. The results of the new

assessment were reported in the fall of 1999 as

the NAEP 1998 Civics Report Card for the

Nation.

1

 The results of the special study are

reported here as a summary of trends in

students’ knowledge and teachers’ classroom

practices over the ten-year period.

The results are based on the assessment of

a sample of students at each grade that is

statistically representative of the entire nation.

Students’ performance is described in terms of

average percentage correct, rather than the

traditional NAEP scaled scores. The reason for

this departure is that the relatively small set of

test questions repeated from 1988 in grades 8

and 12 did not allow comprehensive coverage

of the 1988 test objectives, nor did it allow the

reliable development of scaled scores parallel to

those used in 1988.

This report provides results for subgroups

of students defined by various background and

contextual characteristics. The analyses focus on

differences between 1988 and 1998, rather than

differences among groups within each year.
viii NAEP CIVICS ASSESSMENTS–1988 AND 1998 • EXECUTIVE SUMMARY

To illustrate the civics knowledge that was

assessed, numerous samples of the test questions

are provided. The report also explores trends in

the classroom coverage of civics topics from

1988 to 1998, as well as trends in classroom

instructional activities over the decade. A

summary of the major findings from the NAEP

1998 special study is presented below.

Overall Assessment Results

 In both 1988 and 1998, students at each of

the three grade levels answered about twothirds of the assessment questions correctly.

 Fourth-grade students in 1998 answered

more questions correctly, on average, than

did fourth-grade students in 1988.

 Eighth-grade students in 1998 answered

fewer questions correctly, on average, than

their counterparts in 1988.

 The performance of twelfth-grade students

in 1998 was not significantly different from

their counterparts in 1988.

Results for Student Subgroups

Gender

 Fourth-grade males had a higher percentage

of correct responses in 1998 than in 1988,

while twelfth-grade males had a lower

percentage correct in 1998.

 The percentage of correct responses for

female students at grades 4, 8, and 12 did

not change significantly between 1988

and 1998.

Race/Ethnicity

 In 1998, the percentage of correct responses

increased for white students in grade 4 and

decreased for Hispanic students in grade 12.

 At all three grades in both 1988 and 1998,

white students consistently achieved a

higher percentage correct than either black

or Hispanic students.

Trends in Civics Topics Studied

 A trend was noted toward less frequent

social studies classes in grade 4, with

49 percent of students in 1988 reporting

daily classes compared to 39 percent in 1998.

 The percentage of eighth-graders who

reported having studied civics or American

government in grades 5, 6, and 7 rose

between 1988 and 1998.

 The percentage of twelfth-graders who said

they were studying civics or American

government in their current grade rose

between 1988 (61 percent) and 1998

(71 percent).

 Fourth-graders in 1998 reported spending

more time studying three of the six civics

curriculum topics surveyed (elections and

voting, President/leaders of the country,

and judges and courts) than did fourthgraders in 1988.

 For both eighth- and twelfth-graders, the

U.S. Constitution and the Bill of Rights

is the one curriculum topic, of all those

surveyed, that was studied “A lot”

by the majority of students in both 1988

and 1998.
NAEP CIVICS ASSESSMENTS–1988 AND 1998 • EXECUTIVE SUMMARY ix

 The amount of time spent studying the

various civics topics surveyed at grades 8

and 12 is similar and has not changed

between 1988 and 1998.

Trends in Contexts for

Learning Civics

 The frequency with which students at grades

8 and 12 were assigned extra reading material

by their civics or American government

teachers increased between 1988 and 1998.

 The percentage of eighth- and twelfthgraders assigned to work on group projects

at least once or twice a week rose

substantially from 1988 to 1998.

 The percentage of fourth-graders discussing

current events in social studies class at least

once a week increased from 29 percent in

1988 to 39 percent in 1998.

 The amount of civics homework twelfthgraders were doing did not change

significantly between 1988 and 1998.

 Between 1988 and 1998, the percentage of

eighth-graders whose families regularly got a

newspaper fell from 77 percent to 71 percent.

For twelfth-graders, the percentage dropped

from 82 percent to 75 percent.

NAEP CIVICS ASSESSMENTS–1988 AND 1998 • INTRODUCTION 1

INTRODUCTION

INTRODUCTION

f today’s youth are to become successful civic

leaders of tomorrow, they need to be both

engaged in and knowledgeable about civic

matters. There is some concern that they may

not be up to the task. Is this concern well

founded?

Civic Engagement

Political engagement in the United States is low.

Most notably, voter turnout has fallen

dramatically. Only a generation ago, in 1960,

turnout in presidential elections reached a

modern-day high, with over 65 percent of the

eligible electorate going to the polls. By 1996,

that percentage had dropped to less than 51.

Two other measures of democracy—political

and interpersonal trust—fell sharply over the

same period. In the early 1960s, more than

three quarters of Americans said that the federal

government could usually or just about always

“be trusted to do the right thing”; in the 1990s,

that figure was closer to one quarter.1

 More

broadly, participation in numerous forms of

politics and civil society is reportedly down

sharply.2

Moreover, lack of engagement is especially

noticeable among America’s youth. Among

1 National Commission on Civic Renewal. (1997). A nation of

spectators: How civic disengagement weakens America and what we

can do about it. Washington, DC: Author.

2 Putnam, R. (1995). Bowling alone. Journal of Democracy (9),

65–78.

18–24-year-olds, less than one-third voted in

1996, which is considerably less than among

both older Americans and young people in

earlier election years. In surveys of college

freshmen, the numbers who claim to discuss

politics, keep up-to-date on politics, and vote in

student elections are all at record lows.3 “Being

a good American who cares about the good of

the country” has come to rank relatively low

among young people’s priorities.4

 Overall,

young adults are said to be indifferent,

distrustful, and politically disengaged, thus

raising the prospect of even greater attrition in

political and civic involvement in the future.

While it is true that participation in many

kinds of civic activities has declined, civic

volunteerism is high and, by some accounts, is

on the rise. In the 1996 National Household

Education Survey, for example, just over half of

young people in grades 9 through 12 said they

participated in community service activity or

volunteer work at school or in the community

during that school year.5

 In another study,

3 Astin, A.W., Parrott, S.A., Korn, W.S., & Sax, L.J. (1997).

The American freshman: Thirty year trends. Los Angeles: Higher

Education Research Institute, University of California at

Los Angeles.

4 National Association of Secretaries of State. (1999). New

Millennium Project, Part I: American youth attitudes on politics,

citizenship, government and voting. Washington, DC: Author.

5 Nolin, M.J., Chaney, B., & Chapman, C. (1997). Student

participation in community service activity. (NCES 97–331).

Washington, DC: National Center for Education Statistics.

I
2 NAEP CIVICS ASSESSMENTS–1988 AND 1998 • INTRODUCTION

which asked teenagers about 15 different areas

of activity, 60 percent of 14–17-year-olds

claimed to have performed volunteer activity in

the past twelve months.6

 Among college

freshmen, the percentage reporting that they

had performed volunteer work during the past

year reached an all-time high of 75 percent in

1998, after ranging between 62 and 73 percent

over the prior 14 years. However, these same

students’ commitment to broader forms of civic

involvement—community leadership, improving

racial understanding, helping those in difficulty,

and cleaning up the environment—is static or

trending lower.7

The growing evidence of a decline in citizen

participation and commitment has led to a call

for a “civic renewal.” High on the list of recommendations for such a renewal is a revitalization

of civic education, including teaching more civics

in schools, instituting character education, and

encouraging or requiring community service.8

Political Knowledge

Among Youths

Advocates of more civic education in the schools

point out that the problem is deeper than a lack

of political participation: young people are also

uninformed. The results of a number of studies,

including recent NAEP assessments in geography,

history, and civics, all seem to point to deficiencies

6 Hodgkinson, V.A. and Weitzman, M.S. (1996). Giving and

volunteering in the U.S. Washington, DC: Independent Sector.

7 Sax, L.J., Astin, A.W., Korn, W.S., & Mahoney, K. (1999). The

American freshman: National norms for fall 1999. Los Angeles:

Higher Education Research Institute, UCLA.

8 National Commission on Civic Renewal. (1998). A nation of

spectators. Washington, DC: Author.

Council on Civil Society. (1998). A call to civil society: Why

democracy needs moral truths. New York: Institute for American

Values.

Brown University. (1999). Presidents’ Fourth of July declaration

on the civic responsibility of higher education. Providence, RI:

Brown University, Campus Compact.

National Association of Secretaries of State. (1999). New

millennium project, part I: American youth attitudes on policies,

citizenship, government, and voting. Washington, DC: Author.

in young people’s knowledge of the social and

geopolitical world.9

 For example, based on a

nationally representative survey of 18–24-yearolds, a report from the National Association of

Secretaries of State concluded that young people

“lack any real understanding of citizenship...,

information and understanding about the

democratic process..., and information about

candidates and political parties.”10 Likewise, an

extensive analysis of more than 50 questions

covering governmental structure, historical

persons and events, and contemporary politics

showed that young people (post-baby boomers)

were often the least knowledgeable of the

people surveyed.11 Furthermore, the Secretaries

of State report noted that most young people

do not seek out political information and that

they are not very likely to do so in the future.12

Despite evidence of low political engagement,

low levels of political and interpersonal trust,

and relatively low levels of civic knowledge,

there is little to suggest that today’s youths are

less knowledgeable than their predecessors.

9 National Assessment of Educational Progress. (1990). The

geographic learning of high school seniors. Washington, DC. Author.

Anderson, L., Jenkins, L.B., Lerning, J., MacDonald, W.B.,

Mullis, I.V.S., Turner, M.J., & Wooster, J.S. (1990). The civics

report card–trends in achievement from 1976 to 1988 at ages 13

and 17–achievement in 1988 at grades 4, 8, and 12. Washington,

DC: National Center for Education Statistics.

Lutkus, A.D., Weiss, A.R., Campbell, J.R., Mazzeo, J., &

Lazer, S. (1999). NAEP 1998 civics report card for the nation.

Washington, DC: National Center for Education Statistics.

 10Brown University. (1999). Presidents’ Fourth of July declaration

on the civic responsibility of higher education. Providence, RI:

Brown University, Campus Compact.

National Association of Secretaries of State. (1999). New

millennium project, part I: American youth attitudes on policies,

citizenship, government, and voting. Washington, DC: Author,

34–38.

 11Delli Carpini, M.X. and Keeter, S. (1996). What Americans

know about politics and why it matters. New Haven: Yale

University Press.

12Brown University. (1999). Presidents’ Fourth of July declaration

on the civic responsibility of higher education. Providence, RI:

Brown University, Campus Compact.

National Association of Secretaries of State. (1999). New

millennium project, part I: American youth attitudes on policies,

citizenship, government, and voting. Washington, DC:

Author, 35–36.
NAEP CIVICS ASSESSMENTS–1988 AND 1998 • INTRODUCTION 3

One recent study did find that the knowledge

gap between people aged 18–29 and those

aged 45–54 increased dramatically between

mid-century and 1989, but the comparison was

based on an extremely small number of questions

and cannot be considered conclusive.13 Moreover, major changes in student demographics

over this period—for example, a twelfth-grade

population that is far more diverse today than it

was half a century ago—make such comparisons

and interpretations even more difficult than they

would be otherwise.

NAEP itself has examined changes in civics

knowledge levels over time for 13- and 17-yearolds using studies conducted in 1976, 1982,

and 1988.14 This type of comparison has the

advantage of “holding constant” the education

level of the students taking the assessment. The

results show that changes in knowledge levels

were small and inconsistent. On a 0–100

“proficiency scale,” 13-year-olds performed as

well as or better in 1988, but 17-year-olds

performed less well (declining by 2 points)

between 1976 and 1988. However, on a

constructed-response (open-ended) question

about the role of community newspapers,

13-year-olds gave acceptable answers less often

in 1988 than in 1976, while the change was

insignificant among 17-year-olds. Finally, on a

constructed-response question on the definition

of democracy asked only of 17-year-olds, there

was a significant increase in acceptable answers.

Finally, it appears that changing trends in

the way information about government and

13Delli Carpini, M.X. and Keeter, S. (1996). What Americans

know about politics and why it matters. New Haven: Yale

University Press.

14Anderson, L., Jenkins, L.B., Lerning, J., MacDonald, W.B.,

Mullis, I.V.S., Turner, M.J., & Wooster, J.S. (1990). The civics

report card–trends in achievement from 1976 to 1988 at ages 13

and 17–achievement in 1988 at grades 4, 8, and 12. Washington,

DC: National Center for Education Statistics. (Note that the

sample for this trends assessment was based on age rather than

grade.)

politics is obtained—less newspaper reading,

more television watching, and the growth of

new technologies like the Internet—do not

necessarily lead to decreased awareness of

political subjects. A frequently cited example is

that “cop shows” on television could be one of

the reasons that students in the 1988 NAEP

civics assessment were often able to answer

questions about civil and criminal rights.15

The relationship between civic knowledge

and civic participation is not simple and direct.

The trend toward political disaffection and

disengagement among young people seems

quite clear, but has not been accompanied by a

parallel trend in decreased civic knowledge.

Educational policymakers, however, do believe

that enhancing civic knowledge—with a

particular stress on civic responsibility—will, in

time, lead to greater participation. In the

meantime, NAEP will continue to monitor the

progress American schools make towards teaching young people the fundamental concepts of

government and civic life. Toward that goal, this

report compares the results of civics assessments

administered in 1988 and 1998.

Origins of the 1988 and 1998

Civics Special Study

The publication of A Nation at Risk16 in 1983

led to many changes in education, including a

movement toward standards-based education,

greater use of portfolios and other such means

of assessment, and an increased visibility of state

and national assessments. Each of these

developments had an effect in the area of civics.

15Niemi, R.G. and Junn, J. (1998). Civic education: What makes

students learn? New Haven: Yale University Press.

16National Commission on Excellence in Education. (1983). A

nation at risk: The imperative for educational reform: A report to

the nation and the Secretary of Education. Washington, DC:

Author.
4 NAEP CIVICS ASSESSMENTS–1988 AND 1998 • INTRODUCTION

A major effort by the Center for Civic

Education, with the assistance of numerous

individuals and organizations, resulted in the

1994 publication of the voluntary National

Standards for Civics and Government.

17

The Standards volume was premised on the

belief that civic instruction was often overlooked

due to a false belief that civic knowledge and

skills “emerge as by-products of the study of

other disciplines or as an outcome of the process

of schooling itself.”18 It identified five major

themes that defined the content of civics

knowledge and provided detailed descriptions of

“not only the content to be mastered in civics

and government, but also what students should

be able to do in relation to that content.”19

The NAEP Civics Consensus Project relied

heavily on the Standards when it constructed

the Civics Framework for the 1998 National

Assessment of Educational Progress. A decision

was made very early to construct an entirely new

test rather than repeating or simply revising the

1988 instrument. The new assessment would

include questions about topics that received

little or no attention in the previous assessment,

but were regarded as vital to a proper education

for democracy. These included, among many

other examples, more emphasis on the gap

between the goals and reality of American

democracy, the skills needed by citizens for

monitoring government, and the place of the

United States in world affairs. Also, at the

specific direction of the National Assessment

Governing Board (NAGB) which sets policy

for NAEP, there was an emphasis on civil

society, that is, voluntary associations and

17Center for Civic Education. (1994). National standards for civics

and government. Calabasas, CA: Author.

18 Ibid., p.2.

19 Center for Civic Education. (1994). National standards for civics

and government. Calabasas, CA: Author.

nongovernmental organizations.20 In addition,

it was decided that the format of the assessment

should differ from that used in 1988. Nearly

half of the new assessment would be devoted to

short- or extended-constructed-response

questions rather than multiple-choice questions,

and much more use would be made of stimulus

material such as political cartoons and major

historical documents.

Because the 1998 assessment was to be based

upon a new framework, its results would not be

comparable to those from 1988. The Steering

and Planning committees for the assessment

thus strongly recommended that the 1998

assessment include a special study that would

look back to assess change in student

performance between 1988 and 1998. This was

to be accomplished by using a limited-sized

sample and by readministering a portion of the

1988 assessment under the same conditions as

those used in 1988. This report presents the

results of that special study. Details of the 1988

and 1998 assessment and the procedure used in

implementing it are summarized below and

explained more fully in Appendix A.

The Assessment Instrument

The questions used in the special study were

written for the NAEP 1988 civics assessment

and were based on the 1988 Civics Objectives.21

The Objectives used a three-dimensional matrix

to represent the interrelated components

of the assessment questions: content, cognitive

skills, and the context in which questions were

asked. Although all three components were

taken into account in developing the assessment

20National Assessment Governing Board. (1998). Civics framework for the 1998 National Assessment of Educational Progress,

Washington, DC: Author.

21National Assessment of Educational Progress. (1987). Civics

objectives: 1988 assessment. Princeton, NJ: Educational Testing

Service.
NAEP CIVICS ASSESSMENTS–1988 AND 1998 • INTRODUCTION 5

questions, only the content component has been

used in analyzing the trend results.22

The objectives outlined four content areas:

 Democratic principles and the purpose of

government

 Political institutions (structures and functions)

 Political processes

 Rights, responsibilities, and the law

Table i.1 shows the distribution of questions

across the four content areas.

For the special study, NAEP administered

three 1988 test booklets, one at each of grades

4, 8, and 12. The booklets contained a block

of general background questions, a block of

background questions related to social studies

or civics instruction, and three blocks of

cognitive (or “content”) questions. The three

test booklets were administered intact; that is,

they were identical in content and appearance

to the booklets administered in 1988.

In 1988 there was only one test booklet

developed at grade 4; it contained three

ten-minute blocks of multiple-choice questions.

Consequently, fourth-grade students in the

special study received all of the 51 questions

from 1988. The 1988 assessments at grade 8

and grade 12 were each comprised of seven

fifteen-minute blocks of cognitive questions

(six blocks of multiple-choice questions and one

block containing a single open-ended question),

with each participating student receiving three

blocks. The test booklets chosen for administration in the special study contained three

22Because of discrepancies in the data records concerning the

content classifications assigned to the assessment questions in

1988, all questions used for the trend study were reclassified and

the new classifications were used for the analyses in this report.

†One question was deleted from the analysis for technical reasons (see Appendix A).

‡Percentages may not add to 100 due to rounding.

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP),

Civics Assessments–1988 and 1998.

Democratic principles Rights,

and the purpose Political Political responsibilities,

of government institutions processes and the law Total

1988 1998 1988 1998 1988 1998 1988 1998 1988 1998

Table i.1

Distribution of questions and percentage of total assessment by content area,

grades 4, 8, and 12: 1988 and 1998

Grade 4

Number 9 9 26 26 5 5 11 11 51 51

Percentage of

total questions‡ 18 18 51 51 10 10 22 22 100 100

Grade 8

Number 21 8 49 24 39 21† 43 24 152 77

Percentage of

total questions‡ 14 10 32 31 26 27 28 31 100 100

Grade 12

Number 19 7 46 28 49 18 35 21† 149 74

Percentage of

total questions‡ 13 9 31 38 33 24 23 28 100 100
6 NAEP CIVICS ASSESSMENTS–1988 AND 1998 • INTRODUCTION

blocks of multiple-choice questions. As a result,

at grades 8 and 12, the study included approximately half of the total pool of questions from

the original assessment, 77 questions for eighthgraders, and 74 questions for twelfth-graders.

For each grade level, Table i.1 displays the

number of test questions used in each

assessment year, as well as the percentage of the

total assessment that was devoted to each

content category. Because the entire set of test

questions was repeated for grade 4, the

percentages in the “Percentage of total

questions” row are identical for 1988 and 1998.

For grades 8 and 12, inspection of the within

category pairs in the “Percentage of total

questions” row indicates that the percentages of

the total assessment from each content category

were quite similar in the 1988 assessment and

the 1998 special study, even though the special

study employed only about half as many test

questions as the 1988 assessment.

Description of School

and Student Samples

The NAEP 1998 civics special study was

conducted nationally at grades 4, 8, and 12.

The assessments were administered between

January and the end of March 1998 in the same

schools that hosted the 1998 civics national

assessment (and provided the data for the NAEP

1998 Civics Report Card for the Nation). A

representative sample of public and nonpublic

school students, selected through stratified

random sampling procedures, was assessed.

Thus, the results presented in this report are based

on representative samples of students.

As with the main NAEP assessments,

schools were permitted to exclude students with

disabilities (SD) and limited English proficient

students (LEP) if they were unable to participate

meaningfully in the assessment. Details on the

numbers and percentages of students with

disabilities and students with limited English

proficiency who were excluded from the

assessment are given in Appendix A. In order to

preserve the accuracy of the 1988 and 1998

comparison, testing accommodations or

adaptations were not offered to students with

disabilities or limited English proficiency because

these were not available in the 1988 assessment.

Table i.2 presents the data on student

sample sizes and on the number of schools that

participated in the civics special study.

Each selected school that participated in the

assessment, and each student assessed,

represents a portion of the target population.

Grade 4 2,088 111

Grade 8 2,055 104

Grade 12 2,193 102

SOURCE: National Center for Education Statistics, National Assessment of

Educational Progress (NAEP), Civics Assessments–1988 and 1998.

Number of assessed Number of

students schools

Table i.2

Number of students assessed and

participating schools, grades 4, 8, and 12,

civics trend: 1998

Reporting the Civics Special

Study Results

The results from the 1988 and 1998 assessments

are presented in this report in two ways. First,

student and subgroup performance is shown as

the average percentage correct achieved on the

assessment as a whole, and then on individual

questions illustrative of a content category. This

presentation is a departure from the traditional

use of a NAEP scale and achievement levels by

which to chart performance. Scaled scores were
NAEP CIVICS ASSESSMENTS–1988 AND 1998 • INTRODUCTION 7

not used because the reduced number of test

questions given at grades 8 and 12 both limited

coverage of the test objectives and made it

difficult to create reliable scales comparable

to those from 1988. The three levels of

achievement—Basic, Proficient, and Advanced—

now commonly used in reporting NAEP results

were not in use in 1988. There were four levels

of proficiency defined in 1988, but they were

derived from that assessment’s scale. Since the

special study assessment is not scaled, the

proficiency levels are not relevant.

Second, the results of contextual variables,

discussed in chapters 2 and 3, are presented as

student response percentages, typically with no

associated student performance data. Chapters 2

and 3 focus on trends in frequency of classes,

curriculum topics covered in civics, and

instructional activities. Changes in these areas

over a decade are of interest to educators and

curriculum developers even when not connected

to shifts in student performance.

Cautions in Interpretation

The reader is cautioned against using the NAEP

results in this report to make simple causal

inferences related to subgroup performance, to

the relative effectiveness of public and

nonpublic schools, or to other educational

variables discussed in this report. A relationship

that exists between performance and another

variable does not reveal the underlying cause of

that relationship, which may be influenced by a

number of other variables. Differences in civics

performance may reflect a range of

socioeconomic and educational factors not

discussed in this report or addressed by the

NAEP program. Finally, differences in civics

performance may reflect not only the

effectiveness of educational programs, but also

the challenges posed by economic constraints

and student demographic characteristics.

NAEP CIVICS ASSESSMENTS–1988 AND 1998 • CHAPTER 1 9

CHAPTER 1

ow do students in today’s fourth, eighth,

RESULTS & SAMPLES

and twelfth grades compare in knowledge

of civics topics to their counterparts in 1988? To

find out, a portion of the 1988 civics assessment

questions from grades 8 and 12, and the whole

set of questions from grade 4, were

readministered to students in the same grades in

the winter of 1998. A new civics “trend line”

will commence with the test questions from the

1998 national assessment, and will next be

reported after the 2003 civics assessment.

Overall, the civics special study revealed that

students in both 1988 and 1998 correctly

answered about two-thirds of the test questions.

There were small but significant changes between

1988 and 1998.

1988 and 1998 Civics

Assessment Results for the

Nation and Sample Questions

Table 1.1 presents the overall 1988 and

1998 results, by grade, for the nation.

Between 1988 and 1998, the average

percentage of correct responses rose for fourthgraders, but declined for eighth-graders.

Twelfth-graders in 1998 also had a lower overall

percentage correct than in 1988, but the

difference was not statistically significant. It

should be noted that the observed changes were

relatively small.

Results for Selected Subgroups

Tables 1.2 and 1.3 present results for two major

subgroups: gender and ethnicity. In addition to

reporting the percentage of students in each

subgroup, the tables include the average

percentage of correct responses, as an estimate

of the students’ overall performance in 1988

and 1998.

In reviewing the results in these tables, the

reader is cautioned against making simple or

causal inferences about the relationship between

subgroup membership and performance.

Gender

The 1988 and 1998 average percentage correct

data for males and females are presented in

Table 1.2. Looking at differences in performance

within gender groups across years, males at

grade 4 answered a higher percentage of items

Average percentage of correct responses

to civics questions by grade: 1988 and 1998

Table 1.1

Grade 4 62 < 64

Grade 8 64 > 62

Grade 12 68 66

> 1988 significantly greater than 1998.

< 1988 significantly less than 1998.

SOURCE: National Center for Education Statistics, National Assessment

of Educational Progress (NAEP), Civics Assessments–1988 and 1998.

1988 1998

H
10 NAEP CIVICS ASSESSMENTS–1988 AND 1998 • CHAPTER 1

correctly in 1998 than in 1988. At grade 12,

however, the percentage of correct responses for

males decreased from 70 percent in 1988 to 67

percent in 1998. The percentages correct for

female students at grades 4, 8, and 12 did not

vary significantly between 1988 and 1998.

Twelfth-grade males had a higher average

percentage of correct responses than females in

1988 (70 percent and 66 percent respectively).

This gap closed between 1988 and 1998 as a

result of the decrease in the average number of

correct responses by males from 70 percent to

67 percent.

Race/Ethnicity

The results for students in various racial/ethnic

subgroups are summarized in Table 1.3. Results

are included for white, black, and Hispanic

students in 1988 and 1998. Although both

Asian/Pacific Islander and American Indian

students were assessed in 1988 and 1998, the

sample sizes were too small to provide reliable

estimates for these subgroups.

The average percentage correct for white

students in grade 4 increased from 64 percent

in 1988 to 67 percent in 1998. Fourth-grade

black students also showed a higher percentage

Average percentage of correct responses to civics questions by gender:

1988 and 1998

Table 1.2

* 1998 is significantly different from 1988.

NOTE: Percentages of students may not add to 100 due to rounding.

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), Civics Assessments–

1988 and 1998.

GRADE 4

Female

Male 63

66

61

63

GRADE 8

Female

Male 65

62

64

63

0 20 40 60 80 100

GRADE 12

Female

Male 70

67

66

66

50

52

1988

1998

50

48

50

51

50

49

52

51

48

49

Percentage

of students

Percentage

of students

Percentage

of students

*

*
NAEP CIVICS ASSESSMENTS–1988 AND 1998 • CHAPTER 1 11

Table 1.3

* 1998 is significantly different from 1988.

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), Civics Assessments–

1988 and 1998.

Average percentage of correct responses to civics questions

by race/ethnicity: 1988 and 1998

GRADE 4

Black

White 69

67

1988

1998

16

15

70

67

15

15

10

14

 

Percentage

of students

Percentage

of students

Percentage

of students

Hispanic

GRADE 8

Black

White 67

66

56

54

Hispanic 57

55

GRADE 12

Black

White 70

69

58

58

Hispanic 62

58

0 20 40 60 80 100

11

14

75

70

12

14

 9

12

 

*

*

64

67

55

59

56

56

correct in 1998 compared to 1988, although

this difference was not statistically significant.

The average percentage correct for twelfthgrade Hispanic students decreased from 62

percent in 1988 to 58 percent in 1998.

In 1988 and 1998, white students had a

higher percentage of questions correct than

either black or Hispanic students at all three

grade levels. While both black and Hispanic

students averaged more than half of the questions

correct (between 54 and 62 percent), white

students averaged closer to two-thirds of the

questions correct (between 64 and 70 percent).
12 NAEP CIVICS ASSESSMENTS–1988 AND 1998 • CHAPTER 1

1 Educational Testing Service. (1987). Civics: United States

government and politics objectives: 1988 assessment. Princeton, NJ:

Educational Testing Service, National Assessment of Educational

Progress.

Sample Questions

by Civics Content Area

The test questions in the special study were

categorized by type of content according to the

specifications in the 1988 Civics Objectives.

1 The

four content areas were: Democratic principles

and the purpose of government; Political

institutions (structures and functions); Political

processes; and Rights, responsibilities, and the law.

The following section presents sample test

questions, by grade, for each of the content

areas. Student performance data for each

question are also displayed, showing the

percentage of students choosing each option.

Due to the limited number of questions in

some of the content areas and the fact that the

special study included only a subset of the entire

pool of questions for grades 8 and 12, the

performance results are not aggregated by

content area. The data reflect the change or

consistency in performance over time for the

individual test questions.

Democratic Principles and

the Purpose of Government

This content area covers the principles and ideas

expressed in the country’s core documents and

laws, the origins of the United States political

system and the purposes served by governments,

changes in constitutional values over time, and

the differences between the U.S. political system

and those in other countries. (Brief descriptions

Grade 4 sample questions, 1988 and 1998

Content area: Democratic principles and the purpose of government

Percentage of students

1988 1998

63 < 72

11 9

25 > 19

4 4

82 84

11 10

3 2

> 1988 significantly greater than 1998.

< 1988 significantly less than 1998.

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), Civics

Assessments–1988 and 1998.

Figure 1.1

Does a member of Congress ALWAYS have to obey the

laws of the United States?

Yes

B No

C I don’t know.

Which of the following is a major purpose of

government in the United States?

A To print newspapers

To make laws

C To make speeches

D To choose all reading materials

Percentage of students

1988 1998

63 < 72

11 9

25 > 19

4 4

82 84

11 10

3 2
NAEP CIVICS ASSESSMENTS–1988 AND 1998 • CHAPTER 1 13

of the repeated questions and student

performance in 1988 and 1998 are displayed in

Appendix C, under content code “1”).

The first question in Figure 1.1 measures

students’ understanding of a fundamental

principle of the rule of law. Fewer than twothirds of students answered it correctly in 1988,

rising to nearly three-fourths in 1998. One

possible explanation for the increase in

performance on this item is that events at the

time the assessment was administered in 1998

contributed to an increased awareness among

students of issues related to this topic. It should

be noted that the percentage of correct responses

increased and the percentage of responses in the

“I don’t know” category decreased. In the

second question in Figure 1.1, better than

four out of five fourth-graders could identify that

making laws is a major function of government.

Performance was consistent over time.

In the first question in Figure 1.2, shown

below, approximately 6 out of 10 eighth-graders

in both 1988 and 1998 knew that the Constitution is the document that sets forth the basic

rights of United States citizens.

In the next question, less than half of eighthgraders knew that countries other than the United

States have written constitutions. Performance

did not differ significantly over the decade on

either question.

If citizens of the United States want to find a statement

of their civil rights, in which one of the following should

they look?

A In the Bible

In the Constitution

C In the Articles of Confederation

D In the Declaration of Independence

E I don’t know.

Is the United States the only country which has

a written constitution?

A Yes

No

C I don’t know.

Grade 8 sample questions, 1988 and 1998

Content area: Democratic principles and the purpose of government

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), Civics

Assessments–1988 and 1998.

Figure 1.2

Percentage of students

1988 1998

3 2

61 62

12 11

17 19

7 6

32 28

43 45

25 27
14 NAEP CIVICS ASSESSMENTS–1988 AND 1998 • CHAPTER 1

In the first question in Figure 1.3 below,

twelfth-grade students were asked to identify

which historical document was the source of the

quote. Fifty-nine percent of students in 1988

correctly chose the Declaration of Independence,

but only half of the students in 1998 knew the

answer.

The second question measured whether

students could define the term “bicameralism,”

which describes the structure of the U.S. Congress

as well as most of the nation’s state legislatures.

Approximately one-third of students in either

year scored correctly, with nearly as many thinking

that bicameralism referred to a government with

two branches.

Grade 12 sample questions, 1988 and 1998

Content area: Democratic principles and the purpose of government

> 1988 significantly greater than 1998.

< 1988 significantly less than 1998.

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), Civics

Assessments–1988 and 1998.

Figure 1.3

“We hold these truths to be self-evident, that all men are

created equal, that they are endowed by their Creator with

certain unalienable Rights, that among these are Life,

Liberty, and the Pursuit of Happiness.”

This quotation is taken from the

Declaration of Independence

B United States Constitution

C Bill of Rights

D Articles of Confederation

Bicameralism is best defined as a

A government composed of two principal

branches

B multilevel judicial system containing a

higher court for appeals

C system of checks and balances between two

branches of government

legislative system composed of two houses

or chambers

Percentage of students

1988 1998

59 > 50

19 21

17 < 21

5 < 8

31 30

14 12

21 21

34 38
NAEP CIVICS ASSESSMENTS–1988 AND 1998 • CHAPTER 1 15

Political Institutions

(Structures and Functions)

The Political institutions (structures and

functions) content area covers basic historical

information about the development of the

United States political system, the organizational

principles of the government, and the

structures and operations of both the federal

government and state and local governments.

(Brief descriptions of the repeated questions

and student performance in 1988 and 1998

are displayed in Appendix C, under content

code “2”).

Student knowledge of important national

symbols falls within the political institutions

content area, and questions asking students to

identify symbols such as the flag and holidays

such as July 4 are appropriate for fourth-graders. In

the first question in Figure 1.4, we see a rather

large 15 percentage point gain between 1988

and 1998 in the ability of fourth-graders to

identify why Americans observe July 4.

Another type of question focused on fourthgraders’ ability to distinguish things done by

the government from things done by private

individuals. The second question in Figure 1.4 is

an example. Although students in 1998 were

better able than their 1988 counterparts to

identify gas stations as nongovernmental, fewer

than half answered correctly.

Grade 4 sample questions, 1988 and 1998

Content area: Political institutions (structures and functions)

> 1988 significantly greater than 1998.

< 1988 significantly less than 1998.

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), Civics

Assessments–1988 and 1998.

Figure 1.4

July 4 is a national holiday because it observes

the day that

the American colonies declared their

independence

B the Pilgrims arrived in the New World

C slaves won their freedom

D women won the right to vote in elections

In the United States, is a gas station usually

owned and run by some level of government?

A Yes

No

C I don’t know.

Percentage of students

1988 1998

64 < 79

9 > 5

20 > 11

6 4

29 > 24

39 < 45

32 31
16 NAEP CIVICS ASSESSMENTS–1988 AND 1998 • CHAPTER 1

Eighth-graders, like fourth-graders, were

asked to identify the proper functions of

government. Some questions focused on the

functions of the different levels or branches of

government while others, like the first question

below in Figure 1.5, compared what was within

to what was beyond the scope of government

responsibility. About three quarters of eighthgraders knew that the federal government is not

responsible for providing jobs for everyone,

although performance dropped from 1988

to 1998.

The next question in Figure 1.5 was a

straightforward attempt to determine whether

students knew the meaning of the term

“separation of powers.” About 6 out of 10

students answered correctly in each year.

Twelfth-graders were expected to be able to

identify the structures and functions of various

units of government, though generally with a

greater emphasis on the federal level than was

expected of fourth- and eighth-graders. The

samples below cover both aspects of this category.

Grade 8 sample questions, 1988 and 1998

Content area: Political institutions (structures and functions)

> 1988 significantly greater than 1998.

< 1988 significantly less than 1998.

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), Civics

Assessments–1988 and 1998.

Figure 1.5

Does the U.S. Federal Government provide jobs

for everyone?

A Yes

No

C I don’t know.

The term “separation of powers” refers to the

A state powers and duties that are not given to

the central government

division of authority among the legislative,

executive, and judicial branches of government

C division of the legislature into two houses

D creation of a national capital that is not a

part of any state

Percentage of students

1988 1998

14 < 19

77 > 71

9 10

12 15

58 61

25 > 19

4 5
NAEP CIVICS ASSESSMENTS–1988 AND 1998 • CHAPTER 1 17

The first question in Figure 1.6 asked

students about the basic structure of Congress.

About three-fourths knew the answer, but

performance declined by 5 percentage points

from 1988 to 1998.

Students found the second question in

Figure 1.6 to be quite difficult both years but

especially in 1998, when only 25 percent

identified the Department of State as being

most concerned with foreign affairs. This

represented a 10 percentage point drop in

performance. In 1998 twice as many students

incorrectly answered Department of Commerce

as chose the right answer.

Political Processes

The Political processes content category covered

a wide area of knowledge, including the range

and importance of decisions made by the federal

government and by state and local governments,

limits on the power of government, the legislative

process, the resolution of conflicts in law and

principles, political parties and interest groups,

and the electoral process. (Brief descriptions of

the repeated questions and student performance

in 1988 and 1998 are displayed in Appendix C,

under content code “3”).

Grade 12 sample questions, 1988 and 1998

Content area: Political institutions (structures and functions)

> 1988 significantly greater than 1998.

< 1988 significantly less than 1998.

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), Civics

Assessments–1988 and 1998.

Figure 1.6

The U.S. Congress is made up of which of the following?

A The House of Representatives and the

Supreme Court

The House of Representatives and the Senate

C The Senate and the Cabinet

D The President and the Cabinet

E I don’t know.

Which of the following departments is MOST concerned

with foreign affairs?

Department of State

B Department of Commerce

C Department of Agriculture

D Department of the Treasury

E I don’t know.

Percentage of students

1988 1998

8 10

77 > 72

5 6

3 < 5

7 7

35 > 25

45 50

6 8

5 5

9 < 13
18 NAEP CIVICS ASSESSMENTS–1988 AND 1998 • CHAPTER 1

In Figure 1.7, the first grade 4 question

deals with the qualifications for voting in the

United States. It was a fairly easy question, with

about 7 of 10 students answering correctly in

1988 and again in 1998.

The second question sought to find out

whether students knew who in the government

makes decisions about the constitutionality of

laws. Fourth-graders were not asked many

questions about the federal government, a topic

of which they have limited knowledge. In this

case, less than 20 percent in either year were

able to answer correctly.

To vote in a national or state election in the

United States, a person must usually be a

citizen

B parent

C worker

D homeowner

What person or group has the authority to decide

whether a law is constitutional?

A The President

B A governor

C The Congress

The Supreme Court

Grade 4 sample questions, 1988 and 1998

Content area: Political processes

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), Civics

Assessments–1988 and 1998.

Figure 1.7

Percentage of students

1988 1998

69 70

16 16

11 8

5 5

36 40

15 15

31 28

18 17
NAEP CIVICS ASSESSMENTS–1988 AND 1998 • CHAPTER 1 19

People in the United States elect their

government officials by

A public surveys

B television newscasts

C telegrams

secret ballots

What person or group has the authority to decide

whether a law is constitutional?

A The President

B A governor

C The Congress

The Supreme Court

Grade 8 sample questions, 1988 and 1998

Content area: Political processes

> 1988 significantly greater than 1998.

< 1988 significantly less than 1998.

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), Civics

Assessments–1988 and 1998.

Figure 1.8

Percentage of students

1988 1998

13 < 23

4 4

3 2

81 > 70

12 < 16

6 5

41 38

40 41

The first question in Figure 1.8 below is

about the electoral process. Eighth-grade

students performed well in both years, but there

was a large decline in the percentage giving the

correct answer, as well as a large increase in the

percentage choosing option A.

As can be seen, the second question is

identical to one given to fourth-graders (shown

in Figure 1.7). A number of questions in the

assessment were asked of students at more than

one grade but, of course, performance

expectations differed by grade. In this question,

4 out of 10 students knew that the Supreme Court

has the authority to decide the constitutionality

of laws. About as many students thought

Congress decided constitutionality as chose the

correct answer.
20 NAEP CIVICS ASSESSMENTS–1988 AND 1998 • CHAPTER 1

Twelfth-graders were expected to know that

the Constitution is the fundamental document

upon which the Supreme Court relies to

decide cases. In both 1988 and 1998 nearly 8

out of 10 did, as seen in the first question in

Figure 1.9 below.

Students were also expected to understand the

interdependence of decisions made by the various

branches and levels of government. The second

question asks about the interdependence between

state and national governments in the building

of interstate highways. Just over half of twelfthgrade students in each year knew the answer.

Which of the following documents is a major

basis for Supreme Court cases?

A Declaration of Independence

United States Constitution

C Mayflower Compact

D Emancipation Proclamation

Which of the following activities is an example

of cooperation between state and national governments?

A Printing money

Building interstate highways

C Collecting and delivering mail

D Making treaties

Grade 12 sample questions, 1988 and 1998

Content area: Political processes

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), Civics

Assessments–1988 and 1998.

Figure 1.9

Percentage of students

1988 1998

10 11

78 79

3 4

8 6

8 10

54 54

23 20

15 15
NAEP CIVICS ASSESSMENTS–1988 AND 1998 • CHAPTER 1 21

Rights, Responsibilities,

and the Law

The fourth content area, Rights, responsibilities,

and the law, covers such topics as the rights and

liberties guaranteed by the Constitution, the

judicial process, the purposes of laws, the

responsibilities of citizens to obey the law and

to participate in civic affairs, and the right to

dissent. (Brief descriptions of the repeated

questions and student performance in 1988

and 1998 are displayed in Appendix C, under

content code “4”).

In the first sample question in Figure 1.10

below, most fourth-graders in both 1988 and

1998 knew that a judge is in charge of a courtroom. But as the next question shows, only

about one-third of the students understood the

due process right to a jury trial for all criminal

defendants, even if there were eyewitnesses to

the crime. There was a significant increase in the

percentage of students who thought that the

defendant need not be given a jury trial.

Grade 4 sample questions, 1988 and 1998

Content area: Rights, responsibilities, and the law

< 1988 significantly less than 1998.

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), Civics

Assessments–1988 and 1998.

Figure 1.10

The person who is in charge of a courtroom

is called a

A juror

B lawyer

C defendant

judge

Suppose you are walking down the street and you

see a man run up to a woman and take her purse.

Just then a police officer rushes up, arrests the man,

and takes him off to jail.

Since you and the police officer both saw this man

rob the woman, should the man be given a trial by

jury if he wants it?

A No, he is guilty and should not be given a

trial by jury.

Yes, he should be given a trial by jury if he

wants one.

C It depends on whether or not he hurt the

woman.

D It depends on how much he stole.

E I don’t know.

Percentage of students

1988 1998

3 3

5 4

3 2

89 91

25 < 33

34 31

10 10

5 4

25 21
22 NAEP CIVICS ASSESSMENTS–1988 AND 1998 • CHAPTER 1

Grade 8 sample questions, 1988 and 1998

Content area: Rights, responsibilities, and the law

> 1988 significantly greater than 1998.

< 1988 significantly less than 1998.

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), Civics

Assessments–1988 and 1998.

Figure 1.11

Questions 15–17 are based on the following.

There are many ways that U.S. citizens can legitimately

oppose laws or actions they believe to be unjust.

Which of the following are legal ways for U.S. citizens

to oppose laws or the actions of officials and which

are not legal?

15. Writing letters of protest to authorities

Legal

B NOT legal

C I don’t know.

16. Not paying a portion of their taxes

A Legal

NOT legal

C I don’t know.

17. Participating in a boycott

Legal

B NOT legal

C I don’t know.

Does Congress have the right to pass a law to establish a

national church?

A Yes

No

C I don’t know.

Percentage of students

1988 1998

78 77

11 11

11 12

6 < 11

87 > 84

6 6

51 < 60

31 > 24

18 16

19 23

63 > 58

17 19

The first three questions shown in Figure 1.11

were part of a 10-question set for grade 8 on

the legality of different forms of dissent. Three

quarters of eighth-graders knew that it is legal

to write letters of protest to authorities. An even

higher percentage knew that it is illegal not to

pay part of their taxes, but there was a slight

decline between 1988 and 1998. More eighthgraders in 1998 were familiar with the idea that

participating in a boycott is a legal activity than

were their counterparts in 1988.
NAEP CIVICS ASSESSMENTS–1988 AND 1998 • CHAPTER 1 23

The fourth question in Figure 1.11 tested

whether eighth-graders were familiar with the

fundamental First Amendment freedom of

religion. As shown, just under two-thirds of the

1988 sample answered correctly. The number

slipped five percentage points, to 58 percent,

in 1998.

Judging by the questions given in the special

study, twelfth-graders have a very good sense of

their constitutional rights. A glance at Table C.3

in Appendix C of this report reveals that 80

percent or more of twelfth-graders in both 1988

and 1998 knew the answers to 14 of the 20

questions in the Rights, responsibilities, and the

law content area. Eight of the 20 questions

were answered correctly by 90 percent or more

of the students. Moreover, performance was

fairly consistent over time. On two of the

fourteen questions, both related to criminal

rights, performance declined by a statistically

significant amount between 1988 and 1998, but

on one of these the drop was less than 3

percentage points.

The two questions in Figure 1.12 were both

designed to measure students’ understanding of

the Constitution and individual rights. In the

first, students were asked to identify the fact that

the first 10 amendments to the Constitution focus

on individual liberties. Over 90 percent of

students answered correctly in both 1988 and

1998. The second question was the most

difficult, dealing with individual rights. Just

under two-thirds of students recognized that

the guarantee of freedom of religion is found in

the First Amendment to the Constitution. In

neither question was there any significant

change over time.

Most of the first ten amendments to the United States

Constitution deal with

individual rights

B the method of electing the President

C the taxing powers of the federal government

D the powers of the Supreme Court

The First Amendment to the Constitution guarantees

A the right of all citizens to vote

B the right to education

freedom of religion

D freedom from slavery

Grade 12 sample questions, 1988 and 1998

Content area: Rights, responsibilities, and the law

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), Civics

Assessments–1988 and 1998.

Figure 1.12

Percentage of students

1988 1998

92 94

3 2

2 2

3 2

23 22

6 4

63 66

8 7
24 NAEP CIVICS ASSESSMENTS–1988 AND 1998 • CHAPTER 1

Summary

This chapter presented the overall performance

of students at grades 4, 8, and 12, and the

performance of students by gender and by race/

ethnicity. Sample questions from each of the

four content areas defined by the 1988 Civics

Objectives were presented.

 Overall, there was a small increase in the

performance of fourth-graders between

1988 and 1998, a small decrease in the

performance of eighth-graders, and no

significant difference in the performance of

twelfth-graders.

 Fourth-grade males had a higher percentage

of correct responses in 1998 than in 1988,

while twelfth-grade males had a lower

percentage correct in 1998 than in 1988.

 The percentage of correct responses

increased in 1998 for white students in

grade 4 and decreased for Hispanic students

in grade 12. White students consistently

achieved a higher percentage correct than

either black or Hispanic students in both

1988 and 1998 at all three grade levels.
NAEP CIVICS ASSESSMENTS–1988 AND 1998 • CHAPTER 2 25

CHAPTER 2

Trends in

Civics Topics Studied

1 For evidence specifically about civics education, see Niemi, R.G.,

& Junn, J. (1998). Civic education: What makes students learn.

New Haven: Yale University Press.

contexts for learning. Students sampled for both

TOPICS STUDIED

the 1988 and 1998 assessments responded to a

student questionnaire in addition to answering

the civics content questions. Using the same

questionnaire makes possible comparisons across

the 10-year time span.

Trends in the Frequency of

Social Studies/Civics Classes

and Years of Study

Table 2.1 displays the frequency of social studies

classes reported by fourth-grade students in

both 1988 and 1998. Almost half of the fourthgraders in 1988 reported having social studies

instruction every day, compared to 39 percent

Overview

tudent performance in most academic

subject areas is generally associated with the

amount of time spent on the topic.1

 Has the

amount of time students indicated spending on

various civics topics changed over 10 years? Has

the amount of coursework taken in civics topics

changed for high school students? Are there

differences by grade or type of school? Information

bearing on these questions is presented in this

chapter. The chapter that follows examines trends

in instructional practices and out-of-school

S

Total 49 > 39 23 < 30 15 18 5 5 9 8

Public 47 37 23 30 15 19 5 6 10 8

Nonpublic 61 54 21 29 11 13 6 2 0 2

>1988 significantly greater than 1998.

<1988 significantly less than 1998.

NOTE: Percentages may not add to 100 due to rounding.

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), Civics Assessments–1988 and 1998.

Percentage of students reporting the frequency of social studies class

by type of school, grade 4: 1988 and 1998

Table 2.1

How often do

you usually have

social studies

class in school? 1988 1998 1988 1998 1988 1998 1988 1998 1988 1998

Percentage of students

3–4 times 1–2 times Less than Never/hardly

Daily a week a week once a week ever
26 NAEP CIVICS ASSESSMENTS–1988 AND 1998 • CHAPTER 2

of the fourth-graders in 1998. Thirty percent

of the fourth-graders in 1998 indicated having

social studies three to four times a week,

compared to 23 percent in 1988. This difference

indicates a trend over the decade toward less

frequent social studies instruction at grade 4.

Note that despite this trend, the results in

Chapter 1 indicated that fourth-graders in 1998

had higher percentages of correct responses on

the assessment than did fourth-graders in 1988.

There was little difference in the

percentages of fourth-grade students in 1988

and 1998 who indicated that they had social

studies less than once a week (5 percent) or

never/hardly ever (9 and 8 percent in 1988 and

1998, respectively). Daily instruction in social

studies was indicated by higher percentages of

students attending nonpublic schools than

public schools in both 1988 and 1998.

2 For high school only, data on the amount of coursework in civics

and American government can be found in Legum, S., Caldwell,

N., Davis, B., Haynes, J., Hill, T.J., Litavecz, S., Rizzo, L.,

Rust, K., & Vo., N. (1998). The 1994 high school transcript study

tabulations: Comparative data on credits earned and demographics

for 1994, 1990, 1987, and 1982 high school graduates —

REVISED (NCES 98–532). U.S. Department of Education.

Washington, DC: U.S. Government Printing Office.

For grades 8 and 12, students were asked

how frequently they studied American

government or civics in terms of the number

of previous grades in which they had been

instructed in the subject.2

 Table 2.2 presents

the results of these survey questions for grades

8 and 12. For the students in grade 8, about

the same percentage (approximately 80

percent) were taking civics in grade 8 in 1988

as in 1998. Some differences emerged,

however, for the eighth-graders in terms of

previous years in which they took civics. Higher

percentages of eighth-graders in 1998 reported

>1988 significantly greater than 1998.

<1988 significantly less than 1998.

NOTE: Percentages may not add to 100 due to rounding.

SOURCE: National Center for Education Statistics, National Assessment of Educational

Progress (NAEP), Civics Assessments–1988 and 1998.

Grade 8

Yes No Don’t know

1988 1998 1988 1998 1988 1998

Grade 12

Table 2.2

Percentage of students reporting they studied American government or civics

by grade levels, grades 8 and 12: 1988 and 1998

Studied American

government or civics in

the following grades:

5th grade 27 < 35 51 > 39 22 < 26

6th grade 35 < 42 46 > 38 18 20

7th grade 54 < 60 35 > 27 12 13

8th grade 78 81 12 9 10 10

9th grade 44 36 50 57 6 7

10th grade 31 35 64 59 5 6

11th grade 51 50 45 45 4 5

12th grade 61 < 71 35 > 25 4 3
NAEP CIVICS ASSESSMENTS–1988 AND 1998 • CHAPTER 2 27

having studied civics or American government

in grades 5, 6, and 7 than did their counterparts

in 1988. In general, the percentages of students

who answered “yes” to having studied civics

increased at each grade, from 35 percent in the

fifth grade to 60 percent in the seventh grade

within the 1998 group.

With respect to grade 12, Table 2.2

indicates that a higher percentage (71 percent)

of twelfth-graders in 1998 said they were

studying American government or civics in their

current grade than did their 1988 counterparts

(61 percent). However, the frequency of

studying civics topics in grades 9, 10, and 11

did not differ between the 1988 and 1998

groups. For both groups, for example, about

half of the students reported studying civics in

their junior year of high school, and about onethird reported studying civics in their sophomore

year. The assessment results in Chapter 1

indicated no significant performance difference

between twelfth-graders in 1988 and 1998.

The larger numbers of students reporting

that they had studied civics or American

government in a number of grades is consistent

with transcript data from a study3

 that showed

that an increased number of high school

graduates had taken civics coursework. Although

large numbers of grade school and high school

students do volunteer community service,

various individual authors and national

commissions have argued that Americans,

especially youths, have become disengaged from

public life. More and improved civic education is

often cited as one of the appropriate responses.4

Trends in Amount of Study of

Selected Topics in American

Government or Civics

Students in grades 4, 8, and 12 were asked to

indicate whether they studied a series of civics

topics “A lot,” “Some,” or “Not at all.” The

topics listed in the survey differed for fourth-,

eighth-, and twelfth-graders. The topics for

grade 4 are presented in Table 2.3, along with

the percentages of students from 1988 and 1998

who selected each option for amount studied.

In grade 4, none of the six topics was

studied “A lot” by a majority of students.

Almost half of the students in 1988 and 1998

studied their community “A lot,” and about a

third of the students in both cohorts indicated

studying “A lot” about the President/leaders

of the country and about the rights and

responsibilities of citizens. Judges and the

courts were studied least.

Comparing fourth-graders’ responses in

1988 and 1998, a significant increase in the

amount of study devoted to several topics in

civics is evident. A greater percentage of

students in 1998 reported studying three topics

“A lot”: judges and courts, the President and

leaders of our country, and elections and

voting. In addition, a smaller percentage of

students in 1998 than in 1988 indicated that

they did not study the rights and responsibilities

of citizens. The finding that fourth-graders were

devoting more time to studying these topics in

1998 than in 1988 may be seen as congruent

with fourth-graders’ better performance overall

on the civics assessment in 1998.

3 Ibid. 4 National Commission on Civic Renewal. A nation of spectators:

how civic disengagement weakens America and what we can do

about it. College Park, MD: Author.
28 NAEP CIVICS ASSESSMENTS–1988 AND 1998 • CHAPTER 2

Topics studied by students in grade 8 are

presented in Table 2.4 and those for students in

grade 12 are presented in Table 2.5. Students in

grade 12 were asked to include topics studied

since the ninth grade.

Of the 10 topics offered to eighth-graders,

shown in Table 2.4, the U.S. Constitution and

Bill of Rights were studied “A lot” by a majority

(56 percent) in both the 1988 and 1998 groups.

Three topics were studied “A lot” by 40 to

49 percent of eighth-graders in both assessment

years: rights and responsibilities of citizens;

political parties, elections and voting; and

Congress. The topic students indicated they

studied least for both 1988 and 1998 was

principles of democratic government.

The student data for grade 8 displayed

striking consistency in the amount of study each

topic received in 1988 compared with 1998.

Across the 10 curriculum topics, only one in the

30 comparisons in Table 2.4 showed any

difference across years.

Table 2.5 displays the percentage data for

topics studied by students in grade 12. The

pattern of topics studied is similar to that of

grade 8. The U.S. Constitution and Bill of

Rights is the single topic studied “A lot” by the

majority of students in both 1988 and 1998.

Topics studied “A lot” by 40 to 47 percent

of twelfth-graders included: Rights and responsibilities of citizens, political parties, elections,

and voting; how laws are made; and Congress.

The percentages of students in grade 12 who

selected the “Not at all” option were very low

(10 percent or lower) across all topics.

Similar to the patterns in the grade 8 data,

there is no difference noted for grade 12

between the 1988 and 1998 students in terms

of amount of time studying any of the 10 civics

topics in the table.

How laws are made 21 24 52 54 27 23

About judges and courts 7 < 13 33 < 38 60 > 49

President and leaders of our country 33 < 37 48 49 19 > 14

Elections and voting 13 < 19 47 < 54 39 > 27

Your community 47 48 40 42 12 10

The rights and responsibilities of citizens 27 33 47 48 25 > 20

>1988 significantly greater than 1998.

<1988 significantly less than 1998.

NOTE: Percentages may not add to 100 due to rounding.

SOURCE: National Center for Education Statistics, National Assessment of Educational

Progress (NAEP), Civics Assessments–1988 and 1998.

Percentage of students

A lot Some Not at all

1988 1998 1988 1998 1988 1998

Table 2.3

Percentage of students who reported how much they studied selected civics topics,

grade 4: 1988 and 1998

How much have you

studied the following

topics in American

government or civics?
NAEP CIVICS ASSESSMENTS–1988 AND 1998 • CHAPTER 2 29

NOTE: Percentages may not add to 100 due to rounding.

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress

(NAEP), Civics Assessments–1988 and 1998.

> 1988 significantly greater than 1998.

NOTE: Percentages may not add to 100 due to rounding.

SOURCE: National Center for Education Statistics, National Assessment of Educational

Progress (NAEP), Civics Assessments–1988 and 1998.

Table 2.4

Percentage of students who reported how much they studied selected civics topics,

grade 8: 1988 and 1998

How much have you studied

the following topics in American

government or civics?

U.S. Constitution and Bill of Rights 56 56 39 40 5 5

Congress 41 42 50 50 9 8

How laws are made 36 40 53 50 11 10

The court system 28 29 51 54 21 17

President and the cabinet 37 35 49 50 15 15

Political parties, elections, and voting 42 46 48 45 10 9

State and local government 30 35 54 52 16 13

Principles of democratic government 21 19 50 54 30 27

Other forms of government 21 24 60 60 19 17

Rights and responsibilities of citizens 45 48 42 42 13 > 10

Percentage of students

A lot Some Not at all

1988 1998 1988 1998 1988 1998

Since the beginning of ninth grade,

how much have you studied

the following topics in American

government or civics?

U.S. Constitution and Bill of Rights 52 53 45 45 3 3

Congress 43 46 53 51 4 4

How laws are made 40 43 54 52 6 5

The court system 36 39 56 56 8 5

President and the cabinet 39 42 56 53 6 5

Political parties, elections, and voting 44 46 51 50 5 4

State and local government 35 39 58 55 7 6

Principles of democratic government 31 35 59 57 10 8

Other forms of government 26 30 65 64 9 7

Rights and responsibilities of citizens 45 47 50 48 5 5

Percentage of students

A lot Some Not at all

1988 1998 1988 1998 1988 1998

Table 2.5

Percentage of students who reported how much they studied selected civics topics,

grade 12: 1988 and 1998
30 NAEP CIVICS ASSESSMENTS–1988 AND 1998 • CHAPTER 2

Thus there was little change between 1988

and 1998 in the amount of time spent studying

the various civics topics at either grade 8 or

grade 12. In addition, the same topics are

studied in almost the same amounts, although

coverage is more complete by grade 12 (note

the generally low percentages in the “not at

all” column in Table 2.5). This pattern was

evident in the 1988 national civics assessment

and the cautionary note sounded in the report

card published in 1990 bears repeating:

...it is important for educators to examine

what is being taught and how it is being

taught. Repetitious instruction from one

grade to the next may reduce the benefits of

civics instruction—or, even worse, diminish

students’ interest in the subject. As an

alternative, instructional approaches that

build on students’ existing knowledge and

understandings from grade to grade, and

that incorporate increasingly challenging

analytic tasks, are more likely to sustain

students’ interest and maximize what they

have studied in previous grades.5

The framework specifications for the 1998

NAEP civics assessment also recognized this

point by suggesting changes in the distribution

of assessment time by grade. In addition, while

it is difficult to make precise comparisons

between the 1988 and 1998 main civics

assessments because they were organized

differently, the 1998 assessment had a

considerably different topical emphasis. Most

readily apparent is the emphasis in 1998 on

the relationship of the United States to other

nations and to world affairs.6

5 Anderson, L., Jenkins, L.B., Lerning, J., MacDonald, W.B.,

Mullis, I.V.S., Turner, M.J. & Wooster, J.S. (1990). The civics

report card—trends in achievement from 1976 to 1988 at ages 13

and 17—achievement in 1988 at grades 4, 8, and 12. Washington,

DC: National Center for Education Statistics.

6 In 1998, this area was intended to be 10, 14, and 20 percent of

the assessment time at grades 4, 8, and 12 respectively. In 1988,

there was little emphasis on these topics.

Trends in the Amount of

Civics Coursework Completed

Twelfth-grade students were asked to indicate

the total number of years of coursework they

had completed in American government or

civics since ninth grade. The results appear in

Table 2.6. Sixty-two percent of twelfth-graders

in 1988 reported having taken one or more

years of American government or civics in high

school. A similar percentage (60 percent) of

twelfth-graders in 1998 had taken one or more

years of the subject. The course-taking pattern

is basically the same for 1998 as it was in 1988

with one exception: a lower percentage of

students in 1998 (5 percent) reported having

taken no civics or American government

compared to the 1988 figure of 8 percent.

>1988 significantly greater than 1998.

NOTE: Percentages may not add to 100 due to rounding.

SOURCE: National Center for Education Statistics, National Assessment

of Educational Progress (NAEP), Civics Assessments–1988 and 1998.

Percentage of students

1988 1998

None 8> 5

Less than one-half year 6 5

One-half year 16 19

Between one-half and one year 9 11

One year 21 24

More than one year 41 36

Percentage of students by number of years

of civics studied, grade 12: 1988 and 1998

Table 2.6

Since the beginning of ninth grade,

how much American government

or civics coursework have you

completed up to now?
NAEP CIVICS ASSESSMENTS–1988 AND 1998 • CHAPTER 2 31

Summary

This chapter focused on civics topics studied

and courses taken, comparing the populations

assessed in 1988 with those assessed in 1998.

The main findings included:

 A trend was noted toward less frequent

social studies classes in grade 4, with 49

percent of students in 1988 reporting daily

classes compared to 39 percent in 1998.

Despite this trend, the 1998 grade 4

students outperformed the 1988 grade 4

students on the assessment.

 Higher percentages of eighth-graders in

1998 reported having studied civics or

American government in grades 5, 6, and 7

than did their counterparts in 1988.

 A higher percentage (71 percent) of twelfthgraders in 1998 said they were studying

American government or civics in their

current grade than did their counterparts

(61 percent) in 1988. About half of the

students in both 1988 and 1998 reported

studying civics or American government in

their junior year of high school, and about a

third reported studying these fields in their

sophomore year.

 In three of the six civics curriculum topics

surveyed (elections and voting, President/

leaders of the country, and judges and

courts), the fourth-graders in 1998 reported

studying these topics more than their

counterparts did in 1988.

 For both eighth- and twelfth-graders, the

U.S. Constitution and Bill of Rights is the

single topic studied “A lot” by a majority of

students in both 1988 and 1998.

 The amount of time spent studying the

various civics topics surveyed at both grades

8 and 12 is similar and has not changed

between 1988 and 1998.

NAEP CIVICS ASSESSMENTS–1988 AND 1998 • CHAPTER 3 33

CHAPTER 3

Instructional Practices and

Out-of-School Contexts

1 For a study that does relate instructional practices and the home

environment to student performance on the 1988 NAEP civics

assessment, see Niemi, R.G. and Junn, J. (1998). Civic

education: What makes students learn? New Haven: Yale

University Press.

Civics Instructional Practices

LEARNING CONTEXTS

One approach to improving achievement scores

across all curricular areas focuses on instructional

methods. Whatever the content, it is argued,

learning will be enhanced if certain kinds of

instructional approaches are used in preference to

others. Recently, more active and “authentic”

approaches have been emphasized in preference to

passive methods.2

 Cooperative learning methods,

which involve teachers working with small groups

of students and students working with each other

in groups, have frequently been promoted.3

Higher order thinking skills, which include

analyzing and organizing material and presenting

it both orally and in writing, have been advocated

for use in the classroom and for homework.4

Instructional practices have sometimes been

classified along four dimensions: individual

versus group instruction; kinds of materials used;

activities undertaken; and methods of

assessment.5

 All four of these dimensions were

addressed by background questions in the NAEP

Overview

tudent learning in civics, as in other subjects,

is influenced by many factors both in and out

of school. Within school, instructional practices

directly affect student interaction with civics

content. Beyond the classroom, students’ home

environment and extracurricular activities can

support or detract from civics learning. Bridging

the two environments, especially at the lower

grades, is homework. Assigning homework is an

instructional practice used by many teachers to

enhance learning, yet the effective completion

of homework assignments often can rest most

heavily upon the home environment.

This chapter examines the results of a

selection of student background questions

addressing instructional practices and out-ofschool contexts for learning civics. Students

who participated in the 1988 and 1998

assessments answered the same set of

background questions. As in the previous

chapter, the results reported here compare

student responses to the questions in 1988 with

student responses in 1998, but they do not

include associated performance data.1

Nevertheless, the information should be of use

to those interested in whether and how civic

education has changed over the span of a decade.

2 Newmann, F.M., and Wehlage, G.G. (1995). Successful school

restructuring: A report to the public and educators. Madison, WI:

Center on Organization and Restructuring of Schools.

3 Slavin, R.E. (1996). Research on cooperative learning and

achievement: What we know, what we need to know.

Contemporary Educational Psychology 21(1), 43–69.

4 Murnane, R.J., and Levy, F. (1996). Teaching the New Basic

Skills: Principles for Educating Children to Thrive in a Changing

Economy. New York: Free Press.

5 Henke, R.R., Chen, X., & Goldman, G. (1999). What happens

in classrooms: Instructional practices in elementary and secondary

schools: 1994–95. Education Statistics Quarterly 1(2), 7–13.

S
34 NAEP CIVICS ASSESSMENTS–1988 AND 1998 • CHAPTER 3

trend study. An additional question inquired about

discussing current events, a topic of special interest

for civics instruction.

Students in grades 8 and 12 were asked to

report how often their civics teachers engaged

them in 10 activities. The 10 activities plus the

results of students’ responses about the

frequency of each activity are presented in

Table 3.1 for grade 8 and Table 3.2 for grade

12. Fourth graders were not asked this series

of questions.

As seen in Tables 3.1 and 3.2, the frequency

with which students performed most of these

activities did not change significantly between

1988 and 1998. There were, however, a few

changes of note. Most notably, it would seem

that, at both grades, there was an increase in the

assignment of extra reading material and in

having students work on group projects.

Among eighth-graders, there was a rise from

1988 to 1998 in the frequency with which their

teachers assigned extra reading material. In

1998, they were more likely to assign extra

reading at least once or twice a week, and less

likely to assign it once or twice a month.

More eighth-grade students were working on group projects in 1998 than

in 1988.

SOURCE: National Center for Education Statistics, National Assessment

of Educational Progress (NAEP), Civics Assessments–1988 and 1998.

Percentage of students doing group projects

at least once a week

Figure 3.1

31%

1998

13%

1988

40

 30

 20

10

0

How often has your American

government or civics teacher asked

you to do each of the following?

Read material from textbook 57 52 35 37 5 7 1 2 2 2

Read extra material not in regular textbook 10 < 13 28 < 34 29 > 25 15 14 17 14

Memorize the material you have read 21 22 27 26 21 24 14 13 17 15

Discuss and analyze the

material you have read 58 53 28 30 7 < 10 3 3 5 4

Write short answers to questions 29 31 43 43 17 17 5 4 6 5

Write a report of three or more pages 3 4 7 8 31 32 35 36 25 20

Work on a group project 3 < 8 10 < 23 28 < 37 28 > 21 31 > 11

Give talks about what you are studying 44 45 20 24 14 14 10 8 12 9

Discuss current events 30 29 39 37 18 20 6 7 8 6

Take a test or quiz 13 12 56 > 49 28 < 37 1 1 1 1

Table 3.1

> 1988 significantly greater than 1998.

< 1988 significantly less than 1998.

SOURCE: National Center for Education Statistics, National Assessment of Educational

Progress (NAEP), Civics Assessments–1988 and 1998.

Percentage of students

Almost Once or Once or A few

every day twice a week twice a month times a year Never

1988 1998 1988 1998 1988 1998 1988 1998 1988 1998

Percentage of students reporting how often their teachers use various classroom

practices, grade 8: 1988 and 1998
NAEP CIVICS ASSESSMENTS–1988 AND 1998 • CHAPTER 3 35

The shift in the popularity of group projects

was even more pronounced. In 1988, 13 percent

of students were assigned to work on group

projects at least once a week and less than onehalf of students were assigned to work on group

projects at least once or twice a month. In 1998,

nearly one-third of students worked on group

projects at least once a week and more than twothirds worked on them at least once a month.

Finally, eighth-graders were given tests or

quizzes less often in 1998 than in 1988. The

percentage of students whose teachers gave them

quizzes once or twice a week declined from 56

percent to 49 percent, while the percentage

getting tests or quizzes once or twice a month

increased from 28 percent to 37 percent.

At the twelfth-grade level, nearly twice as

many students in 1998 than in 1988 reported

being assigned extra reading material almost

every day. The percentage having extra reading

only once or twice a month or a few times a year

decreased from 43 percent to 31 percent.

As at the eighth-grade level, the percentage

of twelfth-graders reporting having group project

40

 30

 20

10

0

Percentage of students reading extra material

Nearly twice as many twelfth-grade students in 1998 as in 1988 reported

being assigned extra reading material almost every day.

SOURCE: National Center for Education Statistics, National Assessment

of Educational Progress (NAEP), Civics Assessments–1988 and 1998.

Figure 3.2

11%

1988

21%

1998

How often has your American

government or civics teacher asked

you to do each of the following?

Read material from textbook 49 48 39 37 7 8 2 3 3 5

Read extra material not in regular textbook 11 < 21 33 37 27 > 21 16 > 10 13 11

Memorize the material you have read 17 20 29 31 22 22 14 12 18 16

Discuss and analyze the

material you have read 57 60 28 26 9 8 3 3 3 3

Write short answers to questions 23 27 45 43 20 20 6 5 6 6

Write a report of three or more pages 4 4 7 10 30 33 36 > 30 23 22

Work on a group project 5 < 11 13 < 25 31 30 28 > 19 23 > 15

Give talks about what you are studying 33 < 41 18 21 17 16 14 > 11 18 > 11

Discuss current events 43 < 54 37 > 28 12 11 5 3 3 3

Take a test or quiz 13 14 66 60 19 24 1 2 1 1

Percentage of students reporting how often their teachers use various classroom

practices, grade 12: 1988 and 1998

Table 3.2

Percentage of students

Almost Once or Once or A few

every day twice a week twice a month times a year Never

1988 1998 1988 1998 1988 1998 1988 1998 1988 1998

> 1988 significantly greater than 1998.

< 1988 significantly less than 1998.

SOURCE: National Center for Education Statistics, National Assessment of Educational

Progress (NAEP), Civics Assessments–1988 and 1998.
36 NAEP CIVICS ASSESSMENTS–1988 AND 1998 • CHAPTER 3

6 Black, S. (1996). The truth about homework. American School

Board Journal, 183(10), 48–51.

assignments almost every day or once a week

rose, while the percentage fell among those

never doing group projects or doing them a few

times a year. The percentage of twelfth-graders

having such assignments almost every day

increased from 5 to 11, while the percentage

having such assignments once or twice a week

rose from 13 to 25. Meanwhile, the percentage

never doing group projects or doing them a few

times a year fell from 51 to 34. Two other shifts

in twelfth-grade instructional practice are also

worth noting. Students reported that their

teachers were more likely to have them talk

about what they were studying and to discuss

current events in 1998 than in 1988.

Fourth-grade students were not asked about

all the classroom practices shown in Tables 3.1

and 3.2. However, they were asked how often

they discussed current events in their social studies

class. The results, presented in Table 3.3, show

an increase in the frequency of such discussions.

Students reported they were more likely to

discuss current events once or twice a week in

1998 than in 1988 (39 vs. 29 percent), and were

less likely to discuss them only a few times a

year (8 vs. 12 percent).

Out-of-School Contexts

This section will focus on two variables of interest

to civics educators: civics homework trends and

the availability of newspapers in the home.

Although homework is an integral part of

instruction in most schools, its contribution to

student learning depends on many factors,

including the quality and appropriateness of the

assignments, the amount of homework assigned,

and the attention given to completed assignments

by teachers. Studies indicate that, in general,

increased time spent on homework is associated

with higher achievement among high school

students, with somewhat higher achievement

among middle school students, and with no

difference in achievement among grade school

students. Specifically, assignments of more than

one hour of homework per day have been seen to

raise achievement among high school students,

but not among elementary or junior high school

students.6

Table 3.3

Almost Once or Once or A few

every day twice a week twice a month times a year Never

1988 1998 1988 1998 1988 1998 1988 1998 1988 1998

22 26 29 < 39 17 15 12 > 8 20 > 13

Percentage of students reporting how often they discuss current events,

grade 4: 1988 and 1998

> 1988 significantly greater than 1998.

< 1988 significantly less than 1998.

SOURCE: National Center for Education Statistics, National Assessment of Educational

Progress (NAEP), Civics Assessments–1988 and 1998.

Percentage of students How often do you

discuss current events

in social studies class?
NAEP CIVICS ASSESSMENTS–1988 AND 1998 • CHAPTER 3 37

Grade 12 students were asked how much time

they spent on civics or government homework

each week. These results are presented in Table 3.4.

According to the reports of twelfth-graders,

there were no significant changes in the amount

of time spent on civics homework between

1988 and 1998.

The availability of reading materials in the

home is related to student literacy and may be

related to academic achievement generally.7

Students participating in the civics assessment at

grades 4, 8, and 12 were asked about the types

of reading materials available in the home. Of

particular interest to the civics and current events

discussions in the classroom are the results of the

question: Does your family get a newspaper

regularly? The percentages of students

responding “yes,” “no,” and “I don’t know”

to this question are shown in Table 3.5.

The data show a decline in the percentage

of families of eighth-and twelfth-graders receiving

newspapers. Between 1988 and 1998, the

percentage of eighth-graders reporting getting a

newspaper fell from 77 percent to 71 percent,

and the percentage of twelfth-graders fell from

82 to 75.8

8 This parallels a national decline in newspaper circulation that has

been underway since the late 1940s, and continued unabated

between 1988 and 1998. See Stanley, H.W. and Niemi, R.G.

(2000). Vital statistics on American politics 1999–2000.

Washington, DC: CQ Press.

7 Greenwald, E.A., Persky, H.R., Campbell, J.R., & Mazzeo, J.

(1999). The NAEP 1998 writing report card for the nation and

the states. Washington, DC: National Center for Education

Statistics.

Grade 4 69 68 24 24 7 9

Grade 8 77 > 71 22 25 2 < 5

Grade 12 82 > 75 17 < 24 1 1

> 1988 significantly greater than 1998.

< 1988 significantly less than 1998.

SOURCE: National Center for Education Statistics,

National Assessment of Educational Progress (NAEP),

Civics Assessments–1988 and 1998.

Table 3.5

Percentage of students reporting receiving

newspapers in the home, grades 4, 8, and 12:

1988 and 1998

Percentage of students

Yes No I don’t know.

1988 1998 1988 1998 1988 1998

Does your

family get a

newspaper

regularly?

Table 3.4

Percentage of students reporting the amount of time they spend

weekly on civics or government homework, grade 12: 1988 and 1998

How much time do you

spend weekly on civics or

government homework?

Percentage of students

Don’t do None, < 1 hour 1–2 hours 3–4 hours 5+ hours

1988 1998 1988 1998 1988 1998 1988 1998 1988 1998

5 5 43 50 36 33 11 9 3 4

NOTE: Some of the data in this table are the result of collapsing the original response

options in the question. The question contained the following choices: I usually haven’t

had homework assigned; I have had homework but I usually haven’t done it; less than 1

hour; 1 hour; 2 hours; 3 hours; 4 hours; 5 hours or more.

SOURCE: National Center for Education Statistics, National Assessment of Educational

Progress (NAEP), Civics Assessments–1988 and 1998.
38 NAEP CIVICS ASSESSMENTS–1988 AND 1998 • CHAPTER 3

Summary

This chapter contains data from student

background questionnaires covering classroom

practices and home contexts related to civics

learning. The variables discussed might also be

related to student performance in civics, but

due to the narrowed range of test questions

used in 1998 compared to 1988, performance

data comparing 1988 with 1998 have not been

included. Highlights of the findings include

the following:

 Extra Reading Material: The frequency

with which students at grades 8 and 12

reported being assigned extra reading

material increased between 1988 and 1998.

 Group Projects: The percentage of eighthand twelfth-graders who reported being

assigned to work on group projects at least

once or twice a week rose substantially

between 1988 and 1998.

 Discussing Current Events: The

percentage of fourth-graders who indicated

discussing current events in their social

studies class at least once a week increased.

 Homework: The amount of civics homework twelfth-graders reported doing did not

change significantly over the period.

 Newspapers in the Home: Between 1988

and 1998, the percentage of eighth-graders

and twelfth-graders who reported their

families regularly got a newspaper fell.
NAEP CIVICS ASSESSMENTS–1988 AND 1998 • APPENDIX A 39

APPENDIX A

PROCEDURES

Overview of Procedures

Used in the 1988 and 1998

NAEP Civics Special Study

his appendix provides information about

the methods and procedures used in

NAEP’s 1988–1998 civics assessment.

This NAEP report is based on results from

nine civics exercise blocks, three each at grades

4, 8, and 12. These blocks of questions came

from the 1988 civics assessment and were based

on the framework used to develop that

assessment. They were readministered to a

subsample of students at each grade in 1998,

making possible a 10-year comparison of

performance on certain of the 1988 items.

The 1988 tasks required students to read

and answer multiple-choice questions based on

a variety of materials. The assessment was

designed to evaluate students’ ability to locate

specific information, make inferences based on

an information passage or graphical stimulus

(e.g., a political cartoon), or perform more

analytical or evaluative tasks such as distinguishing opinion from fact or defending a position.

The assessment administered at grade 4

included 51 questions, at grade 8, 77

questions, and at grade 12, 74 questions. The

number of unique questions administered is less

than the sum from the three grades because

some questions appeared at more than one

grade. Specifically, 16 questions were shared by

the grade 4 and grade 8 assessments, and 27

questions were shared by the grade 8 and grade

12 assessments. This overlap is a function of the

design of the 1988 assessment in which a

number of cross-grade blocks were developed.

 The 1988 civics assessment was

administered in two waves, winter and spring, in

order to provide data to bridge both backward

and forward in time to the new assessment

schedule. Because the 1998 assessment was

offered only in the winter, only half of the 1988

data (the winter portion) could be used for comparison with the 1998 results. This led to smaller

than optimal samples for the comparisons.

The Design of the 1988 and

1998 Civics Special Study

To obtain the 1998 data for the special study,

NAEP re-administered three test booklets—

one at each grade—that had been given as part

of the 1988 civics assessment. Following

normal NAEP practices, the 1988 test booklets

had been assembled using blocks of questions.

In 1988, three 10-minute blocks of questions

were developed for grade 4. These three blocks

were used to construct a single test booklet,

and it was this booklet that was used again in

1998. Seven 15-minute blocks of questions

were developed for each of grades 8 and 12.

The blocks, in various combinations, were used

to assemble seven grade 8 test booklets and

seven grade 12 test booklets. Each booklet

contained three blocks of questions for a total

T
40 NAEP CIVICS ASSESSMENTS–1988 AND 1998 • APPENDIX A

of 45 minutes of testing time. With the advice

of the 1998 civics standing committee

(Appendix D), one of the grade 8 booklets and

one of the grade 12 booklets were selected for

use in the study. Thus, although the special

study blocks had appeared in different positions

in different 1988 test booklets, they appeared

in only one position in 1998 because of the use

of a single test booklet. Analysis of 1988 blockposition effects in grades 8 and 12 suggested

that combining blocks that had originally

appeared in different positions in different

booklets would not distort the 1988–1998

civics comparisons. Another administration

difference between the two assessments could

have affected results as well. The booklets at all

three grades also included one block of general

background questions and one block of

background questions related to social studies

(grades 4 and 8) or civics (grade 12). For the

special study, the test booklets were administered

intact for fourth graders, that is, exactly as they

appeared in 1988.1

Sampling and Data Collection

Sampling and data collection activities for the

1998 civics assessment were conducted by

Westat, Inc. The data collection for all three

grades was carried out in the winter and spring

of 1998 (January–March). The data collection

activities for the 1988 assessment occurred

during the winter and spring (January 4–May 13)

of 1988. However, only the winter (January 4–

March 11) 1988 data were used in the special

study.

As with all NAEP national assessments, the

1988 and 1998 samples included both public

and nonpublic school students. The sample was

selected using a stratified, three-stage sampling

plan. The first stage included defining geographic

primary sampling units (PSUs), which are

typically groups of contiguous counties, but

sometimes a single county; classifying the PSUs

into strata defined by region and community

type; and then selecting PSUs with probability

proportional to size. In the second stage, both

public and nonpublic schools were selected within

each PSU that was selected at the first stage.

The third stage involved randomly selecting

students within a school for participation. A small

number of students selected for participation

was excluded by school personnel because of

limited English proficiency or severe disability.

The student sample sizes for the civics special

study are presented in Table A.1. The

differences in the sample sizes in 1988 are

accounted for by the fact that one test booklet

was used in grade 4 in 1988, whereas grade 8

and 12 had seven booklets and the blocks used

in the special study appeared in six of them (not

necessarily together). Therefore the total

numbers of students are greater in grades 8 and

12 than in grade 4.

1 For additional information on the design of the 1988 civics

assessment, see Johnson, E.G., Zwick, R., et.al. (1990).

Focusing the new design: The NAEP 1988 technical report.

Princeton, NJ: Educational Testing Service, National Assessment

of Educational Progress.

NAEP civics special study sample sizes,

grades 4, 8, and 12: 1988 and 1998

Grade 4 Grade 8 Grade 12

SOURCE: National Center for Education Statistics, National Assessment

of Educational Progress (NAEP), Civics Assessments–1988 and 1998.

Table A.1

Trend assessment

1998 assessed

sample size 2,088 2,055 2,193

Excluded 176 93 103

1988 assessed

sample size 1,059 2,416 2,465

Excluded 71 135 95
NAEP CIVICS ASSESSMENTS–1988 AND 1998 • APPENDIX A 41

Participation Rates

Student and school participation rates are given

in Table A.2. Sampled schools that refused to

participate were substituted with similar schools,

usually from the same PSU. School cooperation

rates are shown both before and after replacement. The student participation rates represent

the percentage of students assessed of those

invited to be assessed and includes those assessed

in follow-up sessions when necessary. In NAEP,

there has been a consistent pattern of lower

participation rates for older students. The participation rate in the 1998 special study for grade

12 students was 82 percent, compared with 95

percent for grade 4 students and 92 percent for

grade 8 students. School participation rates (the

percentage of sampled schools that participated in the assessment) have also typically been

lowest at grade 12. As is shown in Table A.2,

the school participation rate was 90 percent for

the fourth grade, 91 percent for the eighth

grade, and 83 percent for the twelfth grade.

The overall response rate before substitution

(the product of the weighted school participation rate before substitution and the weighted

student participation rate) for grade 12 fell below

the NCES reporting target of 70 percent. As a

result, the background characteristics of both

responding schools and all schools in the

writing, national civics, and civics trend target

set were compared to determine whether there

was bias evident. The similarities in the

distribution lend support to the conclusion that

the data are not seriously biased by these low

response rates.

The effect of participation rates on student

performance, however, is unclear. Students may

choose not to participate in NAEP for many

reasons, such as a desire to attend regular classes

so as not to miss important instruction, or fear of

not doing well on NAEP. Similarly, there are a

variety of reasons for which various schools do

not participate. The sampling weights and

nonresponse adjustments, described later in this

appendix in the section “Weighting and Variance

Estimation,” provide an approximate statistical

adjustment for nonparticipation. However, the

effect of some school and student nonparticipation

may have an undetermined effect on results.

Students with Disabilities (SD)

and Limited English

Proficient (LEP) Students

It is NAEP’s intent to assess all selected students

from the target population. Therefore, every

effort is made to ensure that all selected

students who are capable of participating in the

assessment are assessed. Some students sampled

for participation in NAEP can be excluded from

Weighted participation rates (percentages) by

grade and subject type, 1998 NAEP civics special

study reporting samples

Table A.2

Participation Percentage

Grade 4

School participation before substitution 81

School participation after substitution 90

Student participation 95

Overall participation rate before substitution 77

Overall participation rate after substitution 86

Grade 8

School participation before substitution 77

School participation after substitution 91

Student participation 92

Overall participation rate before substitution 71

Overall participation rate after substitution 84

Grade 12

School participation before substitution 68

School participation after substitution 83

Student participation 82

Overall participation rate before substitution 56

Overall participation rate after substitution 68

SOURCE: National Center for Education Statistics, National Assessment

of Educational Progress (NAEP), Civics Assessments–1988 and 1998.
42 NAEP CIVICS ASSESSMENTS–1988 AND 1998 • APPENDIX A

the sample according to carefully defined criteria.

These criteria were revised in 1996 to more

clearly communicate a presumption of inclusion

except under special circumstances. According to

these criteria, students with Individualized

Education Plans (IEPs) were to be included in

the 1998 NAEP national assessment except in

the following cases:

1. The school’s IEP team determined that the

student could not participate, OR

2. The student’s cognitive functioning was so

severely impaired that she or he could not

participate.

All LEP students receiving academic instruction

in English for three years or more were to be

included in the assessment. Those LEP students

receiving instruction in English for fewer than

three years were to be included unless school staff

judged them incapable of participating in the

assessment in English.

The reporting samples in the main 1998

national civics assessment used these criteria,

with provisions made for accommodations.2

Specifically, in cases where a student’s IEP

required that the student be tested with an

accommodation or adaptation and stated that

the student could not demonstrate his or her

knowledge without that accommodation, the

student was provided with the appropriate

accommodation. The 1998 special study samples

reported here used the same inclusion/exclusion

criteria as were used for the main assessment.

However, in order to keep the 1998 test

administration conditions consistent with those

of 1988, no accommodations or adaptations were

provided for the special study samples.

 The 1998 exclusion rates for the students

with disabilities (SD) and LEP samples in the

Table A.3

2 Lutkus, A.D., Weiss, A.R., Campbell, J.R., Mazzeo, J., &

Lazer, S. (1999). NAEP 1998 civics report card for the nation.

Washington, DC: National Center for Education Statistics.

Excluded 176 6.7 93 4.4 103 3.7

Excluded 123 5.3 72 3.6 91 3.5

Excluded 60 1.6 27 0.9 15 0.4

NOTE: The combined SD/LEP portion of the table is not a sum of the separate SD and LEP

portions because some students were identified as both SD and LEP. Such students would be

counted separately in the bottom portions, but counted only once in the top portion. Within

each portion of the table, percentages may not sum properly due to rounding.

Data for students with disabilities and limited English proficient students were not collected in 1988.

SOURCE: National Center for Education Statistics, National Assessment of Educational

Progress (NAEP), Civics Assessments–1988 and 1998.

Students who are:

Students with disabilities and limited English proficient students excluded

from the NAEP civics special study: National sample, public and nonpublic

schools combined: 1998

Grade 4 Grade 8 Grade 12

Weighted Weighted Weighted

percentage percentage percentage

Number of of students Number of of students Number of of students

students sampled students sampled students sampled

Limited English proficient students

Students with disabilities

SD and LEP students
NAEP CIVICS ASSESSMENTS–1988 AND 1998 • APPENDIX A 43

civics special study are presented in Table A.3

for all three grades. For calculation of the

percentages in Table A.3, the denominator used

was the total number of all students who were

identified for the assessment, including assessed

and excluded students. The columns then show

the raw numbers and weighted percentages of

SD and LEP students who were excluded from

the assessment. The percentages of assessed

students who were in the students with

disabilities and the limited English proficient

categories were not collected for the 1998

special study because the test administration

procedures used from the 1988 assessment did

not call for the collection of these data. However,

the civics special study was conducted in the

same schools as the 1998 civics main assessment.

Consequently, the identification rates and the

percentages of special needs students assessed

without accommodations are likely to be

comparable to those in Table A.3 of the 1998

Civics Report Card for the Nation.

Data Analysis

After the assessment information had been

compiled in the NAEP database, the data were

weighted according to the sample design and

the population structure. The weighting for the

samples reflected the probability of selection for

each student as a result of the sampling design,

adjusted for nonresponse. Through poststratification, the weighting assured that the

representation of certain subpopulations

corresponded to figures from the U.S. Census

and the Current Population Survey.

Analyses were then conducted to determine

the percentage of students who gave various

responses to each cognitive and background

question for the nation and various subgroups

of interest within the nation, and the mean

percentage correct score across all questions in

a grade. Subtotals were formed separately

within each grade level for each of the four

civics content areas.

Tables C.1, C.2, and C.3 give the percentage

correct for each question by content area in

1988 and 1998 (see Appendix C).

As is the case in most NAEP assessments,

the 1998 main NAEP civics assessment was

reported in terms of item-response theory (IRT)

scale scores. IRT scaling was not employed for

the civics special study, however, partly because

of the small sample sizes, but more centrally because the questions, at least in the upper

grades, were not completely representative of the

entire 1988 assessment specifications. The analysis

was motivated by a desire to avoid creating a

potentially misleading “second scale” for 1988.

The NAEP percentage correct scores for the

civics special study and IRT scale scores for most

of the remaining subject areas make it possible

to examine relationships between students’

performance and a variety of background factors

measured by NAEP. The fact that a relationship

exists between achievement and another variable,

however, does not reveal the underlying cause of

the relationship, which may be influenced by a

number of other variables. Similarly, the civics

assessment does not capture the influence of

unmeasured variables. The results are most useful

when they are considered in combination with

other information about the student population

and the educational system, such as changes in

instruction, changes in the school-age population,

and societal demands and expectations.

Adjustments to the Assessment

One test question each in grade 8 and grade 12

was administered but subsequently excluded

from the statistical analyses. In grade 8, a question

on the legality of impeachment was excluded

because of possible ambiguities in the wording.

In grade 12, a question on the “most recent

amendment” was excluded because an
44 NAEP CIVICS ASSESSMENTS–1988 AND 1998 • APPENDIX A

amendment passed in the interim invalidated

the answer that was correct in 1988.

The content classifications of all the 1988

test questions used in the special study were

reviewed for currency and accuracy. The

content classifications of some items were

changed from those used in the 1988 report

card. Comparisons of percentages correct within

a content category for 1988 vs 1998 in this

report consistently use the most recent

classification code.

NAEP Reporting Groups

This report contains results for the nation and

for groups of students within the nation defined

by shared characteristics. The subgroups

defined by race/ethnicity and gender are

defined below.

Race/Ethnicity

Results are presented for students in different

racial/ethnic groups according to the following

mutually exclusive categories: white, black,

Hispanic, Asian/Pacific Islander, and American

Indian (including Alaskan Native). Some

racial/ethnic results are not reported separately

because there were too few students in the

group. The data for all students, regardless of

whether their racial/ethnic group was reported

separately, were included in computing the

overall national results.

Two questions from the set of general

student background questions were used to

determine race/ethnicity:

If you are Hispanic, what is your Hispanic

background?

AI am not Hispanic.

BMexican, Mexican American, or Chicano

CPuerto Rican

DCuban

EOther Spanish or Hispanic background

Students who responded to this question by

filling in the second, third, fourth, or fifth oval

were considered Hispanic. For students who

filled in the first oval, did not respond to the

question, or provided information that was

illegible or could not be classified, responses to

the following question were examined to

determine their race/ethnicity:

Which best describes you?

AWhite (not Hispanic)

BBlack (not Hispanic)

CHispanic (“Hispanic” means someone who

is Mexican, Mexican American, Chicano,

Puerto Rican, Cuban, or from some other

Spanish or Hispanic background.)

DAsian or Pacific Islander (“Asian or Pacific

Islander” means someone who is from a

Chinese, Japanese, Korean, Filipino,

Vietnamese, Asian American, or some other

Asian or Pacific Island background.)

EAmerican Indian or Alaskan Native

(“American Indian or Alaskan Native”

means someone who is from one of the

American Indian tribes, or one of the original

people of Alaska.)

FOther (What?) Write your answer on the

line in your answer book.

Gender

Results are reported separately for males and

females. Gender information was taken from

school records.

Weighting and

Variance Estimation

A complex sample design was used to select the

students who were assessed. The properties of

a sample selected through a complex design

could be very different from those of a simple

random sample, in which every student in the

target population has an equal chance of selection, and in which the observations from different

sampled students can be considered to be statis-
NAEP CIVICS ASSESSMENTS–1988 AND 1998 • APPENDIX A 45

tically independent of one another. Standard

errors calculated as though the data had arisen

from a simple random sample would generally

underestimate sampling error. Therefore, the

properties of the sample for the complex data

collection design were taken into account

during the analysis of the assessment data.

One way that the properties of the sample

design were addressed was by using sampling

weights to account for the fact that the

probabilities of selection were not identical for

all students. All population and subpopulation

characteristics based on the assessment data used

sampling weights in their estimation. These

weights included adjustments for school and

student nonresponse.

The statistics presented in this report are

estimates of group and subgroup performance

based on samples of students, rather than the

values that could be calculated if every student

in the nation answered every assessment

question. It is therefore important to have

measures of the degree of uncertainty of the

estimates. Accordingly, in addition to providing

estimates of percentages of students and their

average percentage correct for test questions,

this report provides information about the

uncertainty of each statistic.

Because NAEP uses complex sampling

procedures, conventional formulas for estimating

sampling variability that assume simple random

sampling are inappropriate. For this reason,

NAEP uses a jackknife replication procedure to

estimate standard errors due to sampling from

means of paired primary sampling units (PSUs).

The reader is reminded that, like results

from all surveys, NAEP results are also subject

to other kinds of errors, including the effects of

necessarily imperfect adjustments for student

and school nonresponse and other largely

unknowable effects associated with the particular

instrumentation and data collection methods

used. Nonsampling errors can be attributed to a

number of sources: inability to obtain complete

information about all selected students in all

selected schools in the sample (some students or

schools refused to participate, or students

participated but answered only certain questions);

ambiguous definitions; differences in interpreting

questions; inability or unwillingness to give

correct information; mistakes in recording, coding,

or scoring data; and other errors of collecting,

processing, sampling, and estimating missing

data. The extent of nonsampling errors is difficult

to estimate. By their nature, the impact of such

errors cannot be reflected in the data-based

estimates of uncertainty provided in NAEP reports.

Drawing Inferences

from the Results

Confidence Intervals Around

Means and Percentages

The use of confidence intervals, based on the

standard errors, provides a way to make

inferences about the population averages and

percentages in a manner that reflects the

uncertainty associated with the sample estimates.

An estimated band of ±2 standard errors from a

sample represents about a 95 percent confidence

interval for the corresponding population

quantity. This means that with 95 percent

certainty, the average performance of the entire

population of interest is within about ±2

standard errors of the sample average.

As an example, suppose that the average

percent correct scores of students in a particular

group was 60, with a standard error of 2.0.

A 95 percent confidence interval for the

population quantity would be as follows:

Average ± 2 standard errors =

60 ± 2 (2) = 60 ± 4 =

60 – 4 and 60 + 4 = (56, 64)
46 NAEP CIVICS ASSESSMENTS–1988 AND 1998 • APPENDIX A

Thus, one can conclude with close to 95 percent

certainty that the average percent correct score

for the entire population of students in that

group is between 56 and 64.

Confidence intervals can be constructed for

percentages, provided that the percentages are

not extremely large or extremely small. For

percentages, confidence intervals constructed in

the manner above work best when sample sizes

are large, and the percentages being tested have

magnitude relatively close to 50 percent.

Statements about group differences should be

interpreted with caution if at least one of the

groups being compared is small in size and/or if

“extreme” percentages are being compared. By

convention, percentages, P, were treated as

“extreme” in NAEP if:

200 P<Plim = , where the effective sample size is

NEFF+2

N P(100 – P) EFF = , and SE

jk is the jackknife

 (SE

jk

)

2

standard error of P.

This “rule of thumb” cutoff leads to flagging

a large proportion of confidence intervals that

would otherwise include values less than 0 or

greater than 100. In either extreme case, the

confidence intervals described above are not

appropriate, and procedures for obtaining

accurate confidence intervals are quite

complicated. In this case, the value of P was

reported, but no standard error was estimated

and, hence, no tests were conducted.

For some of the subgroups of students for

which results were reported, student sample

sizes could be quite small. For results to be

reported for any subgroup, a minimum student

sample size of 62 was required. If students in a

particular subgroup were clustered within a

small number of geographic primary sampling

units (PSUs), the estimates of the standard

errors might also be inaccurate. So, subgroup

data were required to come from a minimum of

five PSUs, in order for results to be reported.

Comparing Means and

Percentages of Different Groups

How large does a numeric difference in NAEP

data have to be in order for a reader to conclude

that it is not merely chance occurrence?

Drawing appropriate and justifiable inferences

from the data gathered in the NAEP assessment

is a particularly complex statistical endeavor

because: 1) a sampling design is used for data

collection, and 2) no single student takes more

than a section of the entire question pool.

(Civics trend was an exception to this general

NAEP practice.) Judging the statistical significance

of differences between subgroups in the NAEP

data requires an understanding of the following:

confidence intervals (as noted above) and standard

errors in the sampling design; the application of

the t-test paradigm; the notion of degrees of

freedom as applied in a stratified sample; and

the application of the concepts of statistical

family size in the context of making multiple

comparisons. Each of these topics will be

touched upon in the following sections. Because

of these complexities, quick “rule of thumb”

estimates of the significance of differences

among, for example, percentage correct in

NAEP subgroups in the data tables in this

report cannot be relied upon to be accurate.

T-tests. Among the major findings reported for

NAEP assessments are mean differences between

groups, for example, comparisons of public and

private school students. Such comparisons are

assessed for statistical significance by a t-test of

the form:
NAEP CIVICS ASSESSMENTS–1988 AND 1998 • APPENDIX A 47

Where:

mi

 and mj are the means for groups i and j, and

S2

mi

 and S2

mj

 are the jackknife estimates of

sampling variance for groups i and j.

The reader should note that this procedure

uses a conservative estimate of the standard error

of the difference (i.e., one that may overstate

sampling variability), since the estimates of the

group averages or percentages will be positively

correlated to an unknown extent due to the

sampling plan. However, since sources of survey

error other than sampling are not accounted for

in the standard errors, using these conservative

estimates has been considered advisable.

Moreover, direct estimation of the standard

errors of all reported differences would involve

too heavy a computational burden to be

implemented in practice.

In some cases, the differences between

groups were not discussed in this report. This

happened for one of two reasons: (a) if the

comparison involved an extreme percentage

(as defined above); or (b) if the standard error

for either group was subject to a large degree of

uncertainty (i.e., the coefficient of variation is

greater than 20 percent, denoted by “**” in the

tables).3

 In either case, the results of any statistical

test involving that group needs to be interpreted

with caution, and so the results of such tests are

not discussed in this report.

Effective degrees of freedom. Because of

the clustered nature of the sample, the “effective

degrees of freedom” for this T-test is considerably

less than the number of students entering into

the comparison, and, indeed, less than the number

of PSU pairs that go into its computation. (See

the forthcoming NAEP 1998 Technical Report

for more details.) The degrees of freedom of

this t-test are estimated by an approximation

given by Johnson & Rust4

 as follows:

where the summation is over the two groups

being compared. The item, dfk

, is the degree of

freedom estimate for the variance of the mean

mk

 and is defined by Satterthwaite5 with a

correction term suggested by Johnson and Rust.

The correction term was derived by matching

estimates of the first two moments of the variance

to those of a chi-square random variable.

Here, j stands for jackknife replicate j, and the

summations are over all replicates, usually 62 in

NAEP. The L term is the number of replicate

pairs (usually 62) in the the sample. The mjk

term is the mean of subgroup k for the jth

jackknife replicate. The term mk

 is the overall

mean for subgroup k, using the overall weights

and the first plausible value.

The number of degrees of freedom for the

variance equals the number of independent pieces

of information used to generate the variance. In

the case of data from NAEP, the pieces of

information are the 62 squared differences

3 Estimates of standard errors that are subject to a large degree of

uncertainty are designated “**.” In such cases, the standard

error—and any confidence intervals or significance tests among

these standard errors—should be interpreted with caution.

4 Johnson, E. and Rust, K. (1992). Effective degrees of freedom

for variance estimates from a complex sample survey. Proceedings

of the Section on Survey Research Methods, American Statistical

Association, 863–866.

5 Satterthwaite, F.E. (1941). Synthesis of variance. Psychometrika

(16) 5, 309–316.
48 NAEP CIVICS ASSESSMENTS–1988 AND 1998 • APPENDIX A

(mjk – mk)2, each supplying at most one degree

of freedom (regardless of how many individuals

were sampled within PSUs). If some of the

squared differences (mjk – mk

)2 are much larger

than others, the variance estimate of mk

 is

predominantly estimating the sum of these larger

components, which dominate the remaining

terms. The effective degrees of freedom of S 2

mk

in this case will be nearer to the number of

dominant terms. The estimate, dfk, reflects

these relationships.

The two formulae above show that when dfk

is small, the degrees of freedom for the t-test,

df, will also be small. This will tend to be the

case when only a few PSU pairs have information

about subgroup differences relevant to a t-test.

It will also be the case when a few PSU pairs

have subgroup differences much larger than

other PSU pairs. With a clustered sample and a

practical sample size, it is not possible to stratify

over every group of potential interest. It is thus

inevitable that in a particular assessment for

some groups, some pairs within some PSUs

will be less well matched than others. Because

this depresses the estimate of effective degrees

of freedom, it has a conservative effect on

declaring significance for comparisons involving

those groups.

Conducting multiple tests. In many

applications of significance testing, the t-test

value is compared to a t-distribution with effective

degrees of freedom, as given in the previous

section, and is assessed at a nominal level of

.05/2 for a two-tailed test. However, in most

sections of this report, many different groups are

compared (i.e., multiple sets of means are being

analyzed). In sets of comparisons, statistical

theory indicates that the certainty associated with

the entire set of comparisons is less than that

attributable to each individual comparison from

the set. To hold the significance level for the set

of comparisons at a particular level ( = .05),

adjustments (called “multiple comparison

procedures”6

) must be made to the methods

described in the previous section. One such

procedure, the Benjamini and Hochberg False

Discovery Rate (FDR) procedure,7

 was used to

control the certainty level.

Unlike the other multiple comparison

procedures, such as the Bonferroni procedure,

that control the familywise error rate (i.e., the

probability of making even one false rejection in

the set of comparisons), the FDR procedure

controls the expected proportion of falsely rejected

hypotheses. In other words, rather than holding

the probability, , of even one true null hypothesis

being rejected at some level, the Benjamini and

Hochberg False Discovery Rate controls the

expected proportion of true null hypotheses

declared significant. The FDR = Expectation

(V/R), where V = Number of true null

hypotheses declared significant, and R = Total

number of hypotheses declared significant. This

is accomplished by ordering the contrasts from

most probable to least probable, and testing

sequentially, with a systematically decreasing

value of , until a null hypothesis is rejected.

That hypothesis and all subsequent hypotheses,

which have smaller probability, are declared

significant. Familywise procedures such as

Bonferroni are considered conservative for large

families of comparisons.8

 Therefore, the FDR

procedure is more suitable for multiple

comparisons in NAEP than other procedures.

A detailed description of the FDR procedure

6 Miller, R.G. (1966). Simultaneous statistical inference. New York:

Wiley.

7 Benjamini, Y. & Hochberg, Y. (1995). Controlling the false

discovery rate: A practical and powerful approach to multiple

testing. Journal of the Royal Statistical Society, Series B, No. 1, pp.

298–300.

8 Williams, V.S.L., Jones, L.V., & Tukey, J.W. (1994, December).

Controlling error in multiple comparisons with special attention to

the National Assessment of Educational Progress. Research Triangle

Park, NC: National Institute of Statistical Sciences.
NAEP CIVICS ASSESSMENTS–1988 AND 1998 • APPENDIX A 49

appears in the forthcoming NAEP 1998

Technical Report.

Prior to the 1996 assessment, the Bonferroni

procedure was used for multiple comparisons.

Beginning in the 1996 assessment, the Bonferroni

procedure was used for multiple comparisons

involving small sets of intervals, and the FDR

for large sets. The 1998 assessment was the first

time NAEP used the Benjamini-Hochberg

procedure for all multiple comparisons.

The Benjamini and Hochberg FDR

application consists of arranging the m

significance tests in order, from lowest to

highest probability P(1) ≤ P(2) ...≤ P(m). To

obtain an overall control of the False Discovery

Rate at .05, compare P(m) — the largest

probability with .05. If P(m) ≤ .05, declare all

m comparisons significant. If not, compare

P (m – 1) with m – 1

m . (.05). If less than or

equal to this criterion, declare P(1) – P(m – 1)

significant. If not, follow the same procedure

with P(m – 2) down through P(1). Note that

P(1) ≤ 1

m . (.05)

is the level used in the Bonferroni procedure for

all comparisons.

When we conduct multiple comparisons, the

family size, or number of comparisons in a set,

is a crucial issue. The investigator has a choice

between prespecifying a limited number of

comparisons, and thus maintaining power, or

looking at many contrasts, sacrificing power for

the chance of finding additional significant

contrasts. For example, the parents’ highest

level of education variable has five categories:

Did not finish high school

Graduated from high school

Some education after high school

Graduated from college

I don’t know.

If we choose “graduated from college” as the

target category for the set, and contrast mean

proficiency of each of the other four categories

with the target, our family size is four, and under

FDR, any mean difference with probability

≤ .05/4 = .0125 will certainly be declared

significant. If, on the other hand, we want to test

all possible contrasts, family size becomes ten,

and significance in the lowest probability

category is guaranteed only if its probability

≤ .005. This tension between conserving power

and identifying a larger number of significant

comparisons at the risk of not finding any

requires careful thought and balancing. After

much discussion, it has been determined that

NAEP should use all possible comparisons

within a set in conducting multiple comparisons.

Cautions in Interpretations

As described earlier, the NAEP civics average

percentage correct summaries make it possible

to examine relationships between students’

performance and various background factors

measured by NAEP. However, a relationship that

exists between achievement and another variable

does not reveal the underlying cause of the

relationship, which may be influenced by a

number of other variables. Similarly, the

assessments do not capture the influence of

unmeasured variables. The results are most

useful when they are considered in combination

with other knowledge about the student

population and the educational system, such as

trends in instruction, changes in the school-age

population, and societal demands and

expectations.
50 NAEP CIVICS ASSESSMENTS–1988 AND 1998 • APPENDIX A

Grade 12 Participation Rates

and Motivation

NAEP has been described as a “low-stakes”

assessment. That is, students receive no individual

scores, and their NAEP performance has no

effect on their grades, promotions, or graduation.

There has been continued concern that this lack

of consequences affects participation rates of

students and schools, as well as the motivation of

students to perform well on NAEP. Of particular

concern has been the performance of twelfthgraders, who typically have lower student

participation rates than fourth-and eighthgraders, and who are more likely to omit

responses compared to the younger cohorts.

Motivation

To the extent that students in the NAEP sample

are not perhaps trying their hardest, NAEP

results may underestimate student performance.

The concern increases as students get older and

is particularly pronounced for twelfth-graders.

The students themselves furnish some evidence

about their motivation. As part of the

background questions, students were asked how

important it was to do well on the NAEP civics

assessment. They were asked to indicate

whether it was very important, important,

somewhat important, or not very important to

them. The percentage of students in the 1998

main civics assessment indicating they thought it

was either important or very important to do

well was 88 percent for fourth-graders, 59

percent for eighth-graders, and 33 percent for

twelfth-graders.

Several factors may contribute to this pattern.

NAEP was administered in the late winter, when

high school seniors often have other things on

their minds. As with participation rates, however,

the combined effect of these and other factors

is unknown.

It is also interesting to note that students

who indicated it was very important for them to

do well on NAEP did not have the highest average

scores. These data further cloud the relationship

between motivation and performance on NAEP.

Need for Future Research

More research is needed to delineate the factors

that contribute to nonparticipation and lack of

motivation. To that end, NCES has commissioned

a study of high school transcripts to learn more

about the academic performance of twelfth-grade

students who do not participate in the assessment.

In addition, NCES is currently investigating how

various types of incentives can be effectively used

to increase participation in NAEP.
NAEP CIVICS ASSESSMENTS–1988 AND 1998 • APPENDIX B 51

APPENDIX B

STANDARD ERRORS

Standard Errors

The comparisons presented in this reportare based on statistical tests that consider

the magnitude of the difference between group

averages or percentages and the standard errors

of those statistics. This appendix contains the

standard errors for the estimated averages and

percentages in all the tables and figures

throughout this report. Because NAEP scores

and percentages are based on samples rather

than the entire population(s), the results are

subject to a measure of uncertainty reflected in

the standard errors of the estimates. It can be

said with 95 percent certainty that, for each

population of interest, the value for the whole

population is within plus or minus two standard

errors of the estimate for the sample.

Table B1.2

Standard errors for average percentage of

correct responses to civics questions by gender:

1988 and 1998

1988 1998

Percentage of Percentage Percentage of Percentage

students correct students correct

Grade 4

Male 1.6 1.0 1.2 0.7

Female 1.6 1.0 1.2 0.6

Grade 8

Male 1.2 0.6 1.1 0.7

Female 1.2 0.5 1.1 0.7

Grade 12

Male 1.8 0.8 1.7 1.0

Female 1.8 0.5 1.7 0.7

SOURCE: National Center for Education Statistics, National Assessment

of Educational Progress (NAEP), Civics Assessments–1988 and 1998.

Standard errors for average percentage of

correct responses to civics questions by grade:

1988 and 1998

Grade 4 0.8 0.6

Grade 8 0.4 0.6

Grade 12 0.5 0.7

SOURCE: National Center for Education Statistics,

National Assessment of Educational Progress (NAEP),

Civics Assessments–1988 and 1998.

1988 1998

Table B1.1
52 NAEP CIVICS ASSESSMENTS–1988 AND 1998 • APPENDIX B

Figure B1.1

Standard errors for grade 4 sample questions, 1988 and 1998

Content area: Democratic principles and the purpose of government

SOURCE: National Center for Education Statistics, National Assessment of Educational

Progress (NAEP), Civics Assessments–1988 and 1998.

Responses Percentage of students

1988 1998

1.6 1.3

B 0.9 0.7

C 1.6 1.1

Responses Percentage of students

1988 1998

A 0.9 0.4

1.5 1.1

C 1.1 0.9

D 0.5 0.3

1988 1998

Percentage of Percentage Percentage of Percentage

students correct students correct

Standard errors for average percentage of correct responses to civics

questions by race/ethnicity: 1988 and 1998

Table B1.3

Grade 4

White 1.3 0.9 0.6 0.7

Black 1.2 1.6 0.4 1.2

Hispanic 0.7 1.5 0.5 1.2

Grade 8

White 0.8 0.5 0.3 0.8

Black 0.7 1.0 0.2 1.1

Hispanic 0.6 0.8 0.2 0.8

Grade 12

White 1.0 0.7 0.9 0.7

Black 0.7 1.1 0.7 1.9

Hispanic 0.5 1.1 0.4 1.1

SOURCE: National Center for Education Statistics, National Assessment of

Educational Progress (NAEP), Civics Assessments–1988 and 1998.
NAEP CIVICS ASSESSMENTS–1988 AND 1998 • APPENDIX B 53

Figure B1.2

Standard errors for grade 8 sample questions, 1988 and 1998

Content area: Democratic principles and the purpose of government

SOURCE: National Center for Education Statistics, National Assessment of Educational

Progress (NAEP), Civics Assessments–1988 and 1998.

Responses Percentage of students

1988 1998

A 0.5 0.4

1.4 1.6

C 0.9 0.8

D 1.1 1.3

E 1.0 0.6

Responses Percentage of students

1988 1998

A 1.6 1.5

1.5 1.4

C 1.6 1.2

Figure B1.3

Standard errors for grade 12 sample questions, 1988 and 1998

Content area: Democratic principles and the purpose of government

SOURCE: National Center for Education Statistics, National Assessment of Educational

Progress (NAEP), Civics Assessments–1988 and 1998.

Responses Percentage of students

1988 1998

1.8 1.3

B 1.5 1.1

C 1.0 1.0

D 0.9 0.6

Responses Percentage of students

1988 1998

A 1.9 1.3

B 1.1 0.8

C 2.1 1.0

1.7 1.4
54 NAEP CIVICS ASSESSMENTS–1988 AND 1998 • APPENDIX B

Figure B1.4

Standard errors for grade 4 sample questions, 1988 and 1998

Content area: Political institutions (structures and functions)

SOURCE: National Center for Education Statistics, National Assessment of Educational

Progress (NAEP), Civics Assessments–1988 and 1998.

Responses Percentage of students

1988 1998

1.6 1.4

B 0.9 0.6

C 1.4 1.0

D 0.8 0.6

Responses Percentage of students

1988 1998

A 1.5 1.0

2.1 1.3

C 1.3 1.2

Figure B1.5

Standard errors for grade 8 sample questions, 1988 and 1998

Content area: Political institutions (structures and functions)

SOURCE: National Center for Education Statistics, National Assessment of Educational

Progress (NAEP), Civics Assessments–1988 and 1998.

Responses Percentage of students

1988 1998

A 0.9 0.9

1.3 1.2

C 1.1 0.9

Responses Percentage of students

1988 1998

A 1.1 0.8

1.5 1.3

C 1.4 1.0

D 0.8 0.5
NAEP CIVICS ASSESSMENTS–1988 AND 1998 • APPENDIX B 55

Figure B1.7

Standard errors for grade 4 sample questions, 1988 and 1998

Content area: Political processes

SOURCE: National Center for Education Statistics, National Assessment of Educational

Progress (NAEP), Civics Assessments–1988 and 1998.

Responses Percentage of students

1988 1998

1.8 1.5

B 1.2 1.2

C 1.2 0.7

D 0.8 0.6

Responses Percentage of students

1988 1998

A 1.4 1.5

B 1.4 1.1

C 1.4 1.3

1.2 1.2

Figure B1.6

Standard errors for grade 12 sample questions, 1988 and 1998

Content area: Political institutions (structures and functions)

SOURCE: National Center for Education Statistics, National Assessment of Educational

Progress (NAEP), Civics Assessments–1988 and 1998.

Responses Percentage of students

1988 1998

A 1.0 0.6

1.2 1.2

C 0.8 0.5

D 0.6 0.5

E 1.0 0.8

Responses Percentage of students

1988 1998

2.6 1.2

B 2.1 1.3

C 0.8 0.6

D 0.9 0.6

E 0.8 1.1
56 NAEP CIVICS ASSESSMENTS–1988 AND 1998 • APPENDIX B

Figure B1.9

Standard errors for grade 12 sample questions, 1988 and 1998

Content area: Political processes

SOURCE: National Center for Education Statistics, National Assessment of Educational

Progress (NAEP), Civics Assessments–1988 and 1998.

Responses Percentage of students

1988 1998

A 1.2 0.8

1.9 1.2

C 0.7 0.4

D 0.9 0.7

Responses Percentage of students

1988 1998

A 0.9 0.7

1.8 1.4

C 1.5 1.0

D 1.0 1.1

Figure B1.8

SOURCE: National Center for Education Statistics, National Assessment of Educational

Progress (NAEP), Civics Assessments–1988 and 1998.

Responses Percentage of students

1988 1998

A 1.0 0.8

B 0.7 0.3

C 0.5 0.3

1.4 1.1

Responses Percentage of students

1988 1998

A 1.2 0.8

B 0.9 0.6

C 2.1 1.2

1.8 1.6

Standard errors for grade 8 sample questions, 1988 and 1998

Content area: Political processes
NAEP CIVICS ASSESSMENTS–1988 AND 1998 • APPENDIX B 57

Figure B1.10

Standard errors for grade 4 sample questions, 1988 and 1998

Content area: Rights, responsibilities, and the law

SOURCE: National Center for Education Statistics, National Assessment of Educational

Progress (NAEP), Civics Assessments–1988 and 1998.

Responses Percentage of students

1988 1998

A 0.6 0.4

B 0.9 0.6

C 0.6 0.3

1.1 0.8

Responses Percentage of students

1988 1998

A 2.1 1.3

1.6 1.4

C 1.0 0.8

D 0.7 0.5

E 1.4 1.3

Responses Percentage of students

1988 1998

 15. 1.6 1.1

B 1.1 0.6

C 1.5 0.9

 16. A 0.8 0.8

1.1 1.1

C 0.6 0.6

 17. 2.1 1.0

B 1.6 0.8

C 1.5 1.0

Standard errors for grade 8 sample questions, 1988 and 1998

Content area: Rights, responsibilities, and the law

SOURCE: National Center for Education Statistics, National Assessment of Educational

Progress (NAEP), Civics Assessments–1988 and 1998.

Figure B1.11

Responses Percentage of students

1988 1998

A 1.5 1.0

1.8 1.2

C 1.3 0.8
58 NAEP CIVICS ASSESSMENTS–1988 AND 1998 • APPENDIX B

Figure B1.12

Standard errors for grade 12 sample questions, 1988 and 1998

Content area: Rights, responsibilities, and the law

SOURCE: National Center for Education Statistics, National Assessment of Educational

Progress (NAEP), Civics Assessments–1988 and 1998.

Responses Percentage of students

1988 1998

1.1 0.8

B 0.5 0.4

C 0.5 0.4

D 0.7 0.4

Responses Percentage of students

1988 1998

A 1.5 1.5

B 0.6 0.6

1.5 1.7

D 0.8 0.6

** Sample size is insufficient to permit a reliable estimate.

1 The nature of the sample does not allow accurate determination of the variability of this value.

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress

(NAEP), Civics Assessments–1988 and 1998.

Total 3.2 2.5 1.5 2.1 1.6 1.4 0.7 0.7 1.9 1.0

Public 3.6 2.6 1.6 2.1 1.8 1.5 0.7 0.8 2.2 1.1

Nonpublic 8.91 7.71 8.41 5.71 4.41 4.41 3.71 1.01 ** 0.71

Standard errors for percentage of students reporting the frequency of

social studies class by type of school, grade 4: 1988 and 1998

Table B2.1

1988 1998 1988 1998 1988 1998 1988 1998 1988 1998

Percentage of students

3–4 times 1–2 times Less than Never/hardly

How often do you Daily a week a week once a week ever

usually have social

studies class in school?
NAEP CIVICS ASSESSMENTS–1988 AND 1998 • APPENDIX B 59

Table B2.3

SOURCE: National Center for Education Statistics, National Assessment of Educational

Progress (NAEP), Civics Assessments–1988 and 1998.

1988 1998 1988 1998 1988 1998

A lot Some Not at all

Grade 4

Standard errors for percentage of students who reported how much they

studied selected civics topics, grade 4: 1988 and 1998

How much have you

studied the following

topics in American

government or civics?

How laws are made 1.3 1.6 2.0 1.5 2.1 1.5

About judges and courts 1.0 1.5 2.1 1.5 1.7 2.2

President and leaders

of our country 1.4 1.5 1.5 1.6 1.5 1.2

Elections and voting 1.6 1.3 2.1 1.3 2.5 1.6

Your community 1.6 1.9 1.2 1.5 1.3 1.1

The rights and

responsibilities of citizens 1.8 1.6 1.8 1.5 1.4 1.0

Studied American

government or civics in

the following grades:

SOURCE: National Center for Education Statistics, National Assessment of Educational

Progress (NAEP), Civics Assessments–1988 and 1998.

Standard errors for percentage of students reporting they studied American

government or civics by grade levels, grades 8 and 12: 1988 and 1998

Grade 8

Yes No Don’t know

1988 1998 1988 1998 1988 1998

Grade 12

Table B2.2

5th grade 0.9 1.2 1.1 1.2 0.9 1.0

6th grade 1.1 1.5 1.3 1.4 0.9 0.8

7th grade 1.8 1.7 1.5 1.5 0.7 0.8

8th grade 1.8 1.4 1.3 0.9 0.8 0.9

9th grade 2.6 2.5 2.4 2.8 0.5 0.8

10th grade 1.6 2.1 1.6 2.0 0.6 0.7

11th grade 2.3 1.6 2.3 1.7 0.6 0.5

12th grade 2.5 3.3 2.5 3.2 0.4 0.5
60 NAEP CIVICS ASSESSMENTS–1988 AND 1998 • APPENDIX B

How much have you

studied the following

topics in American

government or civics?

U.S. Constitution and Bill of Rights 1.8 2.2 1.9 2.0 0.7 0.7

Congress 1.9 2.3 1.5 1.8 0.8 1.1

How laws are made 1.8 2.0 1.4 1.5 0.9 1.2

The court system 1.7 1.6 1.1 1.2 1.5 1.5

President and the cabinet 1.4 1.9 1.2 1.5 1.0 1.5

Political parties, elections, and voting 1.6 1.6 1.2 1.3 1.0 0.9

State and local government 1.5 1.7 1.2 1.4 1.1 1.2

Principles of democratic government 1.2 0.9 1.0 1.4 1.3 1.6

Other forms of government 1.1 1.2 1.2 1.2 1.2 1.0

Rights and responsibilities of citizens 1.1 1.4 1.1 1.1 0.8 0.7

SOURCE: National Center for Education Statistics, National Assessment of Educational

Progress (NAEP), Civics Assessments–1988 and 1998.

1988 1998 1988 1998 1988 1998

A lot Some Not at all

Grade 8

Table B2.4

Standard errors for percentage of students who reported how much they

studied selected civics topics, grade 8: 1988 and 1998

SOURCE: National Center for Education Statistics, National Assessment of Educational

Progress (NAEP), Civics Assessments–1988 and 1998.

1988 1998 1988 1998 1988 1998

A lot Some Not at all

Grade 12

Table B2.5

Standard errors for percentage of students who reported how much they

studied selected civics topics, grade 12: 1988 and 1998

Since the beginning of ninth

grade, how much have you

studied the following topics

in American government

or civics?

U.S. Constitution and Bill of Rights 2.0 1.7 1.9 1.6 0.4 0.6

Congress 1.6 1.9 1.6 1.8 0.5 0.7

How laws are made 1.6 1.8 1.5 1.5 0.6 0.7

The court system 1.5 1.8 1.2 1.7 0.7 0.8

President and the cabinet 1.4 1.6 1.3 1.2 0.6 0.8

Political parties, elections, and voting 1.3 1.7 1.3 1.4 0.4 0.7

State and local government 1.4 1.5 1.4 1.6 0.6 0.9

Principles of democratic government 1.4 1.6 1.4 1.2 0.6 1.0

Other forms of government 1.2 1.4 1.3 1.3 0.8 0.9

Rights and responsibilities of citizens 1.2 1.5 1.2 1.5 0.4 0.7
NAEP CIVICS ASSESSMENTS–1988 AND 1998 • APPENDIX B 61

Table B3.1

SOURCE: National Center for Education Statistics, National Assessment of

Educational Progress (NAEP), Civics Assessments–1988 and 1998.

Percentage of students

Almost Once or Once or A few

every day twice a week twice a month times a year Never

1988 1998 1988 1998 1988 1998 1988 1998 1988 1998

Standard errors for percentage of students reporting how often their teachers

use various classroom practices, grade 8: 1988 and 1998

How often has your American

government or civics teacher asked

you to do each of the following?

Read material from textbook 1.9 2.1 1.8 1.5 0.6 0.8 0.3 0.4 0.4 0.3

Read extra material not in regular textbook 0.8 1.1 1.3 1.5 1.2 1.0 1.0 1.0 1.6 1.3

Memorize the material you have read 1.1 1.4 1.1 1.0 1.0 1.2 0.9 0.9 1.1 0.9

Discuss and analyze the

material you have read 1.7 1.5 1.4 1.2 0.8 0.6 0.4 0.4 0.6 0.6

Write short answers to questions 1.8 1.5 1.3 1.4 1.2 0.9 0.6 0.4 0.6 0.7

Write a report of three or more pages 0.4 0.5 0.6 0.6 0.9 1.6 1.4 1.3 1.8 1.9

Work on a group project 0.5 0.7 1.1 1.5 1.3 1.6 1.3 1.3 1.8 1.3

Give talks about what you are studying 1.6 1.3 0.9 0.9 0.8 0.8 0.7 0.6 1.0 0.8

Discuss current events 1.9 1.6 1.7 1.3 1.4 1.0 0.6 0.7 0.7 0.8

Take a test or quiz 1.3 1.2 1.7 1.8 1.6 1.9 0.2 0.3 0.3 0.2

Standard errors for percentage of students by number of years of civics studied,

grade 12: 1988 and 1998

Table B2.6

SOURCE: National Center for Education Statistics, National Assessment of

Educational Progress (NAEP), Civics Assessments–1988 and 1998.

Since the beginning of ninth grade,

how much American government or

civics coursework have you

completed up to now?

Percentage of students

1988 1998

None 0.9 0.8

Less than one-half year 0.7 0.7

One-half year 1.6 1.4

Between one-half and one year 0.6 0.9

One year 1.1 1.2

More than one year 1.9 1.8
62 NAEP CIVICS ASSESSMENTS–1988 AND 1998 • APPENDIX B

How often has your American

government or civics teacher asked

you to do each of the following?

Read material from textbook 2.1 2.2 1.6 1.8 0.8 0.9 0.4 0.6 0.6 1.1

Read extra material not in regular textbook 0.9 1.7 1.3 1.9 1.0 1.5 0.8 0.9 1.0 0.7

Memorize the material you have read 1.3 1.3 1.2 1.4 1.5 1.2 0.9 0.9 0.9 1.5

Discuss and analyze the

material you have read 2.1 1.6 1.6 1.3 0.8 0.9 0.4 0.5 0.4 0.5

Write short answers to questions 1.4 1.6 1.3 1.6 0.9 1.1 0.5 0.7 0.6 1.0

Write a report of three or more pages 0.5 0.6 0.7 1.1 1.4 1.9 1.7 1.4 1.9 2.0

Work on a group project 0.5 1.0 0.9 1.9 1.5 1.3 1.0 1.5 1.8 1.7

Give talks about what you are studying 1.7 1.9 0.8 1.7 1.2 1.1 1.2 0.9 1.4 1.5

Discuss current events 1.9 2.5 1.7 1.8 1.0 1.0 0.8 0.5 0.3 0.7

Take a test or quiz 1.0 1.3 1.6 2.1 1.4 1.7 0.3 0.4 0.3 0.3

SOURCE: National Center for Education Statistics, National Assessment of

Educational Progress (NAEP), Civics Assessments–1988 and 1998.

Percentage of students

Almost Once or Once or A few

every day twice a week twice a month times a year Never

1988 1998 1988 1998 1988 1998 1988 1998 1988 1998

Table B3.3

Standard errors for percentage of students reporting how often they

discuss current events, grade 4: 1988 and 1998

Percentage of students

Almost Once or Once or A few

every day twice a week twice a month times a year Never

1988 1998 1988 1998 1988 1998 1988 1998 1988 1998

How often do you

discuss current events

in social studies class?

1.2 1.7 1.7 1.3 1.4 1.0 1.0 0.8 1.6 1.0

SOURCE: National Center for Education Statistics, National Assessment of

Educational Progress (NAEP), Civics Assessments–1988 and 1998.

Table B3.2

Standard errors for percentage of students reporting how often their teachers

use various classroom practices, grade 12: 1988 and 1998
NAEP CIVICS ASSESSMENTS–1988 AND 1998 • APPENDIX B 63

Percentage of students

Table B3.4

Standard errors for percentage of students reporting the amount of time they

spend weekly on civics or government homework, grade 12: 1988 and 1998

NOTE: Some of the data in this table are the result of collapsing the original

response options in the question. The question contained the following

choices: I usually haven’t had homework assigned; I have had homework

but I usually haven’t done it; less than 1 hour; 1 hour; 2 hours; 3 hours;

4 hours; 5 hours or more.

SOURCE: National Center for Education Statistics, National Assessment of

Educational Progress (NAEP), Civics Assessments–1988 and 1998.

Don’t do None, < 1 hour 1–2 hours 3–4 hours 5+ hours

1988 1998 1988 1998 1988 1998 1988 1998 1988 1998

How much time do you

spend weekly on civics or

government homework?

0.5 0.7 1.7 2.2 1.1 1.5 1.1 1.1 0.4 0.6

Table B3.5

Standard errors for percentage of students reporting receiving newspapers

in the home, grades 4, 8, and 12: 1988 and 1998

Percentage of students

Yes No I don’t know.

1988 1998 1988 1998 1988 1998

Does your family get a

newspaper regularly?

SOURCE: National Center for Education Statistics, National Assessment of

Educational Progress (NAEP), Civics Assessments–1988 and 1998.

Grade 4 2.4 1.7 1.9 1.7 1.0 0.7

Grade 8 1.1 1.4 1.1 1.3 0.3 0.5

Grade 12 1.0 1.4 1.0 1.3 0.2 0.3

NAEP CIVICS ASSESSMENTS–1988 AND 1998 • APPENDIX C 65

APPENDIX C

Average Percentage Correct

1 The items were categorized as follows: 1 = Democratic principles and the purpose of government; 2 = Structures and

functions of political institutions; 3 = Political processes; 4 = Rights, responsibilities, and the law

> 1988 significantly greater than 1998

< 1988 significantly less than 1998

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), Civics

Assessments–1988 and 1998.

Table C.1

Average percentage of correct responses to each test question in the

civics special study, grade 4: 1988 and 1998

Content

Question description code1 1988 1998

People of the U.S. vote for President 2 97.0 97.0

The White House is the home of President 2 91.4 < 94.4

Judge is person in charge of courtroom 4 89.2 90.6

There are 50 states in the United States 2 88.5 90.0

Vice President becomes President if President of U.S. dies 2 88.1 91.9

Crosswalk laws help people cross safely 4 84.6 88.5

Local governments do not clean family’s house 2 82.3 81.8

Making laws is major purpose of government in U.S. 1 81.8 83.7

Local governments catch lawbreakers 2 79.2 78.4

Local governments take care of parks 2 76.9 74.3

Judge must always obey laws of U.S. 1 76.8 < 81.4

Madison is a candidate for tax collector 3 75.3 75.8

Arrested person has no right to escape 4 74.5 77.8

Local governments clean streets 2 74.5 > 70.1

Local governments do not grow vegetables for food 2 73.3 76.4

Local government does not fix family’s car 2 72.1 71.7

Local governments put up stop signs 2 70.7 66.7

In U.S. a farm is not owned and run by government 2 69.5 73.3

Laws can be changed 4 68.7 73.7

In U.S. police station is owned and run by government 2 68.6 70.2

Must be citizen to vote in national or state election 3 68.5 70.5

Army general must always obey U.S. law 1 65.7 68.5

Laws in U.S. do not say adults must vote 4 64.8 69.3

Courts decide whether someone has broken the law 2 64.3 63.9

Bill of Rights; first 10 amendments to Constitution 4 64.0 > 54.3

3 people will be elected to town council 3 63.9 63.4

July 4 observes Independence Day 2 63.6 < 78.6

PERCENTAGE CORRECTAVERAGE
66 NAEP CIVICS ASSESSMENTS–1988 AND 1998 • APPENDIX C

Table C.1 continued

Average percentage of correct responses to each test question in the

civics special study, grade 4: 1988 and 1998

1 The items were categorized as follows: 1 = Democratic principles and the purpose of government; 2 = Structures and

functions of political institutions; 3 = Political processes; 4 = Rights, responsibilities, and the law

< 1988 significantly less than 1998

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), Civics

Assessments–1988 and 1998.

Content

Question description code1 1988 1998

Member of Congress must always obey laws 1 63.1 < 72.3

President can’t order someone else to break law 1 61.2 < 68.6

Local governments do not run restaurants 2 60.0 63.9

Most democratic; class makes list and votes 1 59.9 < 69.0

In U.S. roads are owned and operated by government 2 59.7 58.7

FBI agent must always obey the law 1 58.8 < 65.2

In U.S. individual citizen has right to vote 4 58.5 62.1

All 3 levels of government try to prevent crime 2 57.3 54.3

National government makes laws for whole country 2 52.3 51.6

In U.S. movie theater is not owned and run by government 2 49.9 52.2

Mayor is usually head of city government 2 49.6 46.5

President has no right to stop radio, newspapers 4 48.5 50.8

President can’t break law even if thinks it’s for good 1 46.5 < 56.0

“Justice for all” means everybody treated equally 1 46.5 < 61.6

Local government checks parking meters 2 44.6 49.8

People in U.S. elect government officials by secret ballot 3 44.2 45.0

President of U.S. is elected for a term of 4 years 2 42.1 < 62.2

In U.S. gas station is not owned and operated by government 2 39.0 < 44.7

Man should be given trial by jury if he wants it 4 34.1 30.8

Separation of powers; division of authority among 3 branches 2 33.4 31.5

All 3 levels of government do not have army/navy/air force 2 31.4 < 38.4

Workers’ action called striking 4 31.0 33.0

Supreme Court decides constitutionality 3 17.8 17.1

Workers get people to boycott 4 17.7 22.5
NAEP CIVICS ASSESSMENTS–1988 AND 1998 • APPENDIX C 67

1 The items were categorized as follows: 1 = Democratic principles and the purpose of government; 2 = Structures and

functions of political institutions; 3 = Political processes; 4 = Rights, responsibilities, and the law

> 1988 significantly greater than 1998

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), Civics

Assessments–1988 and 1998.

Table C.2

Average percentage of correct responses to each test question in the

civics special study, grade 8: 1988 and 1998

Content

Question description code1 1988 1998

Crosswalk laws help people cross safely 4 96.3 95.4

Vice-President becomes President if President of U.S. dies 2 95.8 94.0

Accused person has right to a lawyer 4 94.9 92.5

Madison is a candidate for tax collector 3 94.5 > 91.5

Accused person has right to know what accused of 4 94.2 93.2

Local governments take care of parks 2 92.9 91.0

Local governments clean streets 2 92.0 > 89.2

Local governments do not clean family’s house 2 91.0 88.3

Refusing to obey laws is not legal 4 88.8 > 84.4

Must be citizen to vote in national or state election 3 87.4 89.9

Not paying portion of taxes is illegal 4 87.3 > 83.5

Picking neighbors is not a constitutional right 4 86.3 82.3

State governors are usually elected 3 86.0 > 78.1

In U.S. individual citizen has right to vote 4 85.6 83.8

Accused person has right to remain silent 4 85.4 82.5

Accused can’t go free if returns money 4 83.3 82.0

3 people will be elected to town council 3 83.2 80.9

Presenting views to media is legal 4 80.8 80.4

President of U.S. is elected for a term of 4 years 2 80.8 83.9

People in U.S. elect government officials by secret ballot 3 80.8 > 69.8

“Justice for all” means everybody treated equally 1 79.8 81.8

Writing letters of protest is legal 4 77.7 76.8

Drawing c; poll tax is illegal 4 76.8 73.5

Local governments do not run restaurants 2 76.8 > 72.8

Federal government does not provide jobs for everyone 2 76.7 > 71.3

U.S. court can decide if sex discrimination on job 3 75.6 78.7

Rights in Constitution protect individual freedom 1 74.1 73.0

U.S. is not only country with elected leaders 1 74.0 77.1

Accused has right to lawyer and jury trial (I & II) 4 73.9 74.7

Police can’t jail person until collect evidence 4 73.3 > 67.7

Two candidates for election good: provides choice 1 73.1 70.3

U.S. is not only country with political parties 1 72.4 70.4

Police do nothing unless law is broken 4 71.6 > 65.3

Suing in court is legal 4 70.7 70.0

Bill of Rights; first 10 amendments to Constitution 4 70.7 75.1

Congress has no right to curtail freedom of press 4 68.7 66.5

Local governments operate public schools 2 67.9 67.4

Court can decide whether one’s idea taken 3 67.5 68.0

Regulating foreign trade is function of federal government 2 66.7 64.9
68 NAEP CIVICS ASSESSMENTS–1988 AND 1998 • APPENDIX C

Table C.2 continued

Average percentage of correct responses to each test question in the

civics special study, grade 8: 1988 and 1998

1 The items were categorized as follows: 1 = Democratic principles and the purpose of government; 2 = Structures and

functions of political institutions; 3 = Political processes; 4 = Rights, responsibilities, and the law

> 1988 significantly greater than 1998

< 1988 significantly less than 1998

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), Civics

Assessments–1988 and 1998.

Content

Question description code1 1988 1998

Bill of Rights prohibits cruel and unusual punishment 4 66.3 65.8

Court in U.S. cannot declare war 3 65.8 62.9

Congress cannot establish national church 4 63.5 > 57.6

Local governments do not make laws about military service 2 63.1 62.5

Issue in headlines concerns local government 2 62.7 61.9

Statement of Civil Rights in Constitution 1 60.7 61.9

U.S. senators are part of legislative branch 2 60.4 60.1

Federal government regulates food and drug quality 2 60.0 59.5

Secretary of Defense is appointed 3 59.8 61.8

Separation of powers; division of authority among 3 branches 2 57.6 61.1

Engaging in hunger strike is legal 4 57.2 > 51.5

U.S. Congress is House of Representatives and Senate 2 54.8 > 45.8

Powers of Congress set forth in Constitution 1 53.4 > 47.8

In U.S. senator is elected to office 3 52.1 50.7

Governor is chief executive of a state 2 51.4 47.5

Participating in boycott is legal 4 50.8 < 59.9

Federal government helps farmers with money and services 2 49.1 > 27.6

Local governments do not issue passports 2 46.6 47.9

Supreme Court stops war by declaring act unconstitutional 3 45.2 > 40.8

Organizing a recall election is legal 3 44.9 39.4

Laws for marriage are not a function of federal government 2 44.4 46.0

Representatives are elected to office 3 43.9 49.0

Secretary of State is member of President’s cabinet 2 43.8 > 36.5

Lobbying for votes is legal 3 43.5 43.7

U.S. is not only country with written constitution 1 42.6 45.0

Chart 4; legislative process for bill 3 41.9 46.7

Supreme Court decides constitutionality 3 40.3 41.2

Library needed support of former city council 3 38.8 35.3

Congress can pass law to double income tax 3 34.3 30.1

NRA activity called lobbying 3 32.5 > 22.2

Local governments do not license lawyers 2 31.2 33.0

Federal government helps buy homes and start businesses 2 31.2 35.9

Effect of laws; prevented some groups from voting 4 29.9 < 40.4

Senate approves appointments to Supreme Court 2 29.4 28.7

Appeal case means refer to higher court 4 28.3 27.1

Department of State most concerned with foreign affairs 2 20.7 > 14.3

Presidential candidate nominated by national convention 3 15.8 18.1
NAEP CIVICS ASSESSMENTS–1988 AND 1998 • APPENDIX C 69

Table C.3

Average percentage of correct responses to each test question in the

civics special study, grade 12: 1988 and 1998

1 The items were categorized as follows: 1 = Democratic principles and the purpose of government; 2 = Structures and

functions of political institutions; 3 = Political processes; 4 = Rights, responsibilities, and the law

> 1988 significantly greater than 1998

< 1988 significantly less than 1998

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), Civics

Assessments–1988 and 1998.

Content

Question description code1 1988 1998

Accused person has right to a lawyer 4 98.1 97.5

Accused person has right to know what accused of 4 97.4 95.4

Picking neighbors is not a constitutional right 4 96.0 93.9

Accused person has right to remain silent 4 96.0 > 93.3

State governors are usually elected 3 92.8 91.6

Accused can’t go free if returns money 4 92.8 90.8

First 10 amendments deal with individual rights 4 92.0 94.1

Not required to vote in national election; poll tax 4 91.6 92.1

Right to counsel means representation by lawyer 4 90.9 90.9

Police can’t jail person until collect evidence 4 90.3 > 84.1

Duty of jury; determine if guilt proven 4 89.8 88.5

National government prints money 2 87.2 87.6

Regulating foreign trade is function of federal government 2 85.3 81.2

Police do nothing unless law is broken 4 85.3 86.4

Legal way to change law; lobby 4 84.0 81.9

Congress has no right to curtail freedom of press 4 83.9 84.1

Congress cannot establish national church 4 82.8 82.5

Federal government does not provide jobs for everyone 2 82.4 > 78.3

Federal government regulates food and drug quality 2 82.3 81.7

Local governments do not make laws about military service 2 81.1 79.1

Secretary of Defense is appointed 3 80.8 76.5

Highest level of local government is county 2 80.3 78.6

U.S. Constitution; major basis for Supreme Court cases 3 78.5 79.0

U.S. Congress is House of Representatives and Senate 2 76.9 > 72.1

Not freedom of speech to yell ‘fire’ in a theater as joke 4 76.6 82.1

Both groups allowed to speak 4 76.5 73.9

Invoke 5th Amendment; not answer to avoid self-incrimination 4 74.9 76.3

In U.S. senator is elected to office 3 74.6 72.3

Governor is chief executive of a state 2 73.8 > 66.3

Conflict between order vs. freedom of speech 1 73.7 75.5

Constitution divides power between states and national gov. 2 73.3 76.3

Local governments operate public schools 2 73.2 74.4

Party members choose candidates in primary election 3 72.4 68.8

Federal government helps farmers with money and services 2 71.5 > 59.6

Member elected to preside over house is Speaker 2 71.1 70.3

Most likely to vote; middle-aged professionals 4 69.8 < 77.5

Statement says citizens of democracy need to be informed 1 69.3 67.0
70 NAEP CIVICS ASSESSMENTS–1988 AND 1998 • APPENDIX C

Table C.3 continued

Average percentage of correct responses to each test question in the

civics special study, grade 12: 1988 and 1998

1 The items were categorized as follows: 1 = Democratic principles and the purpose of government; 2 = Structures and

functions of political institutions; 3 = Political processes; 4 = Rights, responsibilities, and the law

> 1988 significantly greater than 1998

< 1988 significantly less than 1998

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), Civics

Assessments–1988 and 1998.

Content

Question description code1 1988 1998

Declaration of war approved by Congress 2 68.6 64.5

Brown V. Board of Ed.; segregated schools unconstitutional 2 68.0 65.5

Supreme Court can declare an act unconstitutional 3 67.9 65.8

Separation of powers; prevent too much power in one body 2 67.4 66.3

Government of U.S. is representative democracy 1 65.8 65.3

Laws for marriage are not a function of federal government 2 65.6 60.9

Attorney General heads Department of Justice 2 64.4 > 57.7

Chief Justice appointed by President, consent of Senate 3 64.0 59.0

Lobbyists called third house because they influence law 3 64.0 64.4

First Amendment guarantees freedom of religion 4 63.0 66.5

Number of representatives varies according to state population 2 62.5 63.1

Table shows more educated are more politically active 4 62.2 62.8

Criticism of Electoral College; win does not reflect popular vote 3 61.5 62.6

Each state has two senators in the U.S. Senate 2 59.6 58.1

Quote is from Declaration of Independence 1 59.2 > 50.3

Secretary of State is member of presidential cabinet 2 58.4 > 50.7

Local governments do not issue passports 2 57.0 55.7

National conventions nominate presidential candidates 3 55.3 > 47.7

Judicial review; decide constitutionality of government’s acts 3 54.4 52.5

Interstate highways; cooperation state and federal government 3 54.2 53.9

Checks and balances; Congress investigates agency 2 53.9 51.9

Supreme Court least influenced by lobbying 3 53.3 55.8

Federal judges appointed by President, confirmed by Senate 2 49.7 > 41.7

Referendum; vote yes or no to question on ballot 3 48.3 43.6

Power to tax belongs to Congress 2 47.6 50.8

Voters with party affiliation vote in closed primary 3 44.0 > 35.9

Treaty must be ratified by the Senate 2 43.0 38.9

Local governments do not license lawyers 2 40.5 37.9

Nomination procedure for President set by political party 3 40.3 43.1

Congress can pass law to double income tax 3 37.7 40.1

Department of State most concerned with foreign affairs 2 35.2 > 24.8

Federal government helps buy homes and start businesses 2 34.9 < 42.4

Bicameralism is legislative branch with 2 houses 1 34.1 37.6

Authors of quote delegates to Seneca Falls convention 1 33.4 33.1

PAC set by special interests to raise money for candidate 3 32.7 34.2

Statement reflects theory of social contract 1 26.4 31.9
NAEP CIVICS ASSESSMENTS–1988 AND 1998 • APPENDIX D 71

APPENDIX D

John Eyster

Parker High School

Janeville, WI

David Harris

Oakland Schools

Waterford, MI

Milton Kato

Jefferson School

Madera, CA

Sheilah Mann

American Political Science Association

Washington, DC

Members of the NAEP Civics

Standing Committee

he NAEP civics standing committee provided advice on the scope and

content of the NAEP 1998 Civics Report Card for the Nation and the report

on the civics 1988 and 1998 assessments, The Next Generation of Citizens.

James Marran

New Trier Township High School

Winnetka, IL

Mabel McKinney-Browning

American Bar Association

Chicago, IL

Richard Niemi

University of Rochester

Rochester, NY

John Patrick

Indiana University

Bloomington, IN

T

COMMITTEE MEMBERS

NAEP CIVICS ASSESSMENTS–1988 AND 1998 • ACKNOWLEDGMENTS 73

ACKNOWLEDGMENTS

The NAEP project at ETS is directed by Stephen

Lazer and John Mazzeo. Sampling and data

collection activities were conducted by Westat under

the direction of Rene Slobasky, Nancy Caldwell,

Keith Rust, and Dianne Walsh. Printing,

distribution, scoring, and processing activities were

conducted by NCS under the direction of Brad

Thayer, Patrick Bourgeacq, Charles Brungardt,

Mathilde Kennel, Linda Reynolds, and Connie

Smith.

The complex statistical and psychometric

activities necessary to report results for the NAEP

1998 civics trend assessment were directed by

Nancy Allen, John Barone, and Frank Jenkins.

The analyses presented in this report were

conducted by Spence Swinton with assistance

from Laura Jerry, Ed Kulick, and Tanya

Petrovicheva.

The design and production of the report were

overseen by Rod Rudder. Kelly Gibson designed

the cover. Carol Errickson led the considerable

production efforts with assistance from Barbette

Tardugno, Kelly Gibson, and Sharon DavisJohnson. Data checking was done by Wendy

Grigg, Alice Kass, and Barbette Tardugno.

Editorial assistance was provided by Shari

Santapau.

Production of the World Wide Web version of

this report was coordinated by Pat O’Reilly with

assistance from Rick Hasney. Revisions to the

civics page of the NAEP website were produced

by Madeline Goodman, Richard Bohlander,

Jeffrey Jenkins, and Andrew Baird.

Many thanks are due to the numerous

reviewers, both internal and external to NCES

and ETS, who reviewed this publication.

his report is a product of the many

individuals who contributed their

considerable knowledge, experience, and

creativity to the NAEP 1998 civics special study.

The NAEP civics assessment was a collaborative

effort among staff from the National Center for

Education Statistics (NCES) in the U.S.

Department of Education, the National

Assessment Governing Board (NAGB),

Educational Testing Service (ETS), Westat, and

National Computer Systems (NCS).

In addition, the program benefited from the

contributions of hundreds of individuals at the

state and local levels—governors, chief state

school officers, state and district test directors,

state coordinators, and district administrators—

who tirelessly provided their wisdom, experience,

and time. Most importantly, NAEP is grateful to

the thousands of students and hundreds of

teachers and administrators who made the

assessment possible.

The NAEP 1988 and 1998 civics special

study was funded through NCES, in the Office

of Educational Research and Improvement of

the U.S. Department of Education. The Acting

Commissioner of Education Statistics,

Gary W. Phillips, the Associate Commissioner of

Education Assessment, Peggy Carr, and the

NCES staff—Arnold Goldstein, Steve Gorman,

Andrew Kolstad, Marilyn McMillen, Barbara

Marenus, Patricia Dabbs, Carol Johnson, and

Shi-Chang Wu—worked closely and collegially

with the authors to produce this report. The

authors were also provided invaluable advice

and guidance by the members of the

NAGB staff.

T