U.S. DEPARTMENT OF EDUCATION

Evaluation of the Teaching 

American History Program
Evaluation of the Teaching American History Program

Prepared by:

Daniel C. Humphrey

Christopher Chang-Ross

Mary Beth Donnelly

Lauren Hersh

Heidi Skolnik

SRI International

For:

U.S. Department of Education

Office of Planning, Evaluation and Policy Development

2005
This report was prepared for the U.S. Department of Education under Contract Number ED-01-C0-

0059/0003 with SRI International. Dena Gross served as the contracting officer’s representative. The

views expressed herein do not necessarily represent the positions or policies of the Department of

Education. No official endorsement by the U.S. Department of Education is intended or should be

inferred.

U.S. Department of Education

Margaret Spellings

Secretary

Office of Planning, Evaluation and Policy Development

Tom Luce

Assistant Secretary

Policy and Program Studies Service

Alan Ginsburg

Director

Program and Analytic Studies

David Goodwin

Director

September 2005

This report is in the public domain, except for the photograph on the front cover, which is used with

permission and copyright, 2005, Getty Images. Authorization to produce this report in whole or in part is

granted. While permission to reprint this publication is not necessary, the suggested citation is: U.S.

Department of Education, Office of Planning, Evaluation and Policy Development, Policy and Program

Studies Service, Evaluation of the Teaching American History Program, Washington, D.C., 2005.

To order copies of this report, write:

ED Pubs

Education Publications Center

U.S. Department of Education

P. O. Box 1398

Jessup, MD 20794-1398;

via fax, dial (301) 470-1244;

You may also call toll-free: 1-877-433-7827 (1-877-4-ED-PUBS). If 877 service is not yet available in

your area, call 1-800-872-5327 (1-800-USA-LEARN); Those who use a telecommunications device for

the deaf (TDD) or a teletypewriter (TTY), should call 1-800-437-0833.

To order online, point your Internet browser to: www.edpubs.org.

On request, this publication is available in alternate formats, such as Braille, large print, audiotape, or

computer diskette. For more information, please contact the Department’s Alternate Format Center at

(202) 260-9895 or (202) 205-8113.
iii

Contents

List of Exhibits........................................................................................................................ v

Acknowledgments................................................................................................................. vii

Executive Summary ............................................................................................................... ix

Chapter 1 - Introduction......................................................................................................... 1

Origins of the TAH Program............................................................................................... 3

The TAH Program: An Overview........................................................................................ 3

Report Overview................................................................................................................. 4

Chapter 2 - How Are the Projects Organized to Improve the Teaching of

American History? .............................................................................................. 7

Needs Assessments and Weaknesses in Teaching American History ................................... 7

Structure of Project Activities ............................................................................................. 9

Backgrounds and Roles of Project Directors and Professional

Development Providers......................................................................................................10

TAH Project Partnerships...................................................................................................10

Recruiting Teachers for TAH Projects................................................................................11

Conclusion.........................................................................................................................13

Chapter 3 - Did Teaching American History Projects Reach the Districts

and Teachers Most in Need?..............................................................................15

Demographics of the Districts That Received TAH Grants and the Schools

They Served.......................................................................................................................15

Participants in FY 2001 and FY 2002 Projects ...................................................................17

Experience and Credential Status of TAH Participants.......................................................19

TAH Participants’ Training in History ...............................................................................22

Challenges of Recruiting the Least Qualified Teachers.......................................................25

Conclusion.........................................................................................................................26

Chapter 4 - What Is the Nature of Teaching American History

Professional Development? ...............................................................................27

TAH Professional Development Content............................................................................27

TAH Projects and NAEP Historical Periods, Themes, and Skills........................................29

TAH Projects and the Characteristics of Effective Professional Development ....................34
iv

TAH Projects’ Follow-up Activities...................................................................................35

The Challenge of Creating Effective Professional Development.........................................36

Conclusion.........................................................................................................................37

Chapter 5 - What the Evaluators Learned about the Effectiveness

of TAH Projects.................................................................................................39

Internal TAH Project Evaluations.......................................................................................39

Project Directors’ and Participants’ Reports on TAH Project Effectiveness........................41

Review of Teacher Materials: An Exploratory Sub-study ...................................................42

Results of the Exploratory Study of Teacher Materials.......................................................44

Conclusion.........................................................................................................................47

Chapter 6 - Conclusions.........................................................................................................49

Implications for the TAH Projects......................................................................................50

Policy Implications for the TAH Program ..........................................................................51

References ..............................................................................................................................53

Appendix A

Additional Information.................................................................................................... A-1

Appendix B

Methodology, Data Collection Strategies, and Procedures................................................B-1

Appendix C

Teaching American History Project Director Survey ........................................................C-1

Appendix D

Teaching American History Project Participant Survey ................................................... D-1

Appendix E

Teaching American History Project Materials Request Form............................................E-1

Appendix F

Teaching American History Project NAEP-Based Materials Review Rubric .................... F-1
v

Exhibits

Exhibit E-1 TAH Activities Reported by Project Directors............................................. xi

Exhibit E-2 NAEP Themes Addressed by TAH Projects............................................... xii

Exhibit E-3 Historical Methods Addressed by TAH Projects........................................ xiii

Exhibit E-4 Elements of Project Evaluations................................................................ xiv

Exhibit E-5 TAH Project Directors’ Reports on Improvement

in Selected Areas ........................................................................................xv

Exhibit E-6 Teacher Reports of the Contribution Made by Their TAH Project ...............xv

Exhibit 1-1 NAEP American History Assessment Results, 1994 and 2001...................... 1

Exhibit 1-2 TAH Program Appropriation Statistics for FY 2001

and FY 2002 ................................................................................................ 4

Exhibit 2-1 How School Districts Participating in the TAH Program

Determined Their Needs for the Grant.......................................................... 7

Exhibit 2-2 Needs of Participating School District(s) that TAH Projects

Aimed to Address ........................................................................................ 8

Exhibit 2-3 TAH Activities Reported by Project Directors.............................................. 9

Exhibit 2-4 Varied Approaches to Collaboration by TAH Projects ................................11

Exhibit 2-5 Reasons for Participating in a TAH Project .................................................12

Exhibit 2-6 The Five Most Common Incentives for Participation in

TAH Projects, as Reported by Project Directors and Participants ................13

Exhibit 3-1 Key Demographics for Students In Grantee Districts, Compared

With National Data .....................................................................................16

Exhibit 3-2 Total Participants in TAH Projects for Grants Awarded in

FY 2001 and FY 2002.................................................................................18

Exhibit 3-3 School Levels Taught by Participants, 2002-03...........................................19

Exhibit 3-4 Years of Teaching Experience Reported by TAH Participants.....................20

Exhibit 3-5 The Ten Subject Areas in Which TAH Participants Most

Frequently Reported Certification ...............................................................21

Exhibit 3-6 Majors and Minors in History Held by TAH Participants

and by Teachers Nationally.........................................................................22

Exhibit 3-7 Participants’ Reports of Professional Development Opportunities

Outside TAH Projects.................................................................................23
vi

Exhibit 3-8 Number and Percentage of Out-of-Field and Underprepared

High School California Teachers Assigned to Teach

Social Studies, 2003-04...............................................................................25

Exhibit 4-1 History and Social Studies Standards Consulted During Planning of

TAH Projects .............................................................................................28

Exhibit 4-2 NAEP Chronological Periods Addressed by TAH Projects..........................29

Exhibit 4-3 NAEP Themes Addressed by TAH Projects................................................30

Exhibit 4-4 NAEP “Ways of Knowing and Thinking About U.S. History”

Addressed by TAH Projects........................................................................31

Exhibit 4-5 Incorporating Historical Thinking Skills......................................................32

Exhibit 4-6 Historical Methods Addressed by TAH Projects..........................................33

Exhibit 4-7 Using Historical Methods............................................................................34

Exhibit 4-8 Follow-up Offered by TAH Projects ...........................................................36

Exhibit 5-1 Elements of TAH Project Evaluations.........................................................39

Exhibit 5-2 TAH Project Directors’ Reports on Improvement in Selected Areas............41

Exhibit 5-3 Teacher Reports of Contribution of Their TAH Project ...............................42

Exhibit 5-4 Sample from the TAH NAEP-based Materials Review Rubric ....................43

Exhibit 5-5 Average Score by NAEP-based Historical Skills ........................................46
vii

Acknowledgments

The authors wish to thank the many individuals who assisted in this evaluation.

SRI International: Nancy Adelman, Lee Anderson, Katherine Baisden, Vanessa Barrat,

Deborah DuCharme, Stacey Eaton, Bonnee Groover, Heather Hough, Ruth Krasnow,

Amy Lewis, Bethany Martinez, Cimone Satele, and Michael Smith.

U.S. Department of Education: Dena P. Gross, Elizabeth Eisner, Alan Ginsburg,

David Goodwin, Daphne Kaplan, and Stacy Kreppel.

Technical Work Group: Thomas Adams, Maria Mazzenga, Jeffrey Mirel,

Cynthia Mostoller, James A. Percoco, Theodore K. Rabb, Jeffrey Smith, and William White.

Materials Review Panel: Thomas Adams, Ralph Giannini, Bill Hanna, Randal Mitchell,

Chauncey Monte-Sano, Michael Stachon, Vivian White, and William Wilhelm-Safian.

And finally, our deep appreciation to the project directors, participants, and training providers

who participated in the study’s data collection efforts.

ix

Executive Summary

Nearly 20 years ago, the first national assessment of student achievement in U.S. history

yielded disappointing results. Although policy-makers and researchers expressed great concern

about the low scores, the federal government did not undertake large-scale efforts to address

poor student performance, and few research dollars were dedicated to uncovering the causes of

the problem. In 2001, after the release of another report describing the woeful state of history

education, Congress acted, charging the Department of Education with creating the Teaching

American History (TAH) Program to improve teacher content knowledge of and instructional

strategies for U.S. history. In its first two years, the program’s total funding increased from $50

million to $100 million and grants were awarded to 174 local districts that proposed to serve a

total of 24,000 teachers.

During this time period, the TAH program found a receptive audience and appeared to be

providing the resources needed to meet its stated goals. The evaluation of the 2001 and 2002

grantee cohorts indicates, however, that the projects may not have reached those teachers

typically considered most in need of additional professional development, and that the training

provided did not always match research-based definitions of effective professional development.

The following executive summary provides key findings from this evaluation, which

examined the implementation of the program and characteristics of the activities, content, and

teacher participants for TAH projects awarded during the first two years of the program. The

findings are based on surveys of participants and project directors, case studies, extant

documents, and a pilot study of teacher-produced lesson plans.

_ Key Findings

Participants

_ TAH grants funded projects in districts with high-need student populations.

The grants went to districts that served large numbers of high-need students. Generally

speaking, students in grantee districts were more likely to be from minority backgrounds,

limited in English proficiency, and eligible for free and reduced-price lunches than

students nationally, according to the Common Core of Data.

_ The teachers who participated in TAH projects were often not those traditionally

thought of as most in need of history professional development. Participants were

most likely to be experienced secondary teachers (70 percent) with academic

backgrounds in history. TAH project participants averaged 14 years’ teaching

experience—the same as the national average reported in the 1999-2000 Schools and

Staffing Survey. Almost all TAH teachers were certified (97 percent), although not

necessarily in history. Compared with a national sample of secondary teachers who teach

mostly social studies, TAH participants were more likely to have history degrees

(38 percent vs. 30 percent) and far more likely to have either a major or a minor in

history (61 percent vs. 37 percent). TAH participants also reported that they had
x

completed numerous college-level American history courses—between one and five

courses for 49 percent of participants; between six and 10 courses for another 26 percent;

and more than 10 courses for 22 percent. These findings, coupled with the fact that many

teachers voluntarily participated in time-intensive TAH projects, suggest that TAH

projects likely reached those teachers most interested in American history—not

necessarily those most in need of additional professional development.

_ TAH projects partnered with a wide range of organizations and institutions.

Although the law requires that grantees partner with at least one institution, projects have

the flexibility to choose partners that will best suit their needs.1 The partners were

instrumental in providing teachers with historical expertise, planning for professional

development, working with teachers to design lessons and unit plans for classroom use,

and providing teachers with historical materials and resources. Although projects

experienced varying degrees of success, the participation of historians appeared to be

key to successful projects.

Services

_ Summer institutes were the professional development activity offered most

frequently. Intensive summer institutes were key among the array of training

opportunities projects offered to meet grantees’ and participants’ needs (see Exhibit E-1).

Institute activities averaged 10 days in length and were attended by 76 percent of TAH

participants.2

 1 In accordance with the No Child Left Behind Act, the law authorizing the TAH Program, each grantee was

required to partner with at least one of the following institutions: (1) an institution of higher education, (2) a

nonprofit history or humanities organization, or (3) museum or library. 2 Source: SRI International, TAH Participant Survey.
xi

Exhibit E-1

TAH Activities Reported by Project Directors

10

32

57

81

90

0%

20%

40%

60%

80%

100%

Summer

institute

Workshop Conference/

symposium

On-campus

college or

university

course

Distance

learning

course

Source: SRI International, TAH Project Director Survey.

Exhibit reads: Ninety percent of all project directors reported summer

institutes as an activity offered through their TAH projects.

_ Projects covered a wide range of historical content, thinking skills, and methods.

TAH projects exposed participants to historical content from early America to the

present. The American Revolution and the Civil War were the most frequently covered

periods in TAH projects, and the four NAEP U.S. History Framework3 themes were

covered during TAH professional development (see Exhibit E-2). TAH projects also

exposed participants to the historical thinking skills outlined in “Ways of Knowing and

Thinking about U.S. History,” the cognitive dimension of the NAEP Framework. The

projects also exposed participants to a variety of historical methods (see Exhibit E-3).

 3 National Assessment Governing Board (NAGB), U.S. Department of Education. (2001). U.S. history framework

for the 1994 and 2001 National Assessment of Educational Progress. Washington, D.C.: Author.
xii

Exhibit E-2

NAEP Themes Addressed by TAH Projects

58

69

83

91

0% 20% 40% 60% 80% 100%

The gathering and interactions of peoples,

cultures, and ideas 

Change and continuity in American democracy:

ideas, institutions, practices, and controversies

Economic and technological changes and their

relation to society, ideas, and the environment

The changing role of America in the world 

Source: SRI International, TAH Project Director Survey.

Exhibit reads: Ninety-one percent of project directors reported that their projects

addressed the NAEP theme entitled “gathering of interactions, peoples, cultures, and

ideas.”
xiii

Exhibit E-3

Historical Methods Addressed by TAH Projects

94

66

63

63

50

86

86

0% 20% 40% 60% 80% 100%

Analyzing historical documents, such as

manuscripts and diaries 

Analyzing history by themes, periods, and regions 

Analyzing historical artifacts, material culture,

and/or media (i.e. video, Internet, music) 

Analyzing the historical significance of place 

Comparing and contrasting differing interpretations

of history and historical events 

Forming hypotheses and making conclusions

based on historical evidence 

Analyzing oral histories 

Source: SRI International, TAH Project Director Survey.

Exhibit reads: Ninety-four percent of project directors reported that participants in their

TAH projects engaged in analyzing historical documents, such as manuscripts and diaries.

Quality and Effectiveness

_ TAH project activities display some, but not all, of the research-based

characteristics of effective professional development. Project directors’ and

participants’ reports suggest that TAH professional development offered active learning,

promoted coherence, and encouraged professional communication; however, generally

speaking, the use of traditional training formats hampered most projects’ ability to offer

other characteristics of research-based, high-quality professional development. Followup activities also fell short of meeting teachers’ classroom needs. Although 92 percent of

project directors reported that project activities were sustained through follow-ups, only

31 percent of participants reported that a TAH project representative visited their

classroom more than once. Ten percent indicated that a representative visited their

classroom four or more times. Other forms of follow-up included dissemination of

additional materials (reported by 73 percent of participants); additional workshops,

training sessions, and meetings (71 percent); mailings (69 percent); and e-mail

(84 percent).
xiv

_ Internal evaluations lacked the rigor to measure projects’ effectiveness accurately.

Teacher self-reports were relied on by 91 percent of project directors to assess project

professional development activities. Substantially fewer directors reported using other

evaluation methods like analyzing the work products of teacher participants (64 percent),

observing the classrooms of teacher participants (48 percent), or analyzing student or

participant content knowledge in American history (46 percent and 41 percent,

respectively) (see Exhibit E-4).

Exhibit E-4

Elements of Project Evaluations

91

82

65

64

48

46

41

25

0% 20% 40% 60% 80% 100%

Teacher self-reports

Observations of project activities

Participant focus groups 

Analysis of participant work products

Classroom observations

Analysis of students' knowledge 

Analysis of partcipants' knowledge 

Analysis of student work 

Source: SRI International, TAH Project Director Survey.

Exhibit reads: Ninety-one percent of project directors reported that their TAH

project evaluations included teacher self-reports.

_ Project directors and participants reported positively on the effectiveness and

quality of TAH projects. Project directors reported that participants improved their

interest in teaching American history a great deal (44 percent) and that their content

knowledge of American history substantially improved (50 percent); 29 percent indicated

that student performance improved a great deal or substantially. Many participants also

strongly agreed that the projects increased their ability to use historical methods

(59 percent) and taught them to use historical resources (47 percent). (See Exhibits E-5

and E-6).
xv

Exhibit E-5

TAH Project Directors’ Reports on Improvement in Selected Areas

A Great Deal Substantial Moderate Some Little or no

Teachers' content knowledge 17% 50% 25% 7% 0%

Teachers' knowledge of

instruction 18% 34% 34% 12% 2%

Teachers' interest in

American history 44% 39% 10% 6% 0%

Student performance 10% 19% 29% 34% 7%

Student interest 12% 39% 23% 23% 4%

Source: SRI International, TAH Project Director Survey.

Exhibit reads: Sixty-seven percent of project directors reported “a great deal” or a “substantial”

amount of improvement in teachers’ content knowledge.

Exhibit E-6

Teacher Reports of the Contribution Made by Their TAH Project

Strongly

Disagree

(1) (2) (3) (4)

Strongly

Agree (5)

Not

Applicable

Increased ability to use historical

methods 2% 3% 11% 23% 59% 2%

Can better assess students in

American history 3% 9% 20% 23% 38% 7%

Taught me to use historical

resources

4% 6% 17% 23% 47% 3%

Consistent with state/district

standards 2% 3% 7% 19% 67% 3%

Source: SRI International, TAH Participant Survey.

Exhibit reads: Fifty-nine percent of participants reported that they strongly agree that their

participation in the TAH resulted in an increased ability to use historical methods.

_ While TAH teacher work products demonstrated teachers’ knowledge of facts, they

also revealed participants’ limited ability to analyze and interpret historical data.

Findings from the exploratory study of teacher work products (lesson plans and research

papers) indicated that while teachers had a firm grasp of historical facts and some lowerlevel historical thinking skills, they had difficulty interpreting and analyzing historical

information. Although the teacher work products reviewed ranged in quality, nearly all

products earned low scores on historical analysis and interpretation.
xvi

Coordination and Support

_ TAH projects were not well-integrated with other local, state, and federal teacher

development initiatives. The first two cohorts of TAH projects were not primarily

organized to help meet No Child Left Behind’s goal of providing a highly qualified

teacher for every student. In addition, TAH projects were not always well-integrated

with state and local teacher development strategies and programs. The extra time, effort,

and energy required to participate in TAH may have discouraged teachers who needed to

meet specific professional development requirements to attain or maintain their teaching

certification from participating in the program.

_ A lack of research on effective professional development for American history

teachers made project directors’ decisions about professional development

somewhat subjective. The lack of definitive research on professional development in

American history left project directors to use their best judgment to plan activities that

they believed would increase teacher knowledge and improve student achievement.

_ Avenues for projects to disseminate and share their materials were insufficient.

In seeking to share the materials and experiences of TAH participants with wider

audiences, projects encountered technical and logistical obstacles. Ultimately, there was

no systematic way to ensure that exemplary lessons and materials were distributed to a

wide audience.

_ Expanding TAH project grants to include the preparation of prospective American

history teachers could result in a more comprehensive approach to the improvement

of American history teaching and learning. TAH projects demonstrated the

effectiveness of having practicing historians provide professional development to

experienced teachers. Enlisting colleges and universities that provide preservice history

teacher training could help ensure that beginning teachers enter the profession already

prepared to teach U.S. history.
1

Chapter 1

Introduction

Since the release of the disappointing results of the first U.S. History National Assessment of

Educational Progress (NAEP) examination in 1986, educators and policy-makers have expressed

grave concerns about poor student achievement in American history. What caught the attention

of many commentators was students’ inability to recall even the most basic facts about the

American historical record or to demonstrate higher order historical thinking skills (Rothman,

1987). Substantive improvements did not occur when the examination was administered again in

1988, 1994, and 2001. In 2001, although some progress had been made over 1994’s results,

overall scores remained low (see Exhibit 1-1). In the 4th, 8th, and 12th grades, fewer than

20 percent of students scored at or above the proficient level. Fourth- and eighth-graders

performed somewhat better in 2001 than in 1994, but the results for 12th-graders did not change

(Lapp et al., 2002). Equally distressing has been the persistent gap in achievement on the NAEP

examination among racial groups, rural students, and low-income test-takers. (Appendix A

provides more information about the NAEP examination and the disparity in scores among racial

groups).

Exhibit 1-1

NAEP American History Assessment Results, 1994 and 2001

Percent of Students at Each Achievement Level

Below Basic Basic Proficient Advanced

1994 2001 1994 2001 1994 2001 1994 2001

Grade 4 36 33 47 49 15 16 2 2

Grade 8 39 36 48 48 13 15 1 2

Grade 12 57 57 32 32 10 10 1 1

Source: Livingston and Wirt (2003).

Exhibit reads: Fifty-seven percent of students in grade 12 scored below the basic achievement

level in 1994 and 2001.

In speculating about the causes of the poor performance, some observers blamed poor

instruction, others targeted inadequate teacher preparation, and still others argued that politics

and activists had watered down the curriculum and sidelined traditional accounts of the past in

favor of more inclusive and multicultural ones. Much of their speculation was grounded in

conjecture, however. Despite the widespread interest over the last 20 years in teaching American

history, the research base on the subject remains sparse. Although a review of the literature

uncovered a substantial amount of material concerning lesson plans, program descriptions, and
2

personal narratives, comparatively few peer-reviewed studies address best practices for teaching

and learning how to teach American history. Policy priorities at the state and federal level have

left history education and research underfunded relative to investments in mathematics and

science instruction (VanSledright, 2002). In addition, history has struggled to redefine its

position in the social studies model (Leming, Ellington, and Porter, 2003).

In fact, researchers and educators have yet to determine the most effective teaching strategies

or professional development approaches for teachers of American history. Most available

studies are small in scale and lack generalizability; nonetheless, they do highlight interesting

themes (Wilson, Floden, and Ferrini-Mundy, 2001). The literature on history teacher

preparation, for example, indicates that teachers do not know how to practice the discipline and,

lacking that capacity, cannot pass critical knowledge and skills on to their students (Bohan

Davis, 1998; Seixas, 1998; Slekar, 1998; Vinten-Johansen and McDiarmid, 1997). Research on

teaching American history mostly concentrates on defining “good” teaching and indicating the

impacts that external factors, like standards and high-stakes assessments, have on instruction

(Brophy, 1992; Brophy and VanSledright, 1997; Evans, 1990; Grant, 2001; Hartzler-Miller,

2001; Leinhardt, 1997; Newmann et al., 1990; Smith and Niemi, 2001; Thornton, 1988; Wilson,

2001; Wineburg and Wilson, 2001a, 2001b, and 2001c). Such research lacks sufficient

explanatory power, however, and few studies have attempted to connect history instruction to

student learning (Wilson, 2001). Lastly, research on effective professional development

approaches for history teachers has been minimal—primarily consisting of research on one statefunded program (Kroesch and Edwards, 2000; Medina et al., 2000; Podany, 2000;

St. John et al., 1999; Seixas, 1999).

Although research has proceeded at a slow pace, nearly every state has undertaken limited

measures to develop student academic standards in history or social studies. These are positive

steps, but have been undermined by uneven quality and a weak focus on history (American

Federation of Teachers, 2001; Finn and Petrilli, 2000; Saxe, 1998; Stern, 2003). Moreover,

attempts to produce a national set of history standards have encountered fiercely partisan

disagreements about what their content should be (Nash, Dunn, and Crabtree, 2000). Fewer than

half of all states conduct history or social studies assessments, and only half of that number has

aligned their tests with their own states’ standards (Education Week, 2003).

Although research in the area of history instruction is limited, education studies in other

subjects have demonstrated the connection between higher student achievement and skilled,

high-quality teachers (Darling-Hammond, 1997; Ferguson, 1998; Haycock, 1998; Hirsch,
3

Koppich, and Knapp, 1998; National Commission on Teaching and America’s Future, 1996).

Logically, then, increasing the content knowledge and skill level of American history teachers

should provide the basis for improving student understanding of, and achievement in, U.S.

history. That rationale undergirds the premise, purpose, and structure of the Department of

Education’s Teaching American History (TAH) program—the most significant federal

investment in the teaching of U.S. history.

_ Origins of the TAH Program

The impetus for the creation of the TAH program derived from one study and reactions by

legislators and the media to a 2000 report on the weak historical knowledge of graduating seniors

at elite colleges and universities (American Council of Trustees and Alumni, 2000). In reaction

to the report, the House and Senate adopted a concurrent resolution “regarding the importance

and value of education in United States history.” Senator Robert Byrd of West Virginia, one of

the primary proponents of the resolution, then sponsored legislation to provide $50 million to the

Department of Education to improve the teaching of traditional American history (P.L. 106-554).

_ The TAH Program: An Overview

In 2001, Congress established the program to support teaching American history, improve

instruction in the subject, and provide high-quality professional development for teachers. The

competitive grant program provides funds to local school districts for developing and operating

three-year professional development projects. As part of the program, grantees are required to

partner with one or more institutions of higher education, nonprofit history or humanities

organizations, or libraries or museums. These entities are expected to provide content expertise

and instructional support in American history to the grantees. The funded projects are expected

to demonstrate the components indicated in research on best instructional practices and support

the development of critical skills necessary for teaching to higher standards. In addition, the

projects are expected to have a long-term plan, be evaluated in regard to the degree to which they

improve teacher effectiveness and student achievement, and demonstrate how school districts

and institutions with content expertise could collaborate to teach American history better. The

Department of Education plans to use some of the products and knowledge developed and

disseminate them to a broader audience.

The application notice for the Teaching American History program’s first year, FY 2001,

stated that the program was “to raise student achievement by improving teachers' knowledge,

understanding, and appreciation of American history.” In FY 2001, 60 local education agencies

received TAH grants. Although the grants averaged about $825,000, the amounts awarded to
4

districts varied (see Exhibit 1-2). During FY 2002, the program’s appropriation and number of

grantees doubled to $100 million and 114 districts. The average size of the grants also grew, and

the range of the grant amounts increased.

Exhibit 1-2

TAH Program Appropriation Statistics for FY 2001 and FY 2002

FY 2001 FY 2002

Total annual appropriation $50,000,000 $100,000,000

Total number of school districts awarded grants 60 114

Grant minimum $386,762 $19,561

Grant maximum $1,000,000 $1,000,000

Average (mean) grant size $827,276 $867,903

Source: U.S. Department of Education, Teaching American History Grant Abstracts.

Exhibit reads: In FY 2001, the total annual appropriation for the TAH program was $50 million.

_ Report Overview

This report is designed to answer three broad groups of research study questions: the types of

activities TAH grantees are implementing; the content of the activities, including the specific

subjects and areas of American history on which projects are focusing; and the characteristics

and qualifications of teachers participating in the activities. (See Appendix A for complete list of

research questions and data sources.) The report does not assess the impact of the TAH projects

on student or teacher learning, but it does offer insights into the progress of the projects toward

the improvement of teaching American history. This report draws on multiple data sources

(project director and participant surveys, case studies of eight TAH projects, an exploratory

analysis of teacher work products, and an extensive document review) to describe both the

challenges and accomplishments of the 174 TAH projects from the 2001 and 2002 cohorts of

grantees. The evaluation was conducted from October 2002 until May 2005.

This report is organized by key findings and is laid out as follows: Chapter 2 indicates how

the projects were organized, and provides an overview of their activities, partners, and project

directors. Chapter 3 discusses the extent to which the grantee districts and teachers met the

Department of Education’s goal for improving American history teaching among the nation’s

least qualified teachers serving the neediest students. Chapter 4 examines the professional

development that project participants received and compares it with research-based definitions of
5

high-quality professional development. Chapter 5 describes the available evidence on the

effectiveness of TAH projects to improve teachers’ content knowledge of American history and

their historical thinking skills. Chapter 6 concludes the report by highlighting the implications of

the research findings for the individual projects and the TAH program as a whole.

7

Chapter 2

How Are the Projects Organized to Improve the Teaching of American History?

Grantees organized Teaching American History (TAH) projects to address many of the

factors that contribute to poor student performance in American history. In preparation for

writing their proposals, school districts and partner organizations identified project leaders,

conducted needs assessments, and planned activities they believed would improve history

instruction and student achievement. Typically, the projects offer intensive summer institutes or

short-term professional development activities, along with opportunities for participants to

engage in follow-up. Chapter 2 examines how the grantees identified their needs and how

project staff and providers sought to improve the teaching of American history.

_ Needs Assessments and Weaknesses in Teaching American History

The Department of Education issued the first TAH application notice on May 23, 2001, and

the application notice for the second cohort of grants on April 3, 2002. Potential grantees had

two months to prepare proposals. Nearly all of the project directors reported that they conducted

a needs assessment, although the short submission timeline probably precluded an extensive

examination of needs. Project directors reported that the grantees undertook a variety of needs

assessments (see Exhibit 2-1).

Exhibit 2-1

How School Districts Participating in the TAH Program

Determined Their Needs for the Grant

(n = 139)

Type of Needs Assessment

Project Directors

Responding “Yes”

(percent)

An analysis of the availability and quality of professional

development for teachers of American history 61

An analysis of student performance in American history on

standardized tests 59

Teacher surveys regarding the need for training in American history

to address state, district, or school standards 55

An analysis of teaching credentials and content knowledge for

teachers of American history 54

Information supplied by the history/social studies coordinator to the

superintendent about needs in the teaching of American history 45

Source: SRI International, TAH Project Director Survey.

Exhibit reads: Sixty-one percent of project directors reported that they conducted an analysis of

the availability and quality of professional development for teachers of American history as a

way of determining their need for a TAH grant.
8

Project directors reported that their needs assessments identified a variety of weaknesses in

the teaching of American history (see Exhibit 2-2). Not surprisingly, all project directors

reported that their TAH projects aimed to address the need to improve teachers’ content

knowledge in American history. Some projects identified other needs, including increasing the

use of technology and addressing students’ weak literacy skills.

Exhibit 2-2

Needs of Participating School District(s) that TAH Projects Aimed to Address

(n = 149)

Need of Participating School District(s)

Project Directors

Responding “Yes”

(percent)

Increase teachers’ content knowledge in American history 100

Increase opportunities for, and quality of, collaboration among teachers of

American history 89

Develop partnerships in American history with a college or university,

museum, library/archive, and/or other historical organization 88

Increase teachers’ knowledge of historical methods 87

Improve academic performance of students who are low performing in

American history 83

Develop best practices for teaching American history 83

Increase the instructional use of technology by teachers of American history 75

Develop district resources for professional development in American history 69

Increase district access to historical resources 66

Improve literacy skills of students through American history 53

Have teachers of American history meet the definition of highly qualified, as

specified by the No Child Left Behind Act 52

Source: SRI International, TAH Project Director Survey.

Exhibit reads: All project directors reported that their TAH project addressed the need to increase

teachers’ content knowledge in American history.

Although only 52 percent of project directors reported that their TAH projects sought to

address teachers’ need to meet the definition of highly qualified under the No Child Left Behind

Act, that relatively low percentage probably reflects the timing of the law’s adoption relative to

the application period of each cohort.
9

_ Structure of Project Activities

Summer institutes were the most common form of TAH professional development (see

Exhibit 2-3), with 90 percent of all project directors reporting that this activity was offered. As

the project directors’ responses suggest, most projects engaged in a variety of activities. Ninetyfour percent of project directors reported that their TAH project activities were “sustained over

time, with ample follow-up activities and experiences.” Typically, participants attended a

summer institute for one or two weeks and then were offered occasional follow-up opportunities

(e.g., workshops, colloquia) that were not always directly focused on the teachers’ work in their

classrooms during the school year. These professional development activities most frequently

took place on a university campus (62 percent), at a historical site (46 percent), or at a museum

(43 percent).

Exhibit 2-3

TAH Activities Reported by Project Directors

(n = 150)

10

32

57

81

90

0%

20%

40%

60%

80%

100%

Summer

institute

Workshop Conference/

symposium

On-campus

college or

university

course

Distance

learning

course

SRI International, TAH Project Director Survey.

Exhibit reads: Ninety percent of all project directors reported that

activities offered through their TAH projects included summer institutes.

Summer institutes averaged 61 hours, or about a week and a half, in length; other activities

lasted from two days to just over a week.
10

_ Backgrounds and Roles of Project Directors and Professional Development

Providers

Most project directors (75 percent) were employed by the school district that received the

grant. Directors typically worked for the districts as curriculum experts (40 percent),

professional development specialists (26 percent), or administrators (26 percent), or in some

combination of those roles. Seventeen percent of project directors were regular classroom

teachers—often heads of their schools’ history or social studies departments. Of those not

employed by the school district, most worked for a college or university (58 percent).

Project directors reported a range of responsibilities, although nearly all indicated that their

primary functions involved “organiz[ing]/coordinate[ing] logistical information and other

institutional support for professional development activities,” and “develop[ing] or select[ing]

the professional development plan and methods of delivery.” Sixty-two percent of project

directors also led the professional development activities in TAH projects.

University faculty also assumed significant roles in the training of teachers. Ninety-eight

percent of project directors, for example, reported that university history faculty led professional

development activities. Project directors also reported that museum educators, master teachers,

representatives from historical associations, university education department faculty, and

curriculum experts also led professional development activities but were far less likely to do so

than university history professors.

_ TAH Project Partnerships

The central role of university faculty in delivering professional development to TAH project

participants derived from the grant’s requirement for grantees to partner with one or more

institutions of higher education, nonprofit history or humanities organizations, or libraries or

museums. Project directors reported that partners were most frequently responsible for leading

and instructing participants, designing plans and approaches for professional development,

working with teachers to design curriculum (e.g., lesson and unit plans) for classroom use, and

producing materials for teachers’ use. Project directors also reported that although nonuniversity partners were slightly less likely to lead professional development activities than were

university faculty, all partners played significant roles in the development and implementation of

the TAH projects.
11

Exhibit 2-4

Varied Approaches to Collaboration by TAH Projects

In one TAH case study, the three primary partners collaborated to ensure that the project functioned

smoothly and met the institutional goals of each member. Initially, two of the partners had been

planning to submit separate grant applications, but on learning that both shared a similar vision for

helping develop history teachers, they decided to work together. Although each partner played distinct

roles in the project, they discussed all significant decisions about summer institutes and follow-up with

all stakeholders. “We meet once a week, and we think out loud,” the co-project director explained.

“We don’t just talk about our piece. Everybody has a voice.” The historian of the project drove the

content of the professional development provided, but all of the partners contributed to decisions

about which historical topics would be covered. After the content of the courses had been chosen, the

codirectors determined how the material was to be presented to the teachers. This approach not only

ensured open communication among all partners but also helped the project provide a more coherent

professional development experience for participants.

In another case study site, meanwhile, the university partner offered courses that were too intensive,

controversial, and, according to the project director, not aligned with the content the state held

participants responsible for in their classrooms. As a result, this grantee severed its official

partnership with the history department and contracted with individual professors in the department to

meet teachers’ needs.

_ Recruiting Teachers for TAH Projects

TAH projects employed a variety of methods to recruit teachers. Some districts distributed

flyers, communicated with school administrators, or promoted their projects at education

conferences. Although some had a formal application process, others used more informal means

of recruiting, such as allowing the partnering districts to choose who would attend. Other

projects had to conform to the standards of the partnering institutions for selecting applicants.

For example, one project had to ensure that participants met the criteria of the partner

university’s admissions office.

Generally speaking, participants reported that they voluntarily participated in a TAH project;

in fact, only 6 percent reported that their school or district administrators required them to do so.

By far, the most common reason for participating in a TAH project was the teachers’ desire to

improve content knowledge and teaching skills in American history (see Exhibit 2-5).
12

Exhibit 2-5

Reasons for Participating in a TAH Project

(Weighted n = 7,774; unweighted n = 976)

Reason Cited by Participants Percent

Wanted to improve content knowledge/teaching skills in American history 89

Was interested in learning more about American history 79

Was seeking opportunities for teacher collaboration 53

Wanted to maintain and/or improve qualifications for current teaching position 36

Wanted to improve use of technology in the classroom 31

Was teaching low-performing students 20

Wanted to become a master teacher who trains others 15

Wanted to be trained to teach American history 8

Wanted to acquire degree/certification in American history 6

Was teaching at a school identified as low-performing 6

Was required to do so by my school site or district administrators 6

Wanted to be trained to teach American history at a different grade level 4

Source: SRI International, TAH Project Participant Survey.

Exhibit reads: Eighty-nine percent of participants reported that they participated in a TAH

project because they wanted to improve their content knowledge/teaching skills in American

history.

Aside from appealing to the internal motivation of participants, project directors also used

multiple incentives to recruit participants (see Exhibit 2-6).4 As the case studies revealed,

incentives often were needed—in some cases, were added after the fact—to solicit and increase

participation.

 4 Other incentives reported by project directors included: release time from classes or other responsibilities

(53 percent); travel opportunity (49 percent); full or partial reimbursement for expenses incurred (48 percent);

membership in a professional organization (34 percent); tuition (35 percent); and scheduled time in contract for

professional development (27 percent). Fewer than 10 percent of all project directors reported a salary increment

or pay increase, initial certification, or additional certification endorsement.
13

Exhibit 2-6

The Five Most Common Incentives for Participation in TAH Projects,

as Reported by Project Directors and Participants

(Project directors: n = 148)

(Participants: weighted, n = 7,500; unweighted, n = 934)

49

59

75

89

95

25 22

45

66

75

0%

20%

40%

60%

80%

100%

Stipend Additional

teacher

resources

Credit toward

recertification

or continuing

education

Leadership

opportunity

College credit

or degree

Reported by Project Directors Reported by Participants

Source: SRI International, TAH Project Director Survey; TAH Project

Participant Survey.

Exhibit reads: Ninety-five percent of project directors (compared with

75 percent of participants) reported stipends were used as an incentive for

participation in a TAH project.

Despite the variations between the project director and participant reports, financial

incentives played a significant role in participant recruitment. These payments, however,

consumed large portions of the projects’ budgets. Among the eight case study projects, for

example, directors devoted as much as one-third of their budgets to financial incentives for

participants.

_ Conclusion

The TAH projects’ efforts to help improve teaching and learning of American history appear

to have been well-organized. The grantees conducted needs assessments, albeit in a short period,

with the weaknesses they identified in their districts mirroring research findings about teaching

American history. The projects planned and organized professional development activities,

typically employing summer institutes lasting more than a week with follow-up activities.
14

In addition, project directors and professional development providers appeared to be wellqualified to meet their responsibilities. Projects also employed a variety of recruitment strategies

and used financial and other incentives to attract teachers.

The projects faced significant challenges in providing training to the teachers most in need of

support, in offering professional development consistent with research-based characteristics of

effective professional development, in providing evidence of their accomplishments, and in

achieving positive outcomes and impacts. Chapters 3 and 4 examine these aspects of the projects

more closely.
15

Chapter 3

Did Teaching American History Projects

Reach the Districts and Teachers Most in Need?

The invitational priority in the Department of Education’s FY 2001 and FY 2002 requests for

proposals for the Teaching American History (TAH) program state that “the Secretary is

particularly interested in receiving applications from high-poverty rural and urban” school

districts to improve “instruction in chronically low-performing schools.” Implicit in that

statement is that TAH projects should also serve those teachers most in need of improving their

instruction in and knowledge of American history. Chapter 3 examines the characteristics of

TAH grantee districts and the qualifications of TAH participants to determine how well the

projects met that objective.

_ Demographics of the Districts That Received TAH Grants and the Schools They

Served

Demographic information for TAH grantee districts suggests that the awards were consistent

with the invitational priority. The Department of Education awarded TAH grants in FY 2001

and FY 2002 to districts that, when compared with national data, had higher proportions of

minority students, of students eligible for free and reduced-priced lunches, and of students

classified as English-language learners. (See Exhibit 3-1, which presents demographic

information from the Common Core of Data for grantees, compared to national figures.)
16

Exhibit 3-1

Key Demographics for Students In Grantee Districts,

Compared With National Dataa

22

73

52

17

77

62

8

39 37

0%

20%

40%

60%

80%

Eligible for Free and Reducedprice lunchesMinority Limited English Proficient

(LEP)

TAH grantees for FY 2001 TAH grantees for FY 2002 National demographic data 

Source: U.S. Department of Education, National Center for Education Statistics,

Common Core of Data.

Exhibit reads: Fifty-two percent of students in TAH grantee districts in FY 2001 and

62 percent of students in TAH grantee districts in FY 2002 qualified for free and

reduced-priced lunches. These figures exceed the national average of 37 percent.

a Because Exhibit 3-1 is based on demographic information only for those districts for which CCD data

was available, results should be interpreted with caution. In FY 2001, of the 60 districts awarded TAH

grants, English-language learner data were available for 49 grantee districts; free and reduced-price

lunch data were available for 51 districts; and data on minorities were available for 51 districts. In FY

2002, of the 114 districts awarded TAH grants, LEP data were available for 100 grantee districts; free

and reduced-price lunch data were available for 91 districts; and minority data were available for 95

districts. National demographic data were compiled from the National Center for Education Statistics’

Overview of Public Elementary and Secondary Schools and Districts: School Year 2001-02, available

at: http://nces.ed.gov/pubs2003/overview03/index.asp.

Demographic information about students in the grantee districts is one indicator of the

projects’ success in meeting the goal of serving teachers who teach the neediest students.

Additionally, because TAH projects served teachers in districts other than grantee districts, it is

also helpful to look at project directors’ reports for a more comprehensive picture of schools

where participants worked. Nearly all project directors (93 percent) reported that TAH

participants worked in schools with large percentages of students eligible for participation in the
17

free or reduced-price lunch program. Nearly three-quarters of project directors reported that

participants in their TAH projects worked in low-performing schools (71 percent) and urban

(69 percent) schools. More than half of project directors (56 percent) reported that teachers in

their TAH projects worked in rural schools.5 Forty-six percent of project directors reported that

their TAH participants worked in schools with a large percentage of English-language learners.

Project directors also reported that participating teachers taught gifted and talented students

(30 percent), special education students (30 percent), students with limited English proficiency

(29 percent), and honors or Advanced Placement students (25 percent).

_ Participants in FY 2001 and FY 2002 Projects

With the expansion of grants from 2001 to 2002 came an increase in the number of teachers

served by TAH projects. Like the size of the appropriation itself, the number of participants the

projects proposed to serve nearly doubled from the program’s first year to its second year (see

Exhibit 3-2). (The report’s appendix includes lists of grant recipients for FY 2001 and FY 2002,

respectively, as well as the projected number of participants for each project.)

 5 As the case study projects revealed, TAH projects sometimes drew participants from geographically diverse

schools and districts (i.e., teachers both in urban and rural schools could participate in the same TAH project).
18

Exhibit 3-2

Total Participants in TAH Projects for Grants Awarded in FY 2001 and FY 2002

FY 2001 FY 2002

Average number of participants 145 133

Participant minimum 18 1

Participant maximum 600 744

Number of participants (estimated) 8,689a 15,159b

Total participants in both cohorts (FY 2001 and FY 2002) 23,848

Source: U.S. Department of Education, Teaching American History Grant Abstracts.

Exhibit reads: The average number of participants in a FY 2001 TAH project was 145.

a Information about the total number of TAH participants projected for each project’s three-year grant cycle was

compiled from project abstracts available through the U.S. Department of Education’s Web site (see

http://www.ed.gov/programs/teachinghistory/2001tahabstracts/index.html). For FY 2001, information for 25 of

the 60 grantees was unavailable. Therefore, the estimate of the total number of teachers participating in the

FY 2001 cohort was estimated by using the total number of participants when project data were available, added

to the mean number of participants (145) for projects when information was unavailable.

b Information about the total number of TAH participants projected for each project’s three-year grant cycle was

compiled from project abstracts available through the U.S. Department of Education’s Web site (see

http://www.ed.gov/programs/teachinghistory/2002tahabstracts/index.html). For FY 2002, information for nine of

the 114 grantees was unavailable. Therefore, the estimate of the total number of teachers participating in the

FY 2002 cohort was estimated by using the total number of participants when project data were available, added

to the mean number of participants (133) for projects when information was unavailable.

Just as the size of the grants varied, the number of individuals served by each TAH project

also differed. A relationship did not necessarily exist, however, between the number of teachers

served by the grant and the size of the grant award. (In FY 2002, for example, the number of

participants in nine grantees’ projects awarded $1,000,000 ranged from 30 to 600.) Instead,

grantees allocated their resources differently, depending on a project’s goals and structure.

Whereas some projects chose to train many teachers less intensively, others focused on

intensively training a small cadre of teachers. Some projects, for example, offered colloquia,

which were open to all history teachers from participating districts, while others designated a

small cohort of teachers to train intensively over the course of the three-year grant.

To develop a more detailed profile of participants, we gathered information on participants

involved in TAH activities from June 1, 2002, to September 30, 2003.6 Ninety-two percent of

participants during this period worked as regular full-time teachers. The remaining 8 percent

 6 Although the time frame for data collection for FY 2001 and FY 2002 projects resulted in provision of data for a

relatively narrow time window, it nonetheless allowed both cohorts to hold professional activities over a full

summer—the time of year in which the most intensive activities (i.e., summer institutes) take place.
19

included administrators (1 percent), itinerant teachers (less than 1 percent), long-term substitutes

(less than 1 percent), postsecondary students or preservice teachers (less than 1 percent),

paraprofessional or teacher aides (less than 1 percent), and others. Most participants taught in

high schools (70 percent) (see Exhibit 3-3).7

Exhibit 3-3

School Levels Taught by Participants, 2002-03

(Weighted n = 7,502; unweighted n = 938)

Both 

elementary 

and secondary 

level

4%

Elementary 

level

26%

Secondary 

level

70%

Source: SRI International, TAH Project Participant Survey.

Exhibit reads: Seventy percent of participants reported that

they taught at the secondary grade level.

_ Experience and Credential Status of TAH Participants

TAH participants averaged 14 years of overall teaching experience. Although the average

participant was an experienced teacher, the projects also served teachers with less experience

(see Exhibit 3-4). The amount of teaching experience for TAH participants is about the same as

teachers nationally. Teachers in the 1999-2000 Schools and Staffing Survey, for example, also

averaged 14 years of experience.

 7 The participant survey asked respondents to list the grade levels and subjects they taught. To collapse the school

level of the respondent into elementary and secondary categories, only the first three assignments each teacher

provided were considered.
20

Exhibit 3-4

Years of Teaching Experience Reported by TAH Participants

(As of the End of the 2002-03 School Year)

(Weighted n = 7,448; unweighted n = 931)

20+ years

27%

6-10 years

22%

11-19 years

25%

1-5 years

26%

Source: SRI International, TAH Project Participant Survey.

Exhibit reads: Twenty-six percent of all participants reported that,

as of the end of the 2002-03 school year,a they had been teaching

between one and five years.

a When calculating statistics for this item, responses of zero (for participants

who had not yet completed a full year of teaching) were excluded. Thus, the

exhibit represents the distribution of the number of years of teaching

experience for participants who had taught for a year or longer.

More specifically, in terms of TAH participants’ American history teaching experience, the

average number of years was less (8 years) than their overall teaching experience (14 years). In

fact, a small number of TAH participants (14 percent), reported that they had not yet had a full

year of experience teaching American history.

In addition to being a fairly experienced group, nearly all TAH participants (97 percent)

reported that they had a regular or standard state certificate or advanced professional certificate

in a subject. Participants also reported, however, that their credential was not always in history

or social studies. In fact, participants reported holding certifications in nearly 40 subject areas

(see Exhibit 3-5 for the 10 most common of these subject areas).
21

Exhibit 3-5

The Ten Subject Areas in Which TAH Participants

Most Frequently Reported Certification

(Unweighted n = 863)

Subject Area Percent of Participants

Social studies (general) 46

Elementary education 32

History (general) 23

American history 13

English/language arts 12

Special education 10

Government/political science 10

World history 7

Secondary education 7

Middle school education 6

Note: Because the survey allowed participants to report certification in more than one field,

percentages exceed 100 percent.

Source: SRI International, TAH Project Participant Survey.

Exhibit reads: Forty-six percent of teachers in the survey reported that they had a certification in

social studies (general).

Compared with the national sample, TAH participants appear to be similarly qualified with

respect to certification. Although these data sets are not directly comparable,8 among all teachers

who teach mostly social studies, 94 percent reported that they held a regular/standard state

certificate in their main teaching field.

 8 The national questionnaire asked teachers whether they had a regular or standard state certificate in their teaching

field, whereas the TAH survey asked whether teachers had a regular or standard state certificate and then asked

them to specify what subject it was in.
22

_ TAH Participants’ Training in History

Although not a perfect comparison, a higher proportion of TAH participants either have a

major or minor in history compared with a national sample of secondary teachers who teach

mostly social studies.9 Whereas 38 percent of all TAH participants and 50 percent of TAH

secondary teachers reported having a major in history, only 30 percent of teachers in the national

sample had history majors. Similarly, 23 percent of all TAH teachers and 27 percent of TAH

secondary teachers reported having minors in history, compared with only 7 percent in the

national sample (see Exhibit 3-6).

Exhibit 3-6

Majors and Minors in History Held by TAH Participants and by Teachers Nationally

(Weighted n = 7,480; unweighted n = 933)a

61

23

38 37

7

30

0%

20%

40%

60%

80%

Major in history Minor in history Major or minor in history

Degrees held by TAH participants Degrees held by teachers in the national sample Source: SRI International, TAH Project Participant Survey; NCES, Schools and

Staffing Survey for 1999-2000 (revised May 2003).

Exhibit reads: Thirty-eight percent of TAH participants reported that they had a

major in history, more than the 30 percent of teachers in the national sample that

reported a major in history.

a Note that in some cases (i.e., this exhibit) the total number of respondents represented in

different categories varies slightly. In those instances, the category with the highest number of

 9 For this analysis, “secondary teacher” is defined as one who teaches students in grades 7-12. About a quarter of

TAH participants were elementary school teachers and were far less likely to have a degree in history than their

secondary school colleagues. The national sample included only secondary teachers.
23

respondents is reported.

The majority of TAH participants not only held majors and minors in history but also

completed significant numbers of university courses in American history. Roughly half (49

percent) of all participants reported that they had taken between one and five courses in

American history. In addition, 26 percent reported that they had taken between six and 10

courses, and 22 percent reported that they had taken more than 10 courses in American history.

TAH secondary teachers were more likely to have completed six or more courses in American

history (61 percent) than were TAH elementary teachers (18 percent).

In addition to university training, TAH participants reported that they had previous

professional development opportunities in American history. Nearly four out of five TAH

participants reported that they had participated in some sort of professional development in

American history outside of the TAH (see Exhibit 3-7).

Exhibit 3-7

Participants’ Reports of Professional Development Opportunities

Outside TAH Projects

(Weighted n = 6,054; unweighted n = 976)

30

35

70 73

0%

20%

40%

60%

80%

Activities

sponsored by

professional

associations,

museums, or

historical

societies

District-sponsored

workshops or inservice programs

Postgraduate

college or

university courses

Noncredit courses

Source: SRI International, TAH Project Participant Survey.

Exhibit reads: Seventy-three percent of TAH participants reported that

they had participated in activities sponsored by professional associations,

museums, or historical societies.
24

Although a history degree, significant course work, and professional development in

American history are no guarantee of effective teaching, the educational backgrounds of the

TAH participants suggest that they are likely to have a stronger knowledge of history and

historical methods than most teachers who teach American history. Taken as a whole, TAH

participants appear to be mostly veteran teachers with comparatively stronger backgrounds in

American history than the average social studies teacher. TAH participants often appear to be

self-motivated individuals who voluntarily pursued professional development opportunities

(most commonly during their summer vacations) to improve their content knowledge and

instructional effectiveness.

In addition to having stronger backgrounds than the average social studies teacher, TAH

participants were far more qualified than the many out-of-field and underprepared teachers

(defined as those without at least a preliminary credential) currently teaching American history.

Although current national data on out-of-field and underprepared teachers are not available,

recent data from California reflects the magnitude of the problem. An analysis of California data

suggests that among fully credentialed high school teachers, large numbers teach a subject for

which they do not hold the proper single subject authorization (see Exhibit 3-8). For example,

among high school teachers assigned to teach at least one social science class, more than 1,200,

or 14 percent, are fully credentialed but do not have a single-subject credential that authorizes

them to teach the subject. An additional 6 percent of high school social science teachers do not

hold a full credential.10

 10 Only full-time teachers in California high schools have been included in this analysis. Teachers have been

"assigned" to a subject if they reported they taught at least one class in a core subject—English, mathematics,

social science, physical science, or life science. Teachers could have more than one assignment. For example, a

teacher who teaches three periods of biology and two periods of English would have an English assignment and a

life science assignment, both of which require the teacher to have the proper single subject authorization.

Teachers who were assigned to a subject but lacked a full credential have been included in the underprepared

category. Teachers who indicated they are fully credentialed and authorized to teach in a secondary classroom,

but do not have subject matter authorization in their assigned subject have been included in the out-of-field

category. The percentages in parenthesis indicate the proportion of out-of-field teachers or underprepared

teachers in the total number assigned to teach at least one class in the subject.
25

Exhibit 3-8

Number and Percentage of Out-of-Field and Underprepared

High School California Teachers Assigned to Teach Social Studies, 2003-04

Assigned

Subject

Fully Credentialed Out-of-Field High

School Teachers

Underprepared High School

Teachers Total

Social science

(N = 8,711) 1,207 (14%) 517 (6%) 1,724(20%)

Source: CDE (2004); SRI analysis.

Exhibit reads: There are 1,207 (or 14 percent) fully-credentialed California social studies

teachers who lack a credential in social studies.

Most TAH participants did not appear to be among the nation’s American history teachers

who most needed to improve their skills and knowledge in this subject. Of course, all American

history teachers can benefit from high-quality professional development. Although TAH

teachers appeared to work with some of the needier populations of students, the projects did not

attract the least qualified teachers who often work with these populations of students. The

reasons for this are complex and tied to ways in which the projects are structured, the difficulties

and demands already placed on the least qualified teachers, and the projects’ recruitment

strategies.

_ Challenges of Recruiting the Least Qualified Teachers

Directors generally organized TAH projects around extended summer institutes or workshops

and a variety of follow-up activities. Some longer projects (three years in some cases) included

extensive university course work that led to an advanced degree in history. All projects required

a significant commitment of time and energy by their participants, and nearly all projects tried to

compensate participants for their contribution with financial incentives; however, as our case

studies suggest, monetary compensation may not have been the most effective method of

attracting the teachers most in need of support.

Most of the case study projects reported difficulty in recruiting teachers. One project, for

example, was unable to attract 30 high school teachers from the grantee district to obtain free

master’s degrees. The project director cited insufficient lead time, informal recruitment

practices, teachers’ intensive workload, and the three-year time commitment required of

participants. To meet its goal, the project accepted applicants from middle and high schools in

13 surrounding school districts.
26

Most project directors interviewed acknowledged that they were most successful in recruiting

self-motivated teachers with a strong interest in American history to their projects. In one case

study, for example, a university historian who provided training to teachers highlighted the

implications of the project’s recruitment problems. She acknowledged that, because this project

(a master’s program) was so labor intensive, it may have served least well those teachers most in

need. She theorized that teacher participants who did not do well in the project may have

“fall[en] by the wayside” and that, in fact, some may have dropped out of the program

altogether.

Nearly all of the project directors were rethinking their recruitment strategies, however. In

one case, the project director had hired an American history teacher from the grantee district to

primarily work on recruitment.

Although more aggressive recruitment strategies may help projects attract the least skilled

American history teachers, the projects’ typical lack of integration into teachers’ regular school

year responsibilities suggests that adopting a more aggressive recruitment strategy may not be

sufficient. Taking the form of summer work, weekend or evening workshops, and periodic afterschool sessions, the TAH projects added to teachers’ workloads. If the projects were able to

integrate their professional development activities better with teachers’ daily work, the projects

would be more likely to work with all teachers, not just the most motivated. Because projects

were rarely able to recruit teams of teachers from a school, merging a project’s activities with the

individual school’s professional development was nearly impossible. Seamless integration of a

TAH project with a district’s broader professional development agenda will require strong

leadership and a clear vision from the superintendent and the principals. TAH projects, however,

had little time to develop a more complex form of professional development that is closely

aligned with a district’s larger teacher development strategy. Furthermore, although partnerships

with universities and other organizations provided much needed expertise in American history,

they also added to the sense of the projects being separate from rather than complementary to the

teachers’ responsibilities at their schools.

_ Conclusion

The TAH projects did a remarkable job of quickly building partnerships, creating an

organizational structure, and subsequently offering professional development to tens of

thousands of American history teachers. This is no small accomplishment. Unfortunately, they

were less successful in targeting the least prepared teachers for professional development

opportunities.
27

Chapter 4

What Is the Nature of Teaching American History Professional Development?

The Teaching American History (TAH) program was designed to improve the historical

knowledge and instructional skills of participating teachers, and have the participants, in turn,

improve their students’ achievement in American history. Meeting these objectives is no small

task. Critics of the quality of teaching American history sometimes point to weaknesses in

professional development as a key part of the problem, specifically raising concerns about the

accuracy and rigor of the information teachers receive (Stotsky, 2004). Although a thorough

assessment of the quality of professional development offered by each TAH project exceeds the

scope of this evaluation, this chapter presents evidence concerning the content of those activities.

The chapter first details the content of the professional development that participants received

and then examines the form that the professional development took.

_ TAH Professional Development Content

TAH projects typically tried to improve teachers’ knowledge of American history and their

ability to teach American history. To do so, most projects offered instruction on a range of

periods in American history, on historical methods, and on strategies for delivering content.

About two-thirds of the projects integrated content, methods, and pedagogy, whereas about onequarter focused on building teachers’ content knowledge. Approximately 5 percent of project

directors reported that their projects either primarily emphasized instructional strategies in

teaching American history or primarily emphasized historical methods and processes.

Most projects developed their own professional development programs, although 27 percent

of project directors reported that they based their projects on an existing model or framework for

teaching and learning American history. The Teachers’ Curriculum Institute’s History Alive!

and the National Council for History Education’s Building a History Curriculum: Guidelines for

Teaching History in Schools were typical of the kinds of curricula or models the projects

adopted. At one case study site that employed History Alive! the professional development

offered to teachers emphasized the focus on standards-based content and multiple intelligence

strategies that the curricular framework prescribes. In contrast, another TAH case study site

developed its own professional development program based on the interest of district history

teachers, the needs of district students, existing research on history education, and the writings of

educational philosophers.

Nearly all project directors chose historical topics that matched what participants are

responsible for teaching in their classrooms. Ninety-five percent of project directors, for
28

example, reported that they consulted state standards in planning their TAH projects

(see Exhibit 4-1). Notably, the National Assessment of Educational Progress (NAEP) American

history standards were less frequently consulted than district or state standards.

Exhibit 4-1

History and Social Studies Standards Consulted

During Planning of TAH Projects

(n = 147)

32

52

95

0%

20%

40%

60%

80%

100%

State standards in

history/social studies

District's standards in

history/social studies

NAEP framework for

U.S. history

Source: SRI International, TAH Project Director Survey.

Exhibit reads: Ninety-five percent of project directors reported that state

standards in history and social studies were consulted during the

planning of their TAH project.

Project directors also reported that they varied the content of the professional development in

accordance with the grade levels teachers taught. Fifty-nine percent of project directors, for

example, reported that content for secondary teachers differed from that for elementary school

teachers. In one project, this difference translated into a focus on themes and historic sites for

elementary school teachers, with a focus on historical content for secondary teachers.
29

_ TAH Projects and NAEP Historical Periods, Themes, and Skills

Although most TAH projects were not organized around NAEP’s U.S. History Framework,11 it

did prove useful in describing the multiple historical periods, themes, and thinking skills that the

projects typically addressed. Overall, the TAH projects covered each of the eight NAEP historical

periods. Project directors reported that the projects gave greater focus to pre-Civil War and Civil

War periods than to other periods in American history (see Exhibit 4-2); the American Revolution

(including such topics as the Revolutionary War, Women in the Revolution, the Revolutionary

Era, Revolutionary Ideals, and the Revolution through Local History) was the most commonly

covered historical period, with the Civil War the second most commonly covered period.

Exhibit 4-2

NAEP Chronological Periods Addressed by TAH Projects

(n = 137)

72

70

69

62

59

54

53

37

0% 20% 40% 60% 80%

Revolution and the New Nation (1763-1815) 

Crisis of the Union: Civil War and Reconstruction

(1850 to 1877) 

Expansion and Reform (1801-1861) 

Colonization, Settlement, and Communities (1607-

1763) 

Development of Modern America (1865 to 1920)

Contemporary America (1945 to present) 

Modern America and the World Wars (1914 to 1945) 

Three Worlds and Their Meeting in the Americas

(Beginnings to 1607)

Note: According to the National Assessment Governing Board (NAGB), the overlap in

NAEP chronological periods reflects “the fact that the periods are not clearly delineated

and that transitions from one context to another are typical” (Loomis and Bourque, 2001).

Source: SRI International, TAH Project Director Survey.

Exhibit reads: Seventy-two percent of project directors reported that their TAH projects

addressed the NAEP Chronological Period “Revolution and the New Nation (1763 to 1815).”

 11 National Assessment Governing Board (NAGB), U.S. Department of Education. (2001). U.S. history framework

for the 1994 and 2001 National Assessment of Educational Progress. Washington, D.C.: Author.
30

Project directors also reported covering the four NAEP themes: (1) The Gathering and

Interactions of Peoples, Cultures, and Ideas; (2) Change and Continuity in American Democracy:

Ideas, Institutions, Practices, and Controversies; (3) Economic and Technological Changes and

Their Relation to Society, Ideas, and the Environment; and (4) The Changing Role of America in

the World (see Exhibit 4-3).

Exhibit 4-3

NAEP Themes Addressed by TAH Projects

(n = 134)

58

69

83

91

0% 20% 40% 60% 80% 100%

The gathering and interactions of peoples,

cultures, and ideas 

Change and continuity in American democracy:

ideas, institutions, practices, and controversies

Economic and technological changes and their

relation to society, ideas, and the environment

The changing role of America in the world 

Source: SRI International, TAH Project Director Survey.

Exhibit reads: Ninety-one percent of project directors reported that their project addressed

the NAEP Theme entitled “The Gathering and Interactions of Peoples, Cultures, and

Ideas.”

According to the project directors, the TAH projects also familiarized participants with two

cognitive domains, which are also referred to as NAEP’s “Ways of Knowing and Thinking

About U.S. History.” Some NAEP historical skills were more likely to be addressed than others

(see Exhibit 4-4). Specifically, project directors reported that the TAH projects were far more

likely to emphasize “applying historical knowledge” and “recognizing multiple perspectives and

seeing an era or movement through the eyes of different groups” than “making defensible

generalizations” and “finding value statements.”
31

Exhibit 4-4

Specific “Ways of Knowing and Thinking About U.S. History”

Addressed by TAH Projects

(n = 139)

93

91

90

81

78

71

70

69

67

60

55

34

0% 20% 40% 60% 80% 100%

Applying historical knowledge 

Recognizing multiple perspectives and seeing an era

or movement through the eyes of different groups 

Knowing and understanding people, events, concepts,

themes, movements, contexts, and historical sources 

Explaining issues, identifying historical patterns 

Establishing cause-and-effect relationships 

Developing a general conceptualization of U.S. history 

Weighing evidence to draw sound conclusions 

Rendering insightful accounts of the past 

Establishing significance 

Sequencing events 

Making defensible generalizations 

Finding value statements 

Note: The scholarly skills outlined above come from the two cognitive domains explained

on pages 42-44 of the U.S. History Framework for the 1994 and 2001 National Assessment

of Educational Progress (NAGB, 2001).

Source: SRI International, TAH Project Director Survey.

Exhibit reads: Ninety-three percent of project directors reported that applying historical

knowledge was a “Way of Knowing and Thinking about U.S. History” addressed by their

TAH project.

a Note that NAEP identifies “explaining issues” and “identifying historical patterns” as two separate “Ways of

Knowing and Thinking About U.S. History.” In SRI’s Project Director Survey, these two items were combined

into one response option due to space considerations.

a
32

One of the case study projects focused on some of the historical thinking skills outlined in

“Ways of Knowing and Thinking about U.S. History,” as Exhibit 4-5 demonstrates.

Exhibit 4-5

Incorporating Historical Thinking Skills

Professional development providers at one TAH project showed participants how historians ask

historical questions, develop interpretations, and present their findings. “We want to give the teachers

diverse perspectives of scholars talking about different topics in different ways,” the project director

explained. “The first year they wanted the right answer, and now they know there are multiple

interpretations. We want teachers to teach like this.” During one day’s activity, the project invited

three historians to discuss their interpretations of the same historical topic. All presented their

research, provided an overview of the methods they used, indicated how they arrived at their

conclusions, and discussed the merits and weaknesses of competing interpretations. Following their

presentations, the group participated in a roundtable discussion with participants; the question-andanswer session allowed the teachers to observe how contentious historical debates can be. With this

approach, instead of simply receiving content passively, the participants understood how historians

produce knowledge and how the same subject can be treated in dramatically different ways depending

on the analytical frameworks and sources the investigator employs. Such work was a key component

of the project’s goal of teaching participants how to “do” history. The project director advocated this

approach rather than having teachers create simple timelines and memorize events, people, and

places. “I’m a big believer in historical thinking skills as the most important thing. My goal in [this

project] is to make sure that those historical thinking skills are never too far away from the teachers.

It’s not enough for them to just replicate [what they’ve learned], but they have to be comfortable

creating it themselves.” To this end, the teachers conducted original research using primary sources

in the state’s historical society’s archives.

Historians often refer to the practice of their discipline as a craft; that is, each historian has

his or her own way of collecting and examining evidence. Accordingly, project directors were

asked to indicate the commonly used historical methods that project participants were exposed to

(see Exhibit 4-6).
33

Exhibit 4-6

Historical Methods Addressed by TAH Projects

(n = 147)

94

66

63

63

50

86

86

0% 20% 40% 60% 80% 100%

Analyzing historical documents, such as

manuscripts and diaries 

Analyzing history by themes, periods, and regions 

Analyzing historical artifacts, material culture,

and/or media (i.e. video, Internet, music) 

Analyzing the historical significance of place 

Comparing and contrasting differing interpretations

of history and historical events 

Forming hypotheses and making conclusions

based on historical evidence 

Analyzing oral histories 

Source: SRI International, TAH Project Director Survey.

Exhibit reads: Ninety-four percent of project directors reported that participants analyzed

historical documents, such as manuscripts and diaries, as part of a TAH project.

According to project directors, TAH participants had opportunities to employ many of the

historical methods historians use. Two of the case study projects offered particularly good

examples of such exercises (see Exhibit 4-7).
34

Exhibit 4-7

Using Historical Methods

One case study project in the Northeast focused on local history and used easily accessible regional

resources to help participants learn, “do,” and teach history. The project integrated content and

pedagogy in its summer institute. During the mornings, teachers participated in seminars or listened to

lectures on historical topics and readings; work in the afternoons included independent research,

discussions of how to incorporate the morning’s ideas into their classrooms, or collaboration on the

development of different instructional materials. Participants then presented their work to fellow

attendees and project staff and received oral or written feedback about how to revise and strengthen

the material.

In another case study project, participants analyzed 19th-century census data and teenage diaries,

and discussed issues associated with the sources (e.g., census-taker biases, diary-keepers’ class

backgrounds). With guidance from university faculty, participants discussed what the materials

revealed about the social and cultural lives of people in the diary (“What does keeping a diary tell us

about the education level of the writer?” “What do the different diaries reveal about the lives of the

writers, their families, their class, and their beliefs?”). The providers also coached participants in using

such primary sources in their classrooms.

Overall, the project directors’ reports suggest that the TAH projects did a good job of

covering NAEP periods, themes, and skills. Next, we examine the form of the TAH projects’

professional development and compare it with research-based definitions of effective

professional development.

_ TAH Projects and the Characteristics of Effective Professional Development

Research to define effective history professional development is minimal, mainly consisting

of evaluations of one state-run program in California (Kroesch and Edwards, 2000; Medina et al,

2000; Podany, 2000; St. John et al., 1999; Seixas, 1999). Because of the limited amount of

history-specific research, findings concerning professional development in other academic

subjects have been taken into consideration in the analysis here. For these other academic areas,

researchers have identified a variety of characteristics that may positively affect teacher

development. They include the following features: sufficient duration of development events

(longer events are more likely to improve teacher practice), collective work (teachers are more

likely to implement new knowledge and skills if they attend events with colleagues), content

focus (teachers need deeper content knowledge to teach to challenging academic standards),

active learning (the more engaged teachers are, the more likely they are to learn), coherence

(teachers are more likely to implement new strategies if the strategies are aligned with curricular

materials, standards, assessments, and other policies affecting teachers), a match with teachers’

professional skill level, and ongoing “reform” activities like study groups or mentoring that seek

to change teacher practice (one-time workshops tend to be minimally effective) (Birman,

Desimone, Porter, and Garet, 2000; Garet et al., 1999; Kennedy, 1999; Loucks-Horsley, 1999).
35

Not surprisingly, the TAH projects did not always offer training consistent with all of these

features. Both participants and project directors, however, described a variety of activities that

matched the research-based definitions of effective professional development. TAH projects, for

example, frequently had teachers work together to develop curriculum and instructional

materials. Among project directors, 78 percent reported such collaborative activities took place

often or very often. Participant reports were similar. The case studies indicated that such

activities provided participants with active learning opportunities, and promoted a sense of

relevance and coherence.

As indicated earlier in this report, the projects appeared to be of sufficient duration and

included some follow-up beyond the initial summer institute or workshop. In addition, both

participants and project directors reported that TAH professional development included formal

knowledge sharing by teachers. More than half (54 percent) of all project directors reported that

such “train the trainer” activities took place often or very often. Participant data support the

finding that formally sharing training was an important component of TAH activities.

Forty-three percent of participants reported that such sharing occurred “regularly” or “more than

occasionally.”

Although project directors’ and participants’ reports suggest that TAH professional

development offered active learning, promoted coherence, and encouraged professional

communication, other characteristics of effective professional development appear to have been

less common. Although comprehensive data have not been collected for all TAH projects, the

case studies suggest that the projects had difficulty recruiting teams of teachers from the same

school and therefore found it hard to undertake schoolwide reform efforts. Projects tended to

follow traditional formats (workshops and courses) rather than incorporate reform structures

(teacher networks, internships, and research projects). Project directors did recognize the

importance of follow-up activities, as discussed below, and indicated that they included such

activities in their programs.

_ TAH Projects’ Follow-up Activities

Nearly all project directors strongly agreed or agreed (92 percent) with the statement that

“project activities were sustained over time, with ample follow-up activities and experiences.”

The follow-up activities, however, did not directly focus on the teachers’ work in their

classrooms. TAH participants reported that follow-up activities were more likely to take the

form of information delivery than visits to classrooms. In fact, only 31 percent of participants

reported more than one visit to their classroom from a TAH project representative

(see Exhibit 4-8).
36

Exhibit 4-8

Follow-up Offered by TAH Projects

(Weighted n = 6,542; unweighted n = 817)

82

73

71

69

31

31

0% 20% 40% 60% 80% 100%

E-mail

Dissemination of addition materials 

Workshops, trainings, meetings 

Mail

Phone

More than one visit to the workplace

Source: SRI International, TAH Project Participant Survey.

Exhibit reads: Only 31 percent of participants reported that follow-up for

the TAH project in which they participated involved more than one visit

to their workplace (classroom).

Similarly, few TAH projects required participants to observe each other’s instruction or visit

another teacher’s classroom. Only 10 percent of participants reported observing another

teacher’s classroom four times or more. Case study findings indicate that projects did not

undertake classroom-focused follow-up because of associated expense and difficult logistics.

Some project directors reported that they provided classroom visits only when individual

participants requested they do so. Regardless of the reasons, most TAH projects did not provide

teachers with feedback about their teaching or observe teaching to assess project effectiveness.

_ The Challenge of Creating Effective Professional Development

The TAH projects goals were ambitious: quickly build the organizational capacity necessary

to provide high-quality professional development to large numbers of teachers, form effective

partnerships between school districts and universities or other organizations, and carefully assess

the project’s effectiveness. Although the projects deserve credit for their many

accomplishments, the program’s overall goal may have been too ambitious to expect total

success within the grant’s timeframe.
37

Ensuring that TAH professional development matched the characteristics of high-quality

professional development proved especially challenging. Although providing such features as

ongoing follow-up can be expensive and logistically difficult, the projects’ structures often made

classroom-based follow-up extremely problematic. Furthermore, although the partnerships with

universities and other organizations provided much needed expertise in American history, they

also conveyed a sense that the projects were not strictly applicable to the teachers’

responsibilities at their schools. Because projects were rarely able to recruit a team of teachers

from a single school, merging a project’s work with an individual school’s professional

development activities was nearly impossible. Integrating a TAH project with a district’s

broader professional development agenda needs the participation of district and school

leadership. No evidence was found, however, of any significant involvement of either group in

the case study projects.

TAH projects had little time to provide professional development aligned with a district’s

teacher development strategy and a school’s daily operations. With the continuing support of the

TAH program and continuing cohorts of grantees, however, projects should be able both to

depart from traditional forms of teacher professional development and to harmonize teacher

learning opportunities with daily classroom and teacher practices.

_ Conclusion

Overall, TAH projects exposed participants to a wide range of historical periods, themes, and

thinking skills that were consistent with the NAEP standards. In addition, the projects provided

opportunities for participants to use historical methods, promoted active learning, and

encouraged professional conversations about subject matter and instructional strategies. Project

emphases, however, did not always correspond to what researchers have identified as key

characteristics of effective professional development. In particular, most TAH projects offered

little in the way of intensive classroom-based follow-up. Chapter 5 examines data on the impact

of TAH activities on teachers.

39

Chapter 5

What the Evaluators Learned About the Effectiveness of TAH Projects

Because this program evaluation focused on the implementation of the first two cohorts of

the Teaching American History (TAH) program, only limited information on the overall

effectiveness of individual projects or the program as a whole is available. Ideally, further study

will reveal the contributions of the program to gains in student achievement and teachers’ skills

and knowledge. Analyses of selected internal project evaluations, participant and project

director surveys, and eight case study projects, as well as an exploratory review of teacher

materials, however, suggest that, although the projects provided benefits for improving the

teaching of American history, they also had limitations.

_ Internal TAH Project Evaluations

Project directors reported using a variety of approaches to assess the effectiveness of their

TAH grants. Teacher self-reports were the most common element of TAH projects’ evaluation

methods—in fact, 91 percent of all project directors reported including this component in their

evaluation. Analyses of student work or achievement, classroom observations, or other more

intensive forms of evaluation were far less common (see Exhibit 5-1).

Exhibit 5-1

Elements of TAH Project Evaluations

(n=148)

91

82

65

64

48

46

41

25

0% 20% 40% 60% 80% 100%

Teacher self-reports

Observations of project activities

Participant focus groups 

Analysis of participant work products

Classroom observations

Analysis of students' knowledge 

Analysis of partcipants' knowledge 

Analysis of student work 

Source: SRI International, TAH Project Director Survey.

Exhibit reads: Ninety-one percent of project directors reported that their

evaluations included teacher self-reports.
40

As Exhibit 5-1 illustrates, fewer than half of the project directors reported that their

evaluations involved observing the classrooms of teacher participants (48 percent) or analyzing

student or participant content knowledge in American history (46 percent and 41 percent,

respectively). A somewhat greater number, 64 percent, analyzed the work products of teacher

participants. This analysis (typically of lesson plans) took place during the summer institutes

and workshops, and was part of the participants’ professional development. Although the

analysis of teachers’ work products is an important component of the projects, few examples

were found of projects using this method to determine what participants had learned during the

professional development activities. Overall, the projects’ efforts to assess students’ or teachers’

knowledge of American history did not appear to be systematic.

Although project directors reported that they undertook various evaluation activities, the case

study data suggest that they relied heavily on teachers' self-reports. One project director, for

example, reported using a combination of teacher surveys and a review of student work. The

project administered surveys at the beginning and end of the summer institute, and again at the

end of the school year. Teachers were also asked to collect the lesson plans and examples of

student work from their study of the Revolutionary War, assess these materials, and report any

changes in student performance. According to the project director, the teachers reported gains in

student learning and teacher knowledge.

Other projects undertook more ambitious evaluation activities but generally did not engage in

systematic data collection. One project, for example, surveyed teachers and found that they

reported positive gains in attitude, content knowledge, pedagogy skills, leadership skills, and

knowledge of technology after they participated in the summer institute. In addition, the coproject director and some of her graduate students observed some participants’ classrooms and

reported that teacher practice had improved, but they lacked the baseline data necessary to make

comparisons.

While most of the case study projects remained dependent on teacher reports for their

evaluations, projects appeared to be moving toward more rigorous evaluation efforts. One case

study project used an external evaluator to study the historical writing ability of students whose

teachers participated in the professional development and compared it with the ability of students

whose teachers did not attend any TAH professional development. Preliminary findings suggest

that the participants’ students earned higher scores than the students of nonparticipants. Recent

competitions (beginning in 2003) for TAH grants have rewarded proposals for including more

experimental or quasi-experimental research designs.
41

_ Project Directors’ and Participants’ Reports on TAH Project Effectiveness

Assessments of the projects’ effectiveness were mostly limited to self-reports from project

directors and participants. On the basis of their evaluation efforts, project directors reported on

improvements in a variety of areas in teaching American history (see Exhibit 5-2).

Exhibit 5-2

TAH Project Directors’ Reports on Improvement in Selected Areas

(n=109)a

A Great Deal Substantial Moderate Some Little or no

Teachers' content knowledge 17% 50% 25% 7% 0%

Teachers' knowledge of

instruction 18% 34% 34% 12% 2%

Teachers' interest in

American history 44% 39% 10% 6% 0%

Student performance 10% 19% 29% 34% 7%

Student interest 12% 39% 23% 23% 4%

Source: SRI International, TAH Project Director Survey.

Exhibit reads: Sixty-seven percent of project directors reported “a great deal” or a “substantial”

amount of improvement in teachers’ content knowledge.

a Note that the number of respondent for each component of this item differed slightly, so the n reported in this

exhibit represents the highest number of respondents that answered any part of this question.

Project directors reported a great deal of improvement in teachers’ interest in teaching

American history (44 percent), as well as a substantial improvement in teachers’ content

knowledge (50 percent). Project directors reported more modest improvements in student

performance in American history, with only 29 percent reporting a great deal or substantial

improvement.

TAH participants also reported that the projects made important contributions to their

historical content knowledge and skills (see Exhibit 5-3).
42

Exhibit 5-3

Teacher Reports of Contribution of Their TAH Project

(Weighted n=7,520/unweighted n=942)a

Strongly

Disagree (1) (2) (3) (4)

Strongly

Agree (5)

Not

Applicable

Increased Ability to Use

Historical Methods 2% 3% 11% 23% 59% 2%

Can Better Assess Students in

American History 3% 9% 20% 23% 38% 7%

Taught Me to Use Historical

Resources 4% 6% 17% 23% 47% 3%

Consistent With State/District

Standards 2% 3% 7% 19% 67% 3%

Source: SRI International, TAH Participant Survey.

Exhibit reads: The majority of respondents (59 percent) strongly agreed that the TAH project in

which they participated increased their ability to use historical methods.

a Note that the number of respondent for each component of this item differed slightly, so the n reported in this

exhibit represents the highest number of respondents that answered any part of this question.

Despite the limits of self-reports, it is notable that the vast majority of project directors and

participants believed that the TAH projects made important contributions to teachers’ knowledge

of American history, historical methods, and pedagogical skills. Exploratory examination of

lesson plans and other products from a sample of the projects suggests that, despite these selfreported gains by project directors and participants, there remains room for further improvement

in teachers’ history knowledge.

_ Review of Teacher Materials: An Exploratory Sub-study

As part of the evaluation, 44 teacher-produced materials (typically lesson plans) were

collected and analyzed from six of the eight sites visited for case studies. The materials were

created by teachers who had participated in a TAH project for at least a year (see Appendix B for

more information on how the teachers and assignments were selected). SRI convened and

trained an expert panel consisting of six experienced elementary and secondary history teachers,

one assessment expert, one historian, and three SRI staff members with degrees in history to

evaluate the materials collected during the site visits.

Over a span of three days the group was trained to use a common rubric and assess the

materials. The first day was spent fine-tuning the assessment tool and conducting a whole-group

review of two lesson plans. For the next two days, the 11 members were divided into three

groups, split to ensure that historical expertise and school level were well-represented in each.
43

The assignments were randomly distributed to the groups, and each group reviewed a different

set of materials from each site. The final scores for each work product were the result of group

consensus. The group used a rubric, based on NAEP’s cognitive domains12 and developed by

SRI in concert with the assessment expert. Because of variations in how TAH projects assigned

and collected teacher work samples, using NAEP provided an objective standard to measure all

projects against. The two cognitive domains in the NAEP framework are: (1) Historical

Knowledge and Perspective, and (2) Historical Analysis and Interpretation. Within each

cognitive domain, the NAEP framework identifies another set of specific skills, known as “Ways

of Knowing and Thinking About U.S. History.” The rubric used to assess the teacher materials

had a three-point scale for each of the specific skills (see Appendix F for the complete rubric).

Under the category of Historical Knowledge and Perspective, for example, the specific skill of

“demonstrating knowledge and understanding of people, events, concepts, themes, movements,

contexts, and historical sources,” had a three-point scale (see Exhibit 5-4):

Exhibit 5-4

Sample from the TAH NAEP-based Materials Review Rubric

Section 1: Historical Knowledge and Perspective

The teacher work product:

1.A. Demonstrates knowledge and understanding of people, events, concepts,

themes, movements, contexts, and historical sources

1.A.1 Insufficient Evidence

1. has many factual errors 2. has some incorrect factual

information

3. is factually accurate

1.A.2. Insufficient Evidence

1. knowledge is of superficial

depth and incomplete

breadth

2. knowledge is in some

depth or in breadth

3. knowledge is of perceptive

depth and

comprehensive breadth

1.A.3. Insufficient Evidence

1. doesn’t use historical

themes or incorrectly

defines historical themes

and incorrectly relates

them to specific factual

information

2. partially defines historical

themes and/or somewhat

relates them to specific

factual information

3. clearly defines historical

themes and accurately

relates them to specific

factual information

 12

National Assessment Governing Board (NAGB), U.S. Department of Education. (2001). U.S. history framework

for the 1994 and 2001 National Assessment of Educational Progress. Washington, D.C.: Author.
44

A number of factors affected the assessment of the teacher work products. These are

discussed in the appendix.

_ Results of the Exploratory Study of Teacher Materials

Overall, the materials reviewed exhibited both strengths and weaknesses, with the projects

earning a combined average score across all rubric items of 1.73. Scores ranged from a high of

2.16 for all materials from one project to a low of 1.27 for another project.

Some projects distinguished themselves as stronger than others and, more specifically,

certain teacher samples stood out among those submitted by the same project. A lesson plan

developed by an elementary school teacher was the only one in our review to receive a perfect

score in each NAEP category. Using local history as a lens, the lesson sought to engage third-,

fourth-, and fifth-graders in understanding how colonial economies functioned (that is, before the

institution of a unified monetary system). The lesson taught how people of Colonial America

used currency and what value it had to them (to fulfill wants and needs). The lesson illustrated

the teacher’s knowledge of and ability to use both lower and higher level historical thinking

skills. Her description of the world and the challenges the initial colonists encountered

demonstrated her firm grasp of the historical period and topic she was teaching, including

knowledge of key people, places, and events. In addition, her lesson reflected her knowledge of

the importance of, and her ability to include, balanced perspectives of both colonists and Native

Americans, allowing her students to weigh the similarities and differences of the two groups.

Moreover, by connecting students’ understanding of money’s current uses (also to fulfill wants

and needs) with those of Colonial America, the teacher demonstrated that she could connect the

past and the present, as well as identify historical patterns. To illustrate this aspect and ground

the lesson in historical artifacts the teacher made good use of primary sources—including

replicas of colonial money and a visit to a local museum—and students’ independent research on

assigned Web sites. This example and others from this site (which had the highest overall

average score) benefited from feedback provided by historians to teachers to guide the revisions

of their initial efforts. Their work reflected what could be done when the proper support from

skilled providers is readily available.

Some other lesson plans tended to lack such a depth of knowledge and historical thinking

skills. For example, several of these lessons dealt with Colonial America but only in a way that

demonstrated a superficial understanding of the era. Although the content presented in the first

of these lesson plans was factually accurate, it lacked depth, and much of the rest of the plan did

not demonstrate an understanding of key higher level historical concepts and thinking skills. No
45

historical theme was established and an appreciation of chronology was lacking. Other than

stating that this lesson covered an aspect of the colonial period, no understanding was

demonstrated of when during the period this aspect was in effect or how the regions discussed

changed over time.

Another lesson plan had similar problems. It explored the meeting of Native Americans and

Europeans in the New World, attempting to gauge the impact of their interaction on the native

peoples. Although the lesson was also factually correct, no sense of time was apparent from the

material presented, no historical context was provided, the explanation of cause and effect was

vague and simplistic, and no attempt was made to consider the topic from multiple

perspectives—in this case, those of Native Americans and European settlers. It was also unclear

where in the New World this contact occurred, when it took place, and which Native Americans

and Europeans were interacting. No discussion of the complexity of this historical event was

provided, merely a decontextualized focus on the physical and cultural violence visited on the

Native Americans. This project’s lack of historical expertise appeared to have hampered its

capacity to provide feedback to the teachers.

Across all six case study projects, the teacher assignments scored best in the Historical

Knowledge and Perspective categories. These items largely measured factual knowledge (e.g.,

the ability to identify and place historical events and people accurately), sequencing of history,

recognition of multiple perspectives, and development of a general conception of history. The

average score across all projects in this section was 2.01 on a three-point scale. The scores

ranged from a high of 2.78 in regard to the teachers’ ability to present factually accurate

information to a low of 1.76 in regard to their ability to recognize multiple perspectives. The

projects scored less well overall in the Historical Analysis and Interpretation categories,

however. These items reflect higher order historical thinking skills. The average score across all

projects in this section was 1.63. The scores ranged from 1.77 in regard to participants’ ability to

establish significance to a low of 1.50 for their ability to identify historical patterns

(see Exhibit 5-5).
46

Exhibit 5-5

Average Score, by NAEP-based Historical Skillsa

Average

Historical Knowledge and Perspective

Demonstrates knowledge and understanding of people, events, concepts,

themes, movements, contexts, and historical sources

Factual accuracy 2.78

Depth and breadth of knowledge 1.92

Define historical themes 1.93

Sequences historical events

Places specifics in historical chronology 1.85

Constructs and labels historical periods 2.19

Recognizes multiple perspectives 1.76

Sees an era or movement through the eyes of different groups 1.79

Develops a general conception of U.S. history 1.89

Historical Analysis and Interpretation

Explains historical issues 1.61

Identifies historical patterns 1.50

Establishes cause-and-effect relationships 1.65

Establishes significance 1.77

Applies historical knowledge 1.64

Weighs evidence to draw sound conclusions 1.74

Makes defensible generalizations 1.63

Renders insightful accounts of the past 1.58

Source: SRI International, Exploratory Study of Teacher Materials.

Exhibit reads: On a three-point scale, the factual accuracy of teachers’ lesson plans averaged 2.78.

a Note that the rubric developed by SRI did not include the NAEP-based skill entitled “finds value statements”

because the evaluation team deemed it too ambiguous.
47

Although this was an exploratory study, preliminary findings suggest that despite project

directors’ reports that historical thinking skills from NAEP’s U.S. History Framework were

covered during the participants’ professional development, some skills were easier for participants

to master than others. Across all projects, most of the participants’ lesson plans and other products

earned weak scores in historical analysis and interpretation. Although participants demonstrated

the ability to be factually accurate, few were able to use that capacity in sophisticated ways.

_ Conclusion

According to project directors’ and participants’ reports, the TAH projects appeared to have

made important contributions to the improvement of participants’ knowledge of American

history, their use of historical methods, and their ability to teach American history. A closer

analysis, however, indicates there is more room for improvement in teachers’ historical analysis

and interpretation skills. At the same time, the exploratory study revealed the potential benefits

of undertaking such an analysis of teachers’ lesson plans and other products. Projects could

benefit from a more careful analysis of the work of their participants. Overall, the examination

of the available data on the projects’ effectiveness suggests the need to move beyond self-reports

to more rigorous evaluation efforts.

49

Chapter 6

Conclusions

Whether examining the quality of the professional development that Teaching American

History (TAH) projects offered or determining whether or not the projects served the teachers

with the greatest needs, this evaluation found that looking below the surface revealed a more

complex picture than first meets the eye. The TAH grants went to many urban and rural districts

with large numbers of low-performing, minority, and poor students. On the surface, this finding

suggested that TAH resources were reaching the teachers with the greatest need for

improvements in their teaching of American history. And, at first glance, those teachers

appeared to mirror the wider population of teachers based on their experience. A closer look at

the academic backgrounds of the TAH participants, however, suggests that TAH teachers were

far more likely to have majored or minored in history than the average social studies teacher.

While the projects, perhaps, did not succeed in serving those teachers typically considered to be

most in need of professional development, the analysis of teacher work products demonstrated

that, like their fellow teachers, TAH participants also need to improve their historical skills.

On the surface, the TAH projects appeared to have incorporated many of the characteristics

of research-based high-quality professional development. But a closer look revealed that the

kinds of follow-up activities that the projects employed rarely included classroom-based support

and assessment. Similarly, although the projects purported to have addressed the historical

thinking skills in the U.S. History Framework for the 1994 and 2001 National Assessment of

Educational Progress, this evaluation’s exploratory study of teacher lesson plans and other

products revealed that the products did not exhibit strong historical analysis and interpretation.

Although looking below the surface reveals important areas for individual projects and the

TAH program to address, it does not detract from the program’s significant accomplishments.

After many decades with few resources dedicated to improving the teaching and learning of

American history, the Department of Education launched the TAH program quickly and

smoothly, and tens of thousands of American history teachers have already participated in

program activities. Whether they started from scratch or built on preexisting relationships, many

of the grantee districts have forged successful partnerships with university history departments

and historical institutions. During the lifespan of the TAH grants they received, districts have

increased their capacity to meet their history teachers’ professional development needs. In

addition, some projects have successfully used the grants to create lesson plans, gathered various

historical materials, and disseminated them for use by teachers who did not directly participate in

the projects. In other words, the challenges that this evaluation revealed should be placed in the
50

context of the overall value of the TAH program’s capacity to the assist American history

teachers and their students.

_ Implications for the TAH Projects

The No Child Left Behind Act (NCLB) has assumed greater prominence since the TAH

projects examined here applied for their grants, and educators’ attention to the requirements and

implications of the law has increased as well. A key NCLB requirement is to provide every child

with a highly qualified teacher, and as a federal program, TAH needs to help meet NCLB goals.

The TAH projects trained large numbers of veteran teachers who were more likely to have a

degree in history than the average social studies teacher. The projects were less successful in

attracting and supporting the large number of American history teachers who lacked preliminary

credentials or were teaching out of field. As the evaluation found, attracting the teachers with

the greatest need will take more than financial incentives or more aggressive recruitment

campaigns. Projects’ inability to integrate their professional development offerings with state

and district efforts hindered their ability to reach the neediest teachers. In addition, existing

burdens on teachers who are already busy trying to bring their credentials into compliance with

NCLB’s “highly qualified” requirement can also preclude their participation in other

professional development activities like TAH.

The evaluation also found that although the projects reflected many of the attributes of

research-based highly effective professional development, they generally did not provide

teachers with the kind of intensive in-classroom follow-up that is most likely to improve teacher

practice. This finding suggests that projects may not be structured to provide training that is

closely attuned to actual classroom needs. Furthermore, the need to attract more of the neediest

teachers also suggests that current TAH projects have not integrated their strategies sufficiently

with the realities of ongoing demands on classroom teachers.

Finally, the evaluation found that most TAH projects relied on self-reports to assess their

effectiveness, and thereby failed to acquire adequate information to make sound decisions about

how to improve the projects. Although subsequent cohorts of TAH grantees have been

encouraged to improve their internal evaluations, even the more recent grantees recognize that

rigorous evaluation is difficult and often costly. However, the first two cohorts of TAH grantees

were often, in fact, assessing and helping teachers refine the lesson plans and other products that

they produced during their participation. At the very least, a more systematic assessment of
51

teachers’ lesson plans and other teacher work products could better inform project improvements

than reliance on teacher reports.

_ Policy Implications for the TAH Program

In addition to supporting the individual projects’ efforts to support NCLB goals by serving

the neediest teachers, enhancing their follow-up activities, and improving evaluation, the TAH

program could expand its current efforts to address the weaknesses of American history teaching

and learning more thoroughly. Among the many needs is a better understanding of what

constitutes effective professional development for American history teachers. As stated in the

report, the research on professional development in history is quite limited. Although much of

the research on effective professional development in other academic subject areas is applicable,

almost nothing is known about the specific features of effective professional development in

American history. The quality of some projects may have been affected by the lack of

information about best practices and successful models. The TAH program could improve this

situation by supporting additional research on the projects.13

In addition, the TAH program currently addresses just one part of the system that contributes

to ineffective American history education. As researchers have argued, American history

textbooks and other materials are often of poor quality. Nonetheless, as the evaluation revealed,

teachers in many of TAH projects were developing rich lesson plans, and projects were

identifying and collecting useful historical artifacts, primary sources, and exemplary units of

study. These resources were rarely distributed beyond the project participants, however.

Finally, as the evaluation indicated, involving university and college historians in the

professional development of teachers had many benefits. The system of preservice teacher

preparation has rarely tapped into the expertise of historians as part of the training provided to

prospective educators. Noteworthy efforts in this regard are under way (e.g., the Carnegie

Corporation of New York’s Teacher for a New Era), but the TAH program could accelerate

efforts to involve historians in the preparation of new teachers by expanding its reach to include

projects that involve teacher preparation programs. Because TAH grants are awarded to

districts, the projects lack the capacity to pursue this endeavor.

 13 As stated earlier, the U.S. Department of Education introduced an invitational priority for experimental and

quasi-experimental evaluation designs as part of its application notice for FY 2003 and subsequent fiscal years.

In addition, the U.S. Department of Education, in its FY 2006 Budget Request to Congress, has asked Congress

for the flexibility to spend some TAH funds on such “national activities” as identifying and disseminating model

lesson plans and materials from TAH projects.
52

The TAH program has an opportunity to improve the research on professional development

for American history teachers, disseminate the resources that the projects have developed, and

address some of the deficiencies of teacher preparation. This more comprehensive approach has

the potential to speed the improvement of teaching and learning American history.
53

References

American Council of Trustees and Alumni (2000). Losing America’s memory: Historical

illiteracy in the 21st century. Retrieved June 18, 2003, from:

http://www.goacta.org/publications/Reports/acta_american_memory.pdf.

American Federation of Teachers (AFT) (2001). Making standards matter 2001. Retrieved June

18, 2003, from: http://www.aft.org/Edissues/standards/MSM2001/Index.htm.

Beatty, A. S., Reese, C. M., Persky, H. R., and Carr, P (1996). NAEP 1994 U.S. History Report

Card: Findings from the National Assessment of Educational Progress. Washington, D.C.:

Office of Educational Research and Improvement, U.S. Department of Education. Retrieved

Sept. 24, 2003, from: http://nces.ed.gov/pubsearch/pubsinfo.asp?pubid=96085.

Birman, B. F., Desimone, L., Porter, A. C., and Garet, M. S. (May 2000). Designing

professional development that works. Educational Leadership, 28-33.

Bohan, C. H., and Davis, O. L., Jr. (1998). Historical constructions: How social studies student

teachers' historical thinking is reflected in their writing of history. Theory and Research in

Social Education, 26, 173-197.

Brophy, J. (1992). Fifth-grade U.S. history: How one teacher arranged to focus on key ideas in

depth. Theory and Research in Social Education, 20, 141-155.

California Department of Education, Educational Demographics Unit (2004b). CBEDS

Professional Assignment Information Form, 2003-04. Retrieved Dec. 21, 2004 from:

http://www.cde.ca.gov/ds/sd/cb/cbedshome.asp.

Council of Chief State School Officers (CCSSO) (2003). State Content Standards: A 50-State

Resource. Washington, D.C.: Author.

Darling-Hammond, L. (1997). Doing what matters most: Investing in quality teaching. New

York: National Commission on Teaching and America’s Future.

Education Week (Dec. 4, 1985). Ravitch: Test shows ‘cultural illiteracy.’ Bethesda, Md.:

Author.

Education Week (2003). Quality Counts 2003: If I can’t learn from you. Bethesda, Md.: Author.

Retrieved May 23, 2003, from: http://www.edweek.org/sreports/qc03.

Evans, R. W. (1988). Lessons from history: Teacher and student conceptions of the meaning of

history. Theory and Research in Social Education, 16, 203-225.

Evans, R. (1989). Teacher conceptions of history. Theory and Research in Social Education,

17, 210-240.

Evans, R. W. (1990). Teacher conceptions of history revisited: Ideology, curriculum, and

student belief. Theory and Research in Social Education, 18, 101-138.

Ferguson, R. (1998). Can schools narrow the black-white test score gap? In C. Jencks and M.

Phillips (eds.), The black-white test score gap. Washington, D.C.: The Brookings Institution.

Finn, C., and Petrilli, M. (eds.) (2000). The state of standards: 2000. Washington, D.C.: Thomas

B. Fordham Foundation. Retrieved June 14, 2003, from:

http://www.fordhamfoundation.org/library/soss2000/2000soss.html#TableB2.
54

Garet, M. S., Birman, B. F., Porter, A. C., Desimone, L., Herman, R., and Yoon, K. S. (1999).

Designing effective professional development: Lessons from the Eisenhower Program.

Washington, D.C.: U.S. Department of Education, Office of the Under Secretary, Planning

and Evaluation Service.

Grant, S. G. (2001). It’s just the facts, or is it? The relationship between teachers’ practices and

students’ understandings of history. Theory and Research in Social Education, 29, 65-108.

Hartzler-Miller, C. (2001). Making sense of “best practice” in teaching history. Theory and

Research in Social Education, 29(4), 672-695.

Hawkins, E., Stancavage, F., Mitchell, J., Goodman, M., and Lazer, S. (1998). Learning about

our world and our past: Using the tools and resources of geography and U.S. history (NCES

98518). Washington, D.C.: National Center for Education Statistics.

Haycock, K. (Summer 1998). Good teaching matters: How well-qualified teachers can close the

gap. Thinking K-16, 3(2), 1-2. Available on the World Wide Web:

http:/www.edtrust.org/K16.pdf.

Hirsch, E., Koppich, J. E., and Knapp, M. S. (1998). What states are doing to improve the

quality of teaching: A brief review of current patterns and trends. Seattle, Wash.: Center for

the Study of Teaching and Policy.

Hoffman, L. (May 2003). Overview of public elementary and secondary schools and districts:

2001-02 (Statistical Analysis Report). Washington, D.C.: U.S. Department of Education,

National Center for Education Statistics (NCES 2003-411). Retrieved Dec. 21 2004, from:

http://nces.ed.gov/pubs2003/overview03/index.asp.

Kennedy, M. M. (1999). Form and substance in mathematics and science professional

development. NISE Brief, 3(2).

Kroesch, G., and Edwards, B. (2000). Getting older and better: Professional development for

experienced teachers. In B. Hanna (ed.), Handbook of history-social science professional

development: From the undergraduate major to the experienced teacher (pp. 55-64).

Oakland, Calif.: University of California Office of the President.

Lapp, M. S., Grigg, W. S., and Tay-Lim, B. S. H. (2002). The nation’s report card: U.S. history

2001 (NCES 2002483). Washington, D.C.: National Center for Education Statistics.

Leming, J., Ellington, L. and Porter, K. (2003). Where did social studies go wrong?

Washington, D.C.: Thomas B. Fordham Foundation.

Loomis, S. C. and Bourque, M. L. (eds.) (July 2001). National Assessment of Educational

Progress achievement levels 1992-1998 for U.S. history. Washington, D. C.: National

Assessment Governing Board, U.S. Department of Education. Retrieved May 12, 2005

from: http://www.nagb.org/pubs/historybook.pdf.

Loucks-Horsley, S. (1999). Research on professional development for teachers of mathematics

and science: The state of the scene. School Science and Mathematics, 99(5), 258-271.

McDiarmid, G. W., and Vinten-Johansen, P. (2002). A catwalk across the great divide. In P.N.

Stearns, P. Seixas, and S. Wineburg, Knowing, teaching, and learning history (pp. 156-177).

New York: New York University Press.
55

Medina, K., Pollard, J., Schneider, D., and Leonhardt, C. (2000). How do students understand

the discipline of history as an outcome of teachers’ professional development? Oakland,

Calif.: Regents of the University of California.

Nash, G., Dunn, R. E., and Crabtree, C. A. (2000). History on trial: Culture wars and the

teaching of the past. New York: Vintage Books.

National Assessment Governing Board (NAGB), U.S. Department of Education (2001). U.S.

history framework for the 1994 and 2001 National Assessment of Educational Progress.

Washington, D.C.: Author.

National Center for Education Statistics (2002). U.S. history highlights 2001 (The nation’s

report card). Washington, D.C.: Author. Retrieved Sept. 17, 2003, from:

http://nces.ed.gov/pubsearch/pubsinfo.asp?pubid=2002482.

National Center for Education Statistics (1997). Good study habits and academic performance:

Findings from the NAEP 1994 U.S. history and geography assessments. Washington, D.C.:

Author. Retrieved Sept. 17, 2003, from: http://nces.ed.gov/pubs97/web/97931.asp.

National Center for Education Statistics (1996a). Results from the NAEP U.S. history

assessment—At a glance. Washington, D.C.: Author. Retrieved Sept. 17, 2003, from:

http://nces.ed.gov/pubsearch/pubsinfo.asp?pubid=96869.

National Center for Education Statistics (1996b). U.S. history: What do students know, and what

can they do? Washington, D.C.: Author. Retrieved Sept. 17, 2003, from:

http://nces.ed.gov/pubs96/web/96895.asp.

National Commission on Teaching and America’s Future (NCTAF) (1996). What matters most:

Teaching for America’s future. New York: Author.

National Council for History Education (NCHE) (2002). Building a history curriculum:

Guidelines for teaching history in schools. Westlake, Ohio: Author.

Podany, A. (2000). Principles of teaching and professional development in history-social

science. In B. Hanna (ed.), Handbook of history-social science professional development:

From the undergraduate major to the experienced teacher (pp. 1-21). Oakland, Calif.:

University of California Office of the President.

Ravitch, D., and Finn, C. E. (1987). What do our 17-year-olds know? A report on the first

national assessment of history and literature. New York: Harper and Row.

Rothman, R. (Sept. 9, 1987). Students’ knowledge of history, literature ‘shameful,’ national

assessment indicates. Bethesda, Md.: Education Week.

Saxe, D. W. (1998). State history standards: An appraisal of history standards in 37 states and

the District of Columbia. Fordham Report, 2(1). Retrieved June 15, 2003, from:

http://www.fordhamfoundation.org/standards/history/history.htm.

Seixas, P. (1998). Student teachers thinking historically. Theory and Research in Social

Education, 26, 310-341.

Seixas, P. (1999). Beyond “content” and “pedagogy”: In search of a way to talk about history

education. Journal of Curriculum Studies, 31, 317-337.
56

Slekar, T. D. (1998). Epistemological entanglements: Preservice elementary school teachers’

“apprenticeship of observation” and the teaching of history. Theory and Research in Social

Education, 26, 485-508.

Smith, J., and Niemi, R. G. (2001). Learning history in school: The impact of course work and

instructional practices on achievement. Theory and Research in Social Education, 29, 18-42.

Stern, S. M. (2003). Effective state standards for U.S. history: A 2003 report card. Washington,

D.C.: The Thomas B. Fordham, Foundation. Retrieved Oct. 6, 2003, from:

http://www.edexcellence.net/institute/publication/publication.cfm?id=320.

St. John, M., Ramage, K., and Stokes, L. (1999). A vision for the teaching of history-social

science: Lessons from the California history-social science project. Inverness, Calif.:

Inverness Research Associates.

Stotsky, S. (2004). The stealth curriculum: Manipulating America’s history teachers. Retrieved

May 1, 2004, from:

http://www.edexcellence.net/institute/publication/publication.cfm?id=331#1040

Thornton, S. J. (1988). Curriculum consonance in United States history classrooms. Journal of

Curriculum and Supervision, 3, 308-320.

U.S. Department of Education. Teaching American history grant program abstracts. Retrieved

Dec. 23, 2004, from: http://www.ed.gov/programs/teachinghistory/awards.html.

U.S. Department of Education, National Center for Education Statistics. Number of public

school districts, by metro status and state, 2001. Washington, D. C.: Author. Retrieved Dec.

23, 2004, from http://nces.ed.gov/surveys/ruraled/data/SDU_Metro.asp.

U.S. Department of Education, National Center for Education Statistics (May 2003). Schools

and staffing survey (SASS) for 1999-2000 (revised May 2003). Washington, D. C.: Author.

U.S. Department of Education, National Center for Education Statistics, Common Core of Data

(CCD) (2002-2003). CCD public school district data for the 2002-2003 school year. Search

for public school districts. Washington, D. C.: Author. Retrieved Dec. 21, 2004, from:

http://nces.ed.gov/ccd/districtsearch/index.asp.

U.S. Department of Education (April 2002). Teaching American history grant program; Notice

inviting grant applications for new awards for fiscal year (FY) 2002. Washington, D.C.:

Author. Retrieved Dec. 23, 2004, from:

http://www.ed.gov/legislation/FedRegister/announcements/2002-2/040302b.html.

U.S. Department of Education (May 2001). Teaching American history grant program: Notice

inviting grant applications for new awards for fiscal year (FY) 2001. Washington, D.C.:

Author. Retrieved Dec. 23, 2004 from:

http://www.ed.gov/legislation/FedRegister/announcements/2001-2/052301d.html.

VanSledright, B. (2002). In search of America’s past: Learning to read history in elementary

school. New York: Teachers College Press.

Vinten-Johansen, P., and McDiarmid, G. W. (1997). Stalking the schoolwork module: Teaching

prospective teachers to write historical narratives. East Lansing, Mich.: National Center for

Research on Teacher Learning. Retrieved June 3, 2003, from:

http://ncrtl.msu.edu/http/rreports/html/pdf/Rr9702.pdf.
57

Williams, P. L., Lazer, S., Reese, C. M., and Carr, P. (1995). NAEP 1994 U.S. History: A first

look (Findings from the National Assessment of Educational Progress). Washington, D.C.:

National Center for Education Statistics. Retrieved Sept. 17, 2003, from:

http://nces.ed.gov/nationsreportcard/pubs/main1994/hhigh.asp.

Wilson, S. (2001). Research on history teaching. In V. Richardson (ed.), Handbook of research

on teaching (4th ed., pp. 527-544). Washington, D.C.: American Educational Research

Association.

Wilson, S. M., Floden, R. E., and Ferrini-Mundy, J. (2001). Teacher preparation research:

Current knowledge, gaps, and recommendations. Seattle, Wash.: Center for the Study of

Teaching and Policy. Retrieved June 4, 2003, from:

http://depts.washington.edu/ctpmail/Reports.html#TeacherPrep.

Wineburg, S., and Wilson, S. (2001a). “Models of wisdom in the teaching of history.” In S.

Wineburg, Historical thinking and other unnatural acts: Charting the future of teaching the

past. Philadelphia: Temple University Press.

Wineburg, S., and Wilson, S. (2001b). “Peering at history through different lenses: The role of

disciplinary perspectives in teaching history.” In S. Wineburg, Historical thinking and other

unnatural acts: Charting the future of teaching the past. Philadelphia: Temple University

Press.

Wineburg, S., and Wilson, S. (2001c). “Wrinkles in time and place: Using performance

assessments to understand the knowledge of history teachers.” In S. Wineburg, Historical

thinking and other unnatural acts: Charting the future of teaching the past. Philadelphia:

Temple University Press.

A-1

Appendix A

Additional Information

A-3

_ Chapter One

The NAEP Exam in American History

Following the 1988 American history NAEP exam, the National Assessment Governing

Board (NAGB) altered the assessment’s framework (Williams et al., 1995). It currently includes

three features: (1) themes in American history, (2) periods of American history, and (3) ways of

knowing and thinking about American history (divided into historical knowledge and

perspective, and historical analysis and interpretation) (Hawkins et al., 1998). Unlike previous

versions of the American history NAEP exam, 56 percent of the questions in the 1994 exam

assessed students’ skills in using “stimuli” such as maps and primary source documents. Student

scores on the NAEP exam in American history are on a scale of 500 and rated according to three

levels: basic, proficient, and advanced. A NAEP score on the “basic” level indicates “partial

mastery of the prerequisite knowledge and skills fundamental for proficient work.” A score of

“proficient” indicates “solid academic performance” and “demonstrated competence.” An

“advanced” score on NAEP indicates “superior” performance (National Center for Education

Statistics, 1996a).14

NAEP Results by Student Race/Ethnicity

As Exhibit A-1 illustrates, at all grades, white students outperformed their black and

Hispanic peers in 1994 and 2001.

 14 Please note that the NAEP American history scores from 1994 and 2001 are not directly comparable with scores

from the NAEP assessments of the 1980s because the American history assessment was substantially revised

before the 1994 administration of the test (National Center for Education Statistics, 1996b).
A-4

Exhibit A-1

NAEP American History Assessment Results, by Student Race/Ethnicity

1994 and 2001

Percentage of Students Scoring at or Above Basic Level

White Black Hispanic

1994 2001 1994 2001 1994 2001

Grade 4 74 79 36 44 41 42

Grade 8 71 75 33 38 41 40

Grade 12 50 49 17 20 22 26

Source: National Center for Education Statistics (2002).

Exhibit reads: Seventy-four percent of white students in grade 4 scored at or above basic level on

the 1994 NAEP American History assessment.

Exhibit A-2 presents the evaluation’s research questions and data sources.
Exhibit A-2 – Research Questions and Data Sources

Document

Review

Literature

Review

Project Director

Survey

Teacher

Survey

Project

Materials

Case

Studies

s and grantee activities

TAH grantees implementing? P P S

 being implemented that are based on research in

nd teacher training? S P P P S

pecific models of professional development and

 history? S P P P

 to implement the activities, and what are the

H projects to enhance their scope and quality? P P S P

ee partnering relationships? P S S P

haracteristics and other background features of the

S P

o provide teachers with professional development in

ant period has ended? P P

 emphasizing American history content? S P P P S

 emphasizing history methodology and processes? S P P P S emphasizing pedagogical skills and pedagogical

S P P P S

eas of history are being taught to teachers? S S P S

s are being covered in the training? S S P S

 of the content training? S S P S

ning support the NAEP U.S. History standards and

S S P S

standards and assessments in American history

ow do TAH projects support such standards and

S S P S

pport and coordinate with other federal programs? P P

/qualifications of participating teachers? How do

S teacher sample in respect to degrees, certification,

S P

o the teacher participants teach? P

 more than one type of activity with TAH funds,

of teachers participating in each type of activity? P P

en to participate in the projects? S P P

rojects helping preservice teachers, new teachers,

S S P S

qualified and the most qualified teachers participate? S P

rticipate? S P

ntive to participate? S P P

enting activities to ensure that all teachers in the

(according to NCLB)? S P P

 of teachers who are highly qualified a project goal? S P P

e in project activities doing anything different in their

eir participation? S P S P

ndary source.
A-6

Exhibit A-3 lists the availability of history and/or social studies content standards by state.

Exhibit A-3

State History and Social Studies Standardsa

State History Standards Social Studies Standards Year Adopted

Alabama  1998

Alaska  1995

Arizona  2000

Arkansas  (state history)  2000

California  1998

Colorado  1995

Connecticut  1998

Delaware  2001

Florida  NA

Georgia  NA

Hawaii  1999

Idaho  2003

Illinois  1997

Indiana  2001

Iowa ---- ----

Kansas  1999

Kentucky  1999

Louisiana  1997

Maine  1997

Maryland  2001

Massachusetts  (H/SSt) 2002

Michigan  1996

Minnesota  1999

Mississippi  2000

Missouri  1999

Montana  2000

Nebraska  2002

Nevada  2000

New Hampshire  1996

New Jersey  1996

New Mexico  2001

New York  1996

North Carolina  1997

North Dakota  2000

Ohio  2002

Oklahoma  2002

 

a This chart is compiled from data gathered by the Council of Chief State School Officers’ State Content Standards:

A 50-State Resource (2003), which does not provide data for the District of Columbia.
A-7

Exhibit A-3 (concluded)

State History and Social Studies Standards

State History Standards Social Studies Standards Year Adopted

Oregon  2001

Pennsylvania  2003

Rhode Island  2001

South Carolina  2000

South Dakota ---- ----

Tennessee  2001

Texas  2000

Utah  2000/2002

Vermont  2000

Virginia  (H/SSt) 2001

Washington  2002

West Virginia  NA

Wisconsin  1998

Wyoming  1999

Total 8 40

Source: CCSSO (2003).

Exhibit reads: The state of Alabama has content standards in social studies, which were adopted

in 1998.

_ Chapter Two

Exhibit A-4 lists the district, location, participant numbers, and funding amount of 2001 TAH

grantees.
A-8

Exhibit A-4

Teaching American History Grant Recipients for Fiscal Year 2001

District City State

Number of

Participants Funding

Southeast Island School District Thorne Bay AK -- $597,135

Calhoun County Board of Ed Anniston AL 18 $523,977

Phoenix Union High School District Phoenix AZ 5 teacher

teams $675,127

Baldwin Park Unified School District Baldwin Park CA 138 $783,786

Los Angeles County Office of Education Downey CA 96 $999,000

Glendale Unified School District Glendale CA 25 $1,000,000

Long Beach Unified School District Long Beach CA 45 $804,666

Los Angeles Unified School District Los Angeles CA 400 $1,000,000

Montebello Unified School District Montebello CA 90 $805,295

Oakland Unified School District Oakland CA 90 $999,238

Jefferson County Public Schools Golden CO 360 $864,977

Bridgeport Public Schools Bridgeport CT -- $724,024

The School Board of Broward County Ft.

Lauderdale FL -- $999,326

Hillsborough County Public Schools Tampa FL 120 $999,734

The DeKalb County School System Decatur GA -- $538,835

Washington Community School District Washington IA 29 $701,133

West Central Four Regional Office of

Education Macomb IL -- $608,286

Lincoln-Way High School District #210 New Lenox IL 300 $956,576

Community Unit School District 60 Waukegan IL 40 $921,966

Unified School District #499 Galena KS -- $633,327

East Baton Rouge Parish School Board Baton Rouge LA -- $993,595

Calcasieu Parish School Board Lake Charles LA -- $618,762

Assumption Parish School Board Napoleonville LA -- $842,644

Boston Public Schools Boston MA 326 $999,187

Mohawk Trail Rural School District Shelburne

Falls MA 225 $999,239

Montgomery County Public Schools Rockville MD 160 $997,920

Baltimore County Public Schools Baltimore MD -- $997,354

Minneapolis Public Schools Minneapolis MN 240 $999,598

School District of Springfield R-12 Springfield MO 30 $854,917

Ritenour School District St. Louis MO -- $911,531

Charlotte-Mecklenburg Schools Charlotte NC -- $994,525

Educational Service Unit 7 Columbus NE 40 $931,935

Somersworth School District Somersworth NH 57 $645,987

West Morris Regional High School District Chester NJ 180 $553,785

Washoe County School District Reno NV -- $893,932

Broome-Tioga BOCES Binghamton NY 100 $814,171
A-9

Exhibit A-4 (concluded)

Teaching American History Grant Recipients for Fiscal Year 2001

District City State

Number of

Participants Funding

New York City Board of Education, Office

of Multicultural Education Brooklyn NY -- $955,584

Monroe Board of Cooperative Educational

Services (BOCES) Fairport NY 120 $921,207

Community School District #30 Jackson

Heights NY 300 $998,400

Office of the Superintendent for

Alternative, Adult and Continuing

Education

Long Island

City NY -- $996,933

Community School District #1 New York NY 600 $872,850

Community School District #4 New York NY -- $498,306

Mansfield City School District Mansfield OH 35 $890,440

Coalgate Public Schools Coalgate OK -- $386,762

Lane Education Service District Eugene OR 80 $798,089

School District of Philadelphia Philadelphia PA -- $999,480

School District of Pittsburgh Pittsburgh PA -- $457,395

Ridgeway Area School District Ridgeway PA 30 $620,965

Capital Area Intermediate Unit 15 Summerdale PA 150 $940,358

Richland School District Two Columbia SC -- $953,361

Pine Ridge School Pine Ridge SD 60 $520,313

Houston Independent School District Houston TX -- $786,025

Region IV Education Service Center Houston TX -- $988,290

San Antonio Independent School District San Antonio TX 245 $999,691

Iron School District Cedar City UT 30 $700,623

Logan City School District Logan UT -- $999,954

Educational Service District 101 Spokane WA -- $529,641

School District of La Crosse La Crosse WI 20 $902,083

Wausau School District Wausau WI 180 $784,370

Regional Education Service Agency III Dunbar WV 120 $949,921

Note: -- Indicates data were not available for these grantees

Source: U.S. Department of Education, Teaching American History Grant Abstracts.

Exhibit reads: Southeast Island School District received a Teaching American History Grant of

$597,135 in FY 2001.

Exhibit A-5 lists the district, location, participant numbers, and funding amount of 2002 TAH

grantees.
A-10

Exhibit A-5

Teaching American History Grant Recipients for Fiscal Year 2002

District City State Number of

Participants Funding

Anchorage School District Anchorage AK 40 $772,053

Madison County Board of Education Huntsville AL 90 $744,596

City of Opelika Board of Education Opelika AL 102 $999,944

Little Rock School District Little Rock AR 123 $995,953

Page Unified School District Page AZ -- $852,974

San Juan Unified School District Carmichael CA 60 $919,014

Clovis Unified School District Clovis CA 60 $995,400

Lawndale Elementary School District Lawndale CA 120 $991,670

Los Angeles Unified School District J Los Angeles CA 240 $1,000,000

Northern Humboldt Union High

School District McKinleyville CA 80 $651,142

Grant Joint Union High School District Sacramento CA 65 $1,000,000

San Diego Unified School District San Diego CA 90 $525,527

Santa Ana Unified School District Santa Ana CA -- $879,925

San Luis Valley Board of Cooperative

Services Alamosa CO 120 $700,092

School District #1 in the City and

County of Denver Denver CO 205 $983,348

Adams County School District #12 Northglenn CO -- $999,518

Hartford Public Schools Hartford CT 52 $991,063

Stratford, Conn., Board of Education Stratford CT 50 $830,790

District of Columbia Public Schools Washington DC 135 $997,959

Indian River School District Selbyville DE 150 $947,547

Polk County School Board Bartow FL 270 $908,214

School Board of Orange County Orlando FL 120 $999,948

Richmond County Board of Education Augusta GA 171 $881,486

Coastal Plains Region Education

Service Agency Lenox GA 45 $806,715

Savannah-Chatham County Public

Schools Savannah GA -- $1,000,000

Hawaii State Department of

Education Honolulu HI 391 $991,056

Mississippi Bend Area Education

Agency Bettendorf IA 70 $708,370

Chicago Public Schools Chicago IL 84 $976,445

Chanute Public Schools Chanute KS 36 $861,021

Ballard County Board of Education Barlow KY 45 $933,705

Harlan Independent School District Harlan KY 45 $942,408

Bourbon County Schools Paris KY 95 $929,811

Ohio Valley Educational Cooperative Shelbyville KY 55 $972,500

Brookline Public Schools Brookline MA 72 $726,493
A-11

Exhibit A-5 (continued)

Teaching American History Grant Recipients for Fiscal Year 2002

District City State Number of

Participants Funding

Fall River Public Schools Fall River MA 200 $970,555

Hudson Public Schools Hudson MA 200 $910,493

Lowell Public Schools Lowell MA 100 $961,600

Plymouth Public School District Plymouth MA 80 $751,771

Springfield Public Schools Springfield MA -- $919,458

Baltimore City Public School System Baltimore MD 108 $942,702

Menominee County Intermediate

School District Menominee MI -- $648,064

Carl Junction R-1 Schools Carl Junction MO 75 $772,053

Winona R-III School District Winona MO 75 $19,561

Hattiesburg Public School District Hattiesburg MS 36 $858,139

Starkville School District Starkville MS 30 $1,000,000

Winona Separate School District Winona MS 25 $525,532

Anaconda School District No. 10 Anaconda MT 90 $757,611

Bozeman School District #7 Bozeman MT 36 $992,106

Missoula County Public Schools Missoula MT 38 $492,524

Shelby Public Schools Shelby MT 80 $997,896

Charlotte-Mecklenberg Schools Charlotte NC 67 $970,045

Durham Public Schools Durham NC 70 $885,434

North Carolina School of Science and

Mathematics Durham NC 160 $996,267

Cumberland County Schools Fayetteville NC 127 $915,057

Pitt County Schools Greenville NC 60 $940,705

Roanoke Rapids Graded Schools Roanoke

Rapids NC 120 $830,025

Grand Forks Public School District #1 Grand Forks ND 310 $940,096

Educational Service Unit #2 Fremont NE 50 $845,484

Lincoln Public Schools Lincoln NE -- $970,260

Bayonne Board of Education Bayonne NJ 65 $470,857

Gloucester City Public Schools Gloucester NJ 75 $405,837

Montville Township Board of

Education Montville NJ 120 $982,130

Bergen County Technical Schools Paramus NJ 172 $979,141

Albany, Schoharie, Schenectady,

Saratoga BOCES Albany NY 1,500 $909,966

Community School District 8 Bronx NY -- $997,660

Community School District 10 Bronx NY -- $999,850

Community School District 15 Brooklyn NY -- $999,936

Community School District 16 Brooklyn NY -- $1,000,000

Community School District 18 Brooklyn NY -- 993,323
A-12

Exhibit A-5 (continued)

Teaching American History Grant Recipients for Fiscal Year 2002

District City State Number of

Participants Funding

Jamestown Public School District Jamestown NY 100 741,804

Office of the Superintendent, Queen's

High Schools Jamestown NY -- $722,168

New York City Board of

Education/Manhattan High Schools

Superintendent's Office

New York NY -- $998,483

Dutchess County Board of

Cooperative Poughkeepsie NY 250 $837,486

Yonkers Public Schools Yonkers NY 300 $930,000

Columbus Public Schools Columbus OH 105 $992,897

Dayton Public Schools Dayton OH 80 $915,615

Euclid City School Districts Euclid OH 60 $976,919

Fremont City Schools Fremont OH 450 $884,891

Lorain City School District Lorain OH 315 $956,963

Mahoning County Educational

Service Center Youngstown OH 100 $834,687

Stratford Public Schools Stratford OK 22 $864,592

Vinita Public Schools Vinita OK 8 $640,302

Linn-Benton Lincoln Education

Service District Albany OR 70 $1,000,000

School District No. 1, Multnomah

County Portland OR 75 $816,952

Central Susquehanna Intermediate

Unit Lewisburg PA 45 $1,000,000

School District of Philadelphia Philadelphia PA 75 $919,908

St. Mary's Area School District St. Mary's PA 25 $567,662

Burrillville School Department Pascoag RI 60 $500,682

Warren County Schools McMinnville TN 100 $936,452

Memphis City Schools Memphis TN 80 $983,227

Region V Education Service Center Beaumont TX 190 $774,697

Education Service Center, Region 2 Corpus Christi TX 110 $996,965

Dallas Independent School District Dallas TX 200 $996,893

The North Hills School Irving TX 120 $644,000

Fort Worth Independent School

District Fort Worth TX 90 $995,191

Region IV Education Service Center Houston TX 300 $970,086

Victoria Independent School District Victoria City TX 36 $656,151

Weslaco Independent School District Weslaco TX -- $756,939

Weber School District Ogden UT 260 $995,860

Granite School District Salt Lake City UT 138 $949,984

Fairfax County Public Schools Fairfax VA 120 $987,585

Rockbridge County Public Schools Lexington VA 20-54 $797,927
A-13

Exhibit A-5 (concluded)

Teaching American History Grant Recipients for Fiscal Year 2002

District City State Number of

Participants Funding

Newport News Public Schools Newport

News VA 84 $565,494

Franklin County Public Schools Rocky Mount VA -- $457,450

Russell County Public Schools Lebanon VA 100 $1,000,000

Virginia Beach City Public Schools Virginia

Beach VA 189 $984,161

Williamsburg-James City County

Public Schools Williamsburg VA 600 $1,000,000

Caledonia Central Supervisory Union Danville VT 150 $993,923

Educational Service District 101 Spokane WA 140 $745,311

Educational Service District 112 Vancouver WA 744 $994,611

Cooperative Educational Service

Agency #12 Ashland WI 26 $876,372

Cooperative Educational Service

Agency #10

Chippewa

Falls WI 220 $999,822

Cooperative Educational Service

Agency #7 Green Bay WI 210 $822,146

Regional Education Services Agency Beckley WV -- $988,818

Note: -- Indicates data were not available for these grantees

Source: U.S. Department of Education, Teaching American History Grant Abstracts.

Exhibit reads: Anchorage School District received a Teaching American History Grant of

$772,053 in FY 2002.

B-1

Appendix B

Methodology, Data Collection Strategies, and Procedures

In this section, we provide an overview of the methodology used for this evaluation,

including our sampling strategy and data collection efforts for the project director and participant

surveys, the procedures used for the selection of case studies and review of teacher materials,

and the approach to data analysis.

B-3

_ Project Director Survey

After OMB clearance in October 2003, a notification letter announcing the project director

survey was mailed to the entire universe of directors of Teaching American History grants

awarded in FY 2001 and FY 2002 (N= 174). The survey packet, consisting of a cover letter and

the 37-item pencil and paper survey, was mailed in November 2003 (see Appendix C). A

toll-free phone number was established for project directors to call if they had questions or

concerns regarding the survey instrument, and was in place throughout the data collection

process. Following standard procedure for survey administration, SRI mailed reminder

postcards approximately one month after the initial mailing of the survey. These postcards urged

those who had not yet responded to complete the survey, and provided contact information for

questions or for an additional copy of the instrument.

Shortly after the postcard reminder, in mid-January, SRI mailed a letter from the Department

of Education, again urging directors to respond to the survey. In addition, this letter requested

the submission of a list of all those who had participated in the project, from inception through

December 2003. These lists became the basis for establishing the universe of TAH participants

(see below). Finally, a second mailing of the survey was completed in late January. SRI closed

out data collection for the project director survey in mid-March 2004. Surveys were received

from 150 of the 174 project directors, for a response rate of 86 percent.

_ Participant survey

Requests for Participant Rosters

While data collection for the project director survey was underway in fall 2003, SRI began to

collect rosters of TAH participants. SRI asked each project directors to provide a list of only

those participants who had received training through their TAH grant from its inception through

December 2003. (This request was deemed less burdensome than asking project directors to

specify exact dates of involvement for participants, in light of the fact that many projects have

grown in size or changed participants during the course of their grants.) Throughout the process,

SRI communicated with project directors regularly about the nature of this request. In midJanuary, SRI mailed a letter from the Department of Education that reminded project directors of

the importance of this request to the overall evaluation.

As the rosters came in, SRI entered the lists of participants into a FileMaker database. SRI

ended roster collection in late February and finalized the participant database in March. The

final participant database included a total of 9,487 participants that participated in TAH projects
B-4

through December 2003.15 This information was drawn from 158 programs,16 yielding a 91

percent response rate for roster collection.

Construction of the Participant Sample

Early analysis of the TAH project resulted in the recognition that, assuming comparable grant

amounts, projects that train large numbers of participants versus those that train smaller numbers

of participants approach professional development differently. With that in mind, a stratified

sampling strategy for participants was developed that created four groups (or strata) of projects,

defined by four levels of expenditures per teacher-participant. This construct of “program

intensity” was calculated by dividing grant amount by number of participants (as listed on

project rosters). The 174 projects were then grouped into four ordered quartiles of program

intensity. Thus, the first group or strata typically included projects with higher numbers of

participants—and lower “intensity” per participant, while the fourth group included projects with

the lowest number of participants, and highest “intensity” per participant.

From within each of the four strata, SRI selected two projects for detailed investigation,

which became the case study sites (see below). All participants from these eight case study sites

were purposively included in the sample. An additional number of names were randomly

selected from each stratum, to bring the total number sampled from each stratum to 550, for a

total sample n of 2,200.

Data Collection for the Participant Survey

Letters announcing the participant survey were mailed to the 2,200 individuals selected for

the participant sample in late March 2004. The survey packets, consisting of a cover letter

clarifying the purpose and estimated time required for response, along with the 44-item pencil

and paper survey were mailed immediately thereafter (see Appendix D). SRI sent postcard

reminders to participants in mid-April. The second mailing of the survey packet was sent to

participants at the end of April.

An additional effort was made to ensure responses from participants in the programs that had

been selected for case study. Approximately one month later, in mid-May, SRI asked project

directors in case study sites to elicit participants’ cooperation in the data collection through

 15 The total number of participants yielded from the roster collection is understandably less than the total number of

projected participants in Exhibit 3-2. This is because rosters provided by project directors listed only those

participants from the beginning of their grants up through 2003, while Exhibit 3-2 lists the total number of

participants project over the grant’s three-year cycle. 16 Because we did not receive rosters from 16 programs, participants from these programs were not able to be

included in the universe of participants for the participant sample.
B-5

e-mail, online discussions, Web sites, or any other method available. In late June, SRI made

another appeal to participants by sending e-mail reminders to all non-respondents. SRI sent a

third mailing of the survey to those respondents who said that they had lost or misplaced the

survey. Throughout the data collection process for the participant survey, SRI maintained a tollfree phone number for participants to call with questions or concerns. SRI closed out data

collection on the participant survey in August 2004, with a final response rate of 55 percent

(respondent n = 1,185, which included eligible and noneligible respondents; see section on data

analysis below).

_ Case Study Data Collection

As detailed above, SRI’s sampling plan for the participant survey included dividing the

projects into four strata based on the projects’ expenditure per participant, and then randomly

selecting two projects from each stratum for case study. Teams of two SRI researchers

conducted site visits, which lasted for two days. While on site, SRI researchers interviewed

project directors and observed training (in the form of summer institutes) that was offered to

teachers through the Teaching American History program. These site visits, which were

conducted from the middle of June 2004 through the end of August 2004, provided SRI with rich

information on such issues as the depth and breadth of training provided to teachers, the roles

and responsibilities of partners, and the structure and organization of project activities.

Teacher Materials Review: A Pilot Study

In addition to the site visits, an in-depth review and assessment of teacher work products was

conducted for a sample of participants from each case study project. SRI conducted a pilot study

in summer 2003 to determine the feasibility of this task. We contacted five project directors and

asked them to submit sample materials from their participants. Based on follow-up

conversations with these individuals, we revised our request procedures.

For the actual review, SRI assembled and analyzed 44 teacher-produced materials from six of

the eight case study sites. Project directors were asked to identify nine individual teachers or

groups of teachers to participate; they were asked to choose nine representative teachers (see

Appendix E). That is, project directors were not to select teachers who were the best or the worst

in their projects. To be selected for this process, teachers had to have been participating in their

TAH project for at least a year. Submitted materials included lesson plans, unit plans, research

papers, and book reviews. Not all sites sent in nine sets of materials (see Exhibit B-1).
B-6

Exhibit B-1

Teacher Work Products Submitted

Site Number of Teacher Work Products Submitted

Site One 9

Site Two 9

Site Three 9

Site Four 8

Site Five 6

Site Six 4

Source: SRI International, Exploratory Study of Teacher Materials.

Exhibit reads: Site one submitted nine teacher work products to SRI.

To ensure that the evaluation had the proper balance of historical expertise and teaching

experience, SRI recruited a group of five experienced elementary and secondary history teachers,

one assessment expert, one historian, and three SRI staff members to evaluate the materials.

Two of the team members have Ph.D.s in history and one has an M.A. in the subject; another has

a Ph.D. in history education; one other has a Ph.D. in education history.

During three days in early August 2004, the group trained and carried out the evaluation. The

team spent the first day refining the assessment tool and conducting a whole group analysis of

two lesson plans. The group used a three-point scale rubric, based on the U.S. history framework

for the 1994 and 2001 National Assessment of Educational Progress (2001), and created by SRI

and the assessment expert (see Appendix F).17 The NAEP framework divides the “Ways of

Knowing and Thinking” into two cognitive domains: Historical Knowledge and Perspective and

Historical Analysis and Interpretation. Each domain is then subdivided into the following skills:

_ Historical Knowledge and Perspective

_ Demonstrates knowledge and understanding of people, events, concepts, themes,

movements, contexts, and historical sources

_ Sequences historical events

_ Recognizes multiple perspectives

_ Sees an era or movement through the eyes of different groups

_ Develops a general conception of U.S. history

 17 Note that the rubric developed by SRI did not include the NAEP-based skill entitled “finds value statements”

under the second cognitive domain because the evaluation team deemed it too ambiguous.
B-7

_ Historical Analysis and Interpretation

_ Explains historical issues

_ Identifies historical patterns

_ Establishes cause-and-effect relationships

_ Establishes significance

_ Applies historical knowledge

_ Weighs evidence to draw sound conclusions

_ Makes defensible generalizations

_ Renders insightful accounts of the past

After each review, the team discussed which features of the rubric worked and which did not.

This activity primarily involved changing the language in some of the assessment categories. In

addition to refining the rubric, these initial reviews also allowed the team to develop some

normative language for discussing and evaluating the materials.

Over the following two days, the eleven members were divided into three groups, split to

ensure that historical expertise and school level were represented in each. The assignments were

randomly distributed to the groups and each group reviewed a different set of materials from

each site. Group consensus was used to determine the final scores for each work product.

_ Survey Data Analysis

Weighting of Participant Sample

The first step in weighting of the participant sample was to determine the number of

“ineligible” names from those sampled. An initial screening item on the participant survey asked

potential respondents18 if they had participated in TAH professional development during the

period of interest, June 1, 2002, to Sept. 20, 2003. Those who had not were asked to indicate as

such and return the non-completed survey to SRI. Of the 1,185 responses (from sample n =

2,200), 209 respondents indicated they had not participated. This group was considered to

represent the population of “non-eligible” names on the project rosters overall.

Weights were assigned to all 1,185 respondents, those eligible (n = 976) and those noneligible (n = 209). From this process, the estimate of the universe of TAH participants between

June 1, 2002, and Sept. 20, 2003 was obtained, a population N of 7,774. As per standard

 18 As explained earlier, the universe of participants for this evaluation was defined as those participants who

received training in a TAH project from its beginning through December 2003.
B-8

weighting procedures, respondent weights were assigned based on the extent to which a given

response represented responses for all participants in a given project.

Data Analysis

For both surveys, responses were scanned into data files via the Cardiff system. Data

analysis was conducted using SAS statistical software. For the project director survey, a

preliminary analysis was conducted to determine if there were any systematic differences on

survey variables between FY 2001 and FY 2002 programs. Analyses were conducted on the

entire set of 150 respondents once it was determined that this was not the case.

Analyses for the participant survey were conducted on both the unweighted sample (n = 976)

and weighted estimate of the universe of participants (n = 7,774). The weighted frequencies can

be viewed as indicative of the population of participants during the study period.
C-1

Appendix C

Teaching American History Project Director Survey

This appendix contains the project director survey sent to TAH project directors.

C-3
C-4
C-5
C-6
C-7
C-8
C-9
C-10
C-11
C-12
C-13
C-14
C-15
C-16
C-17
C-18
C-19
C-20
C-21
C-22
C-23
C-24
C-25
C-26
C-27
C-28
C-29
C-30
D-1

Appendix D

Teaching American History Project Participant Survey

This appendix contains the participant survey sent to TAH participants.

D-3
D-4
D-5
D-6
D-7
D-8
D-9
D-10
D-11
D-12
D-13
D-14
D-15
D-16
D-17
D-18
D-19
D-20
D-21
D-22
D-23
D-24
D-25
D-26
E-1

Appendix E

Teaching American History Project Materials Request Form

This appendix contains the materials request form sent to TAH project directors as part of the

materials review component of this evaluation.

E-3
E-4
E-5
E-6
E-7
E-8
E-9

F-1

Appendix F

Teaching American History Project NAEP-Based Materials Review Rubric

This appendix contains the rubric used during the materials review component of this

evaluation.

F-3

TEACHING AMERICAN HISTORY

NAEP-BASED MATERIALS REVIEW RUBRIC

Work Product Code _____________________________

Section 1: Historical Knowledge and Perspective

The teacher work product:

1.A. Demonstrates knowledge and understanding of people, events, concepts, themes,

movements, contexts, and historical sources

1.A.1 Insufficient Evidence

1. has many factual errors 2. has some incorrect factual

information

3. is factually accurate

1.A.2. Insufficient Evidence

1. knowledge is of

superficial depth and

incomplete breadth

2. knowledge is in some

depth or in breadth

3. knowledge is of perceptive

depth and comprehensive

breadth

1.A.3. Insufficient Evidence 

1. doesn’t use historical

themes or incorrectly

defines historical themes

and incorrectly relates

them to specific factual

information.

2. partially defines historical

themes and/or somewhat

relates them to specific

factual information.

3. clearly defines historical

themes and accurately

relates them to specific

factual information.

1.B. Sequences historical events

1.B.1 Insufficient Evidence 

1. correctly places few

specifics in historical

chronology

2. correctly places some

specifics in historical

chronology

3. clearly and correctly places

most specifics in historical

chronology

1.B.2 Insufficient Evidence 

1. inaccurately constructs

or labels historical

periods

2. somewhat accurately

constructs and/or labels

historical periods

3. accurately constructs and

labels historical periods.
F-4

1.C.1. Recognizes multiple perspectives

Insufficient Evidence 

1. has one limited

historical interpretation

2. has an overemphasis

on one historical

perspective

and/or an under emphasis

of at least one significant

historical interpretation

3. comprehensively describes

the past from balanced

historical interpretation

1.C.2 Sees an era or movement through the eyes of different groups.

Insufficient Evidence 

1. no description of the past

from the viewpoints of

participating individuals

and/or groups

2. limited description of the

past from viewpoints of

participating individuals

and/or groups

3. comprehensively describes

the past from the viewpoints

participating individuals

and/or groups

1.D. Develops a general conception of U.S. history

Insufficient Evidence 

1. does not presents

meaningful topics within

the context of U.S.

history through historical

sources

2. partially presents

meaningful topics within

the context of U.S. history

through historical sources

3. effectively presents

meaningful topics within the

context of U.S. history

through historical sources

Section 2: Historical Analysis and Interpretation

The work product:

2.A. Explains historical issues

Insufficient Evidence 

1. presents no analysis or

superficial analysis of the

points of view, biases,

and value statements

about historical issues

2. presents some analysis of

points of view, biases, and

value statements about

historical issues

3. presents comprehensive

analysis of points of view,

biases, and value statements

about historical issues

2.B. Identifies historical patterns

Insufficient Evidence 

1. presents no ways to

organize information into

historical patterns

2. presents some ways to

organize information into

historical patterns

3. presents significant ways to

organize information into

historical patterns
F-5

2.C. Establishes cause-and-effect relationships

Insufficient Evidence 

1. does not identify key

cause-effect relationships

or makes vague, simple,

and/or implausible

explanations of historical

causes and effects

2. identifies key historical

causes and effects, but the

explanations are not as

clear, detailed, or

comprehensive as possible

3. identifies clear explanations

of key historical causes and

effects

2.D. Establishes significance

Insufficient Evidence 

1. does not address the

important people, events,

themes, and contexts and

fails to establish the

significance of the

historical topic

2. addresses some of the

important people, events,

themes, and contexts and

does not completely

establish the significance

of the historical topic

3. addresses most of the

important people, events,

themes, and contexts that

establish the significance of

the historical topic

2.E. Applies historical knowledge

Insufficient Evidence 

1. makes weak arguments

based on little evidence

2. makes some good

arguments based on

somewhat convincing

evidence

3. makes a consistent argument

based on convincing

evidence

2.F. Weighs evidence to draw sound conclusions

Insufficient Evidence 

1. uses too few or

inappropriate primary

and secondary sources to

assess different views

and to draw conclusions

2. uses some appropriate

primary and secondary

sources to assess different

views and to draw

conclusions

3. uses a variety of

appropriate primary and

secondary sources to assess

different views and to draw

conclusions

2.G. Makes defensible generalizations

Insufficient Evidence 

1. reflects an inability to

compose and support

historical generalizations

2. reflects a limited ability to

compose and/or support

historical generalizations

3. reflects the ability to

compose and support

historical generalizations

2.H. Renders insightful accounts of the past

Insufficient Evidence 

1. provides a simplistic

account of the of the past

2. provides a descriptive

account of the past

3. provides a clear and

perceptive account of the

past
Our mission is to ensure equal access to education

and to promote educational excellence throughout the nation.

www.ed.gov