**TEACHING AMERICAN HISTORY EVALUATION**

**SUMMER 2009 SITE VISITS**

**Project Director Protocol**

  -----------------------------------------------------------------------
  **A. Background and Work Responsibilities**

  -----------------------------------------------------------------------

1\. Please tell me about your background. How long you have been the
project director of this grant? Have you been the only project director?
What do you consider to be your primary responsibilities associated with
this project (e.g., administrative, recruitment, training-related,
accountability)?

2\. Do you have other work responsibilities in addition to directing
this TAH project? What did you do before directing the TAH project?

  -----------------------------------------------------------------------
  **B. Planning and Goals of the TAH Project**

  -----------------------------------------------------------------------

3\. Why did you (or the grantee) decide to apply for this grant?

4\. How did you determine the need for this project prior to designing
the program?

What input was solicited (e.g. teachers, history experts,
administrators, research on professional development in history)?

5\. Were there any prior grants or initiatives in this district(s)
relating to teaching American history, history-related curriculum
development, or to other professional development in history-related
content?

6\. What are the overall goals for the project? How was the project
designed to meet those goals? (Probe for district, state, federal goals,
use of assessment data for decision-making, relevancy of improving
teacher content knowledge/student achievement.)

7\. How did you decide what schools or districts to involve in the
project and what grades to include?

8\. Does the district(s) administer a state or district-wide assessment
of American history? How long has the assessment been administered? Can
you describe the test? Does it include essay questions/writing? To what
extent was this a factor in planning the TAH project and evaluation
activities?

9\. How long has the project been in operation? Describe the
implementation of the grant: How did you identify partners? When did
training begin? What kinds of supports and barriers did you encounter in
planning the project?

10\. How have your goals and implementation activities changed over
time?

  -----------------------------------------------------------------------
  **C. Partners**

  -----------------------------------------------------------------------

11\. Who are the grantee partners for this project and what roles do
they play? (For each partner, probe for possible roles: planning,
delivering training, fiscal, administrative roles, etc.) How were
partners identified and brought into the grant?

12\. What have been the benefits of working with these partner
organizations? What have been some of the challenges?

  -----------------------------------------------------------------------
  **D. Participating Districts, Schools and Teachers**

  -----------------------------------------------------------------------

13\. Please explain how districts were chosen or came together for
participation in this grant?

14\. Please explain the role of schools in the grant: Did schools choose
to participate as a whole, or did specific teachers request to
participate within schools? Was teamwork encouraged within schools?

15\. Please describe the demographic characteristics of students in the
schools of TAH participants. (Note: Project directors will be asked in
advance to locate, or advise SRI/BPA how to locate, this information.)

> \_\_\_\_\_\_\_approximate percentage eligible for free and
> reduced-priced lunch

\_\_\_\_\_\_\_approximate percentage of special needs students

\_\_\_\_\_\_\_approximate percentage of English language learners

\_\_\_\_\_\_\_approximate percentage of minority students

\_\_\_\_\_\_\_approximate percentage of honors or AP students

16\. Please describe the *participating teachers*:

a.  To date, how many participants have received training? How many
    teachers participated *this year*? Do you have different targets for
    next year?

b.  What grades do teachers from each school teach? Approximately what
    percentage of American history teachers from each grade and school
    participate? How representative are participating teachers from all
    the American History teachers in the schools? (Probe for level of
    experience and expertise of teachers, minority teacher involvement,
    and new teacher involvement.)

c.  How were teachers recruited, targeted and/or selected? What
    challenges do you face in the recruitment and retention of teachers?
    What is your overall attrition rate?

```{=html}
<!-- -->
```
d.  What factors motivated teachers to participate, or impeded their
    participation?

e.  To what extent have school principals/administrators been involved
    in implementing training or recruiting teachers?

  -----------------------------------------------------------------------
  **E. Activities**

  -----------------------------------------------------------------------

17\. Is there an overall theme or historical period emphasized? Why did
you choose to address this historical content? What types of history
skills and teaching methodologies are emphasized? Why?

18\. Please describe the activities your project has offered. Describe
the name of the activity, the number of sessions, the total hours, who
provided the training, who taught or facilitated the activities and a
brief description of the content.

+---------+-------+--------+---------+---------+----------------------+
| **Act   | **N   | **     | **Where | **T     | **Content            |
| ivity** | umber | Number | was     | raining | addressed**          |
|         | of    | of     | t       | methods |                      |
|         | sessi | pa     | raining | used**  |                      |
|         | ons** | rticip | prov    |         |                      |
|         |       | ants** | ided?** |         |                      |
|         | **and |        |         |         |                      |
|         | to    |        | **Who   |         |                      |
|         | tal** |        | taught  |         |                      |
|         |       |        | the     |         |                      |
|         | **ho  |        | sess    |         |                      |
|         | urs** |        | ions?** |         |                      |
+---------+-------+--------+---------+---------+----------------------+
| Summer  |       |        |         |         |                      |
| in      |       |        |         |         |                      |
| stitute |       |        |         |         |                      |
+---------+-------+--------+---------+---------+----------------------+
| Multi-  |       |        |         |         |                      |
| session |       |        |         |         |                      |
| wo      |       |        |         |         |                      |
| rkshops |       |        |         |         |                      |
| with\   |       |        |         |         |                      |
| fo      |       |        |         |         |                      |
| llow-up |       |        |         |         |                      |
+---------+-------+--------+---------+---------+----------------------+
| Single  |       |        |         |         |                      |
| session |       |        |         |         |                      |
| wor     |       |        |         |         |                      |
| kshops, |       |        |         |         |                      |
| se      |       |        |         |         |                      |
| minars, |       |        |         |         |                      |
| field   |       |        |         |         |                      |
| trips   |       |        |         |         |                      |
+---------+-------+--------+---------+---------+----------------------+

+---------+-------+--------+---------+---------+----------------------+
| **Act   | **N   | **     | **Where | **T     | **Content            |
| ivity** | umber | Number | was     | raining | addressed**          |
|         | of    | of     | t       | methods |                      |
|         | sessi | pa     | raining | used**  |                      |
|         | ons** | rticip | prov    |         |                      |
|         |       | ants** | ided?** |         |                      |
|         | **and |        |         |         |                      |
|         | to    |        | **Who   |         |                      |
|         | tal** |        | taught  |         |                      |
|         |       |        | the     |         |                      |
|         | **ho  |        | sess    |         |                      |
|         | urs** |        | ions?** |         |                      |
+---------+-------+--------+---------+---------+----------------------+
| Lesson  |       |        |         |         |                      |
| p       |       |        |         |         |                      |
| lanning |       |        |         |         |                      |
| or      |       |        |         |         |                      |
| project |       |        |         |         |                      |
| related |       |        |         |         |                      |
| to      |       |        |         |         |                      |
| cl      |       |        |         |         |                      |
| assroom |       |        |         |         |                      |
| te      |       |        |         |         |                      |
| aching/ |       |        |         |         |                      |
|         |       |        |         |         |                      |
| cur     |       |        |         |         |                      |
| riculum |       |        |         |         |                      |
|         |       |        |         |         |                      |
| deve    |       |        |         |         |                      |
| lopment |       |        |         |         |                      |
+---------+-------+--------+---------+---------+----------------------+
| Review  |       |        |         |         |                      |
| of      |       |        |         |         |                      |
| lesson  |       |        |         |         |                      |
| plans   |       |        |         |         |                      |
+---------+-------+--------+---------+---------+----------------------+
| Obse    |       |        |         |         |                      |
| rvation |       |        |         |         |                      |
| and     |       |        |         |         |                      |
| f       |       |        |         |         |                      |
| eedback |       |        |         |         |                      |
| of      |       |        |         |         |                      |
| t       |       |        |         |         |                      |
| eaching |       |        |         |         |                      |
+---------+-------+--------+---------+---------+----------------------+
| Partic  |       |        |         |         |                      |
| ipation |       |        |         |         |                      |
| in      |       |        |         |         |                      |
| network |       |        |         |         |                      |
| with    |       |        |         |         |                      |
| his     |       |        |         |         |                      |
| torical |       |        |         |         |                      |
| r       |       |        |         |         |                      |
| esearch |       |        |         |         |                      |
| experts |       |        |         |         |                      |
+---------+-------+--------+---------+---------+----------------------+
| Action  |       |        |         |         |                      |
| r       |       |        |         |         |                      |
| esearch |       |        |         |         |                      |
| team    |       |        |         |         |                      |
| related |       |        |         |         |                      |
| to      |       |        |         |         |                      |
| his     |       |        |         |         |                      |
| torical |       |        |         |         |                      |
| topic   |       |        |         |         |                      |
+---------+-------+--------+---------+---------+----------------------+
| O       |       |        |         |         |                      |
| nline/\ |       |        |         |         |                      |
| te      |       |        |         |         |                      |
| chnolog |       |        |         |         |                      |
| y-based |       |        |         |         |                      |
| r       |       |        |         |         |                      |
| esearch |       |        |         |         |                      |
+---------+-------+--------+---------+---------+----------------------+
| Other   |       |        |         |         |                      |
+---------+-------+--------+---------+---------+----------------------+

19\. On average how many hours do teachers participate per year? Do all
participants participate in the program equally or are some participants
more actively involved than others?

20\. To what extent have teachers implemented what they have learned in
the classroom? Have they received training materials, lesson plans,
websites, or other resources for use in the classroom? Which of the
above activities do you feel have been most effective and useful in
improving instruction of teachers? Why? What have been key positive
outcomes for teachers?

21\. What is your assessment of the overall quality and relevancy of the
training teachers have received? What components of your project worked
best? What were the most challenging? What would you do differently?

22\. How typical is the training observed by SRI/BPA in terms of the
kinds of activities that have occurred or will occur as part of your TAH
project? (If yes, in what ways? If not, why not?)

23\. Have there been positive outcomes for teachers or administrators
who have not *directly* participated in the training activities? (Probe
for teachers' sharing of information with colleagues, school-wide access
to website, lesson plans or other resources.)

  -----------------------------------------------------------------------
  **F. Student Achievement**

  -----------------------------------------------------------------------

24\. How does the project assess the achievement of students taught by
TAH project participants? (Include all methods of assessment, including
experiment, quasi-experiment, single group pre-post, case study, focus
groups, attitude surveys etc.)

> a\. *If you did not conduct a research study (experiment,
> quasi-experiment, or single group pre-post)* what went into your
> decision-making not to do so? What have been the challenges in
> measuring student achievement?
>
> b\. *If you did conduct a research study,* please briefly describe the
> research design, type of analysis, any comparison groups and the
> results. (If copies of written reports are available, please collect
> these documents.)
>
> c\. What were the challenges associated with conducting each of the
> evaluation activities? Will you conduct similar studies in coming
> years? What will you do the same or differently?

25\. Do you have an external evaluator? What is his/her role? What have
been the successes and challenges in working with the evaluator to
assess student achievement?

26\. Do you have suggestions for other ways TAH programs could better
assess student achievement? Do you have suggestions for ways to better
assess the relationship between TAH teacher participation and student
achievement?

  -----------------------------------------------------------------------
  **G. Assessment of Teacher Learning**

  -----------------------------------------------------------------------

27\. How does the project assess what *teachers* learn during the
program?

> a\. Do you use:
>
> \_\_\_\_standardized pre and post test of historical knowledge
>
> \_\_\_\_teacher surveys (what kind, pre and post?)
>
> \_\_\_\_teacher interviews or focus groups
>
> \_\_\_\_changes in student standardized test scores
>
> \_\_\_\_classroom observations
>
> \_\_\_\_reviews of lesson plans
>
> \_\_\_\_other (describe)

28\. Please describe each assessment activity. How was it developed?
What content did it cover? What have been the results/findings? Have you
used the results to make any changes in the project? How?

29\. How well do you feel these assessments (or the evaluation) have
measured what teachers have learned by participating in the program?
What have been the challenges in finding ways to assess what teachers
learn by participating in your project? What have been the challenges of
working with external evaluators in this process?

30\. Do you have any suggestions for other ways TAH programs could
assess teacher learning of historical content, historical thinking
skills or changes in teaching methods?

  -----------------------------------------------------------------------
  **H. Reflection and Wrap Up**

  -----------------------------------------------------------------------

31\. Will you sustain the training after the grant ends? What kinds of
resources would you need to do so?

32\. Overall, what factors would you say made the greatest difference in
the success (or shortcomings) of your TAH grant?

33\. Is there anything else we should know to better understand your
experience with the Teaching American History Program?

34\. Can we get a set of the training materials used for the sessions we
observe? Would it be possible to collect syllabi and course materials
from the training? Are there any copies of teacher or student
assessments you are able to share with us?

*Thank you for your participation!*