1

APPENDIX F

DISTRICT INTERVIEW PROTOCOL
2

TIF DISTRICT INTERVIEW PROTOCOL

Introduction

This is _________ from xxxxxxxxxxxx. As you know, The U.S. Department of Education has contracted 

with xxxxxxxxxx to conduct the national evaluation of the Teacher Incentive Fund. The information we 

will obtain from this interview is extremely important for us to fully understand your TIF program, your 

implementation experiences and challenges, how you met those challenges, and to learn about other 

district improvement efforts. This interview should take about 45 to 60 minutes. Before we proceed, I 

would like to confirm that this is still a good time for you.

(If a good time, proceed with text below; if not, ask: when would be a good time to call? On what date? 

Then repeat time and date to respondent.)

We would like you to know that:

 As a requirement of receiving the TIF grant, your district has agreed to be included in the TIF 

evaluation. You are being asked to participate in the district interview because it is a 

component of the evaluation. While cooperation with this activity is a condition of your grant, 

we appreciate your time. 

 Your responses are critical for producing valid and reliable data. You may not respond to any

questions you do not wish to answer; however, we hope that you answer as many questions as 

you can.

 Your responses will not affect your grant—now or in future.

 We will keep your responses confidential. The data you provide will be used only for statistical 

purposes. The reports prepared for this study will summarize findings across the sample and 

will not associate responses with a specific district or individual. We will not provide 

information that identifies you or your district to anyone outside the study team, except as 

required by law. Additionally, no one in your district will see your responses.

 Participation in the district interview will not impose any risks to you as a respondent other 

than accidental disclosure of information. Safeguards are in place to ensure individual 

respondent’s confidentiality including restricted access to interview data and separating 

identifying information such as name and school from interview responses. All <contractor>

staff sign a confidentiality pledge and all staff with access to study data have received clearance 

from the U.S. Department of Education and are subject to severe legal consequences for any 

breach of confidentiality.

 If you have any questions about this study, please contact xxxxxxxxxxxxxx.

 In preparation for the interview, we have refered to your district survey or other program 

documents. We may ask for confirmation of this information during the interview.
3

Program name: ______________________________________________________________

District Name: ________________________________________________________________

Date: _______________________________________________________________________

Person interviewed:

Name: ______________________________________________________________________

Title: _______________________________________________________________________

Phone number: _______________________________________________________________

Email:_______________________________________________________________________

Interviewer name:_____________________________________________________________
4

A. Background and Context

First, I’d like to talk about any existing or additional performance pay programs in your district.

1. Before the 2010 TIF grant, did your district have performance pay program(s) for teachers or 

principals? 

If yes, ask the following questions.

a.

b. Is the program still in operation?*

If no: When was it discontinued?

Why was it discontinued?

2. During the current school year, are there federal, state, district, or privately sponsored 

programs, OTHER than TIF, that offer additional compensation to teachers and/or principals in 

your district? 

If yes, ask the following questions.

a. What are the names of the programs?

b. What are the sources of funding?*

c. For what reasons or factors can educators receive additional compensation? (e.g., high 

effectiveness, extra roles and responsibilities, teacher in high-need schools, etc.)

d. Who is eligible for additional compensation? (e.g., teachers, principals, non-teaching staff)

e. How much extra compensation can teachers and/or principals receive?

f. Are schools required to participate, or can they opt out?*

g. How many schools participate?*

1. Are any of these schools participating in the 2010 TIF (or TIF 3) grant?*

If yes: What are the names of the schools?

3. Have any schools stopped participating in TIF during the period of the grant?

If yes, ask the following questions.

a. How many schools discontinued their participation in TIF?

b. What were the primary reasons for discontinuing participation in TIF (lack of principal 

support? Lack teacher support? Other?)

B. Characteristics of TIF Program 

I’d like to learn about the eligibility criteria for schools, teachers, and principals in the TIF program.

3. Do teachers or principals have to pass any eligibility requirements to be part of the 

performance-based compensation program? (e.g., teach tested subject, full-time teacher)
5

If yes: What are these requirements? (Ask about anything besides what is listed above)

What grade levels/subjects are eligible?

Are non-tested grades/subjects eligible?

4. Do teachers or principals have to apply to be considered for a performance award?]

If yes: What are the application procedures and requirements?

5. Are teachers or principals required to do anything additional to receive a performance award 

(e.g., open their classrooms to observers)?

Next, let’s talk about classroom observations.

6. Are classroom observations used to evaluate teacher performance?

If yes: Describe the observation tool or rubric that is used. Is it commercially published 

or district created?

Describe the observations. How many formal observations are conducted 

annually? Informal? How long are the observations? Who conducts the 

observations? 

Who trains the classroom observers?

Are observers required to meet reliability standards? If so, what is the process 

for meeting reliability (i.e. training to the gold standard, frequency of reliability 

assessment, quality assurance procedures)?

Are there any differences between the treatment and control schools for 

teachers in terms of the number of observations or feedback provided?

Are teachers able to review the rubric prior to observations?

Are teachers shown their rating after an observation?

Next I’d like to ask about evaluation criteria.

7. How do you measure effectiveness for teachers in tested grades/subjects? 

a. What are the components of the evaluation system? (e.g. student achievement, 

observations, portfolio/teaching artifacts)

b. How many performance levels comprise the evaluation system? What is the definition of 

each level? (e.g. satisfactory, unsatisfactory)

c. What growth model is used? What assessment scores are used for elementary? Secondary?

8. Do you have different ways of evaluating teachers who teach in untested grades or subjects?

If yes: How do you evaluate teachers in untested grades or subjects?

9. Are there any differences between the treatment and control schools for teachers in terms 

of how effectiveness was measured?
6

10. How are principals evaluated in your program? 

a. What are the components of the evaluation system? (e.g. student achievement, 

observations, portfolio/teaching artifacts)

i. Describe the principal observation process. Is there are tool or rubric that is 

used? How many formal observations are conducted annually? Informal? How 

long are the observations? Who conducts the observations?

ii. Have there been any challenges about principal observations? If so, what? 

iii. Are there any differences between the treatment and control schools for 

principals in terms of the number of observations or feedback provided?

b. Are the performance levels for principals the same as those for teachers?

If No: What are the performance levels for principals?

c. Is the growth model used for principals the same as that used for teachers?

If No: What growth model is used for principals?

d. Are there any differences between the treatment and control schools for principals in 

terms of how effectiveness was measured?

Now let’s discuss performance pay that is based on the evaluation criteria.

11. Will performance-based bonuses or awards for teachers be based on one overall performance 

rating, separate ratings for each criterion, or something else?

If yes for overall rating: (focusing only on teachers in tested grades and subjects)

What criteria are included in this overall rating?

How do you weight the different criteria? 

What is the maximum expected performance award you expect a teacher could 

receive? 

What is the average, or “typical,” expected performance award you expect a 

teacher could receive?

If yes for separate awards for different criteria:

What are they? 

What is the maximum expected performance award you expect a teacher could 

receive? 

What is the average, or “typical,” expected performance award you expect a 

teacher could receive?

If yes for something else:

How are awards determined? (Could be the threshold model or something else)
7

What is the maximum expected performance award you expect a teacher could 

receive? 

What is the average, or “typical,” expected performance award you expect a 

teacher could receive?

12. Will performance-based bonuses or awards for principals be based on one overall performance 

rating, separate ratings for each criterion, or something else?

If yes for overall rating: 

What criteria are included in this overall rating?

How do you weight the different criteria? 

What is the maximum expected performance award you expect a principal could 

receive? 

What is the average, or “typical,” expected performance award you expect a 

principal could receive?

If yes for separate awards for different criteria:

What are they? 

What is the maximum expected performance award you expect a principal could 

receive? 

What is the average, or “typical,” expected performance award you expect a 

principal could receive?

If yes for something else: 

How are awards determined? (Could be the threshold model or something else)

What is the maximum expected performance award you expect a principal could 

receive? 

What is the average, or “typical,” expected performance award you expect a 

principal could receive?

13. What is the maximum salary a teacher in the district can make? Principal?

14. Can a teacher or principal appeal or dispute his or her performance pay or award amount?

If yes: Is there a formal appeals process for teachers or principals have such a dispute?

If yes: Please briefly describe this process.

15. How are bonuses for teachers handled if the teacher...

Circle one Notes (cut off dates, differences in 

award amounts, etc.)

Gets hired after the beginning of the year at a Bonus
8

TIF group 1 school? No Bonus

Leaves a TIF group 1 school before the end of 

the year?

Bonus

No Bonus

Transfer schools within the district from a TIF 

to a non-TIF school, or vice versa?

Bonus

No Bonus

Transfer schools within the district from a TIF 

group 1 school to a group 2 school?

Bonus

No Bonus

Is on extended leave? Bonus

No Bonus

Now let’s discuss roles and responsibilities in the TIF program.

16. What opportunities do you provide teachers and principals to earn additional pay through the 

TIF program? (e.g. leadership roles, teaching a high-need subject)

a. How much additional pay is available?

b. What criteria must be met to receive the additional pay?

c. What is the selection process for participation in these opportunities? (Probe on each 

as appropriate. Ask specifically if selection is based all or in part on whether the 

applicant has been rated highly effective by the current performance-based evaluation 

system.)

17. Are there any differences between the treatment and control schools for teachers or 

principals in terms of opportunities for additional pay, such as criteria or the number of 

opportunities?

I’d like to ask a little about professional development as part of the TIF program.

18. Do you provide professional development opportunities for teachers and/or principals to 

explain the measures that are used to determine effectiveness?

If yes: Who provides it, and briefly describe how it is done.

19. Are teachers compensated for professional development opportunities?

If yes: How much? 

20. Are there any differences between the treatment and control schools for teachers or 

principals in terms of professional development opportunities?

c. 22. Do you use the prior year’s (2013-14) performance evaluation results to select this 

year’s (2014-15) professional development (PD) offerings?

If yes: Briefly describe the process. Do PD offerings vary by school or for individual staff 

based on their performance results?

C. Recruitment, Hiring, and Transfer Policies

I’d like to learn about your district’s recruiting, hiring, and transfer policies.
9

21. What policies and practices does the district have for recruiting and hiring teachers to work in 

the district?

22. What policies and practices does the district have for transferring teachers within the district?

23. What is the last day in the school year that a transfer to another school is allowed?

24. How much input do principals have in recruitment, hiring, or transfer decisions?

25. Are there any differences between the treatment and control schools for teachers or 

principals in terms of:

a. Policies and practices the district has for recruiting and hiring teachers? 

b. Policies and practices the district has for transferring teachers within the district?

c. Turnover of teachers or principals in treatment or control schools?

If yes: Was turnover more prevalent in T or C schools?

Was turnover last year different that previous years?

Which grade levels? Subject areas?

Did you conduct exit interviews to learn why these positions were 

vacated?

If yes: Why were these positions vacated?

If no: In your opinion, why were the positions vacated? 

Were any of these vacancies because of dismissal?

D. Communication and Participant Support

I’d like to turn to talk about how your district has refined the design and implemented of your TIF 

program.

a.

26. What modifications have been made to the original TIF program design this year (2014-15)?

d. Why were these modifications made?

Next I’d like to talk about communication about the TIF program.

27. When were school staff first made aware that they could receive bonuses for 2013-2014? 

[Month, Year] How was this communicated? 

28. What techniques do you use to communicate information about the program and its benefits?

Have you found a particular technique to be more effective than others? Why? 

29. Is there a process for program participants to provide feedback about the program? 

30. Are there any differences between the treatment and control schools for teachers or 

principals in terms of communication activities about the TIF program?
10

E. Technical Assistance

I’d like to discuss support you have received for implementing the TIF program.

31. What have been the two most useful technical assistance activities the district received as part 

of its participation in the TIF program?

For each activity: Who provided this TA? [If not Mathematica, ask about relationship with 

TA provider]

How often was this TA provided?

F. Challenges and Experiences

Let’s discuss the experiences and challenges your district has experienced.

32. What key successes did the district experience during the 2013-14 period of the TIF program?

33. What has been challenging so far about implementing the program? [Probe specifically on 

resistance to participation, communication, calculating awards amounts, determining 

effectiveness measures (explicit VA model or observation tool to use)]

34. Based on your experiences, would you have done anything differently? 

If yes: Please describe.

G. Plans for 2015-16

Let’s discuss your plans for TIF policy or policy building off of your TIF experience in your 

district.

35. Do you plan to continue additional pay for teachers based solely on their performance?

If no, why not? 

if yes – do you plan to retain the policy in a subset of targeted schools or expand the policy 

to more or all of your schools? Please discuss how TIF informed this choice.

36. Related to teacher performance evaluation measures:

a. Do you plan on using student achievement growth to measure teacher 

performance? 

if yes – are there particular benefits or reasons for wanting this policy? Please 

discuss how TIF informed your choices related to the growth measure you selected 

(same or different from that used with TIF)

If no – are there particular concerns with or challenges to using this policy?)? 

Please discuss how TIF informed this choice.

b. Do you plan on using observations to measure teacher performance? 

if yes – are you using the same rubric you used for TIF? Please discuss how TIF 

informed your choice related to the rubric you selected (same or different from that 
11

used with TIF). Also, please discuss how TIF informed your choice related to number 

and nature of observations you will be conducting.

If no – are there particular concerns with or challenges to using observations to 

measure teacher performance?

c. Do you plan on using measures other than student achievement growth or 

observations to measure teacher performance? Please discuss how TIF informed 

your choice.

37. Related to principal performance evaluation measures: 

a. Do you plan on using student achievement growth to measure principal 

performance? Please discuss how TIF informed your choice.

b. Do you plan on using observations to measure principal performance? Please 

discuss how TIF informed your choice.

Do you plan on using measures other than student achievement growth or observations to 

measure principal performance? Please discuss how TIF informed your choice.