Teacher and Leader Evaluation Systems Study:

District Interview Protocol

Paperwork Burden Statement

According to the Paperwork Reduction Act of 1995, no persons are required to respond to a 

collection of information unless it displays a valid OMB number. The valid OMB control number 

of this information collection is 1850-0890. The time required to complete this information 

collection is estimated to average 90 minutes per interview. If you have any comments 

concerning the accuracy of the time estimates(s) or suggestions for improving this form, please 

write to: U.S. Department of Education, Washington, 20202-4651. If you have comments or 

concerns regarding the status of your individual submission of this form, write directly to:

Melanie Ali, U.S. Department of Education, Institute of Education Sciences, Room 502B, 555 

New Jersey Avenue, NW, Washington, DC 20208-5500.
DISTRICT INTERVIEW PROTOCOL SPRING 2013

2

[Date]

[Recipient Title]

[Recipient Street Address]

[Recipient City, ST Zip]

Dear [Recipient],

Thank you for your district’s continued participation in the Teacher and Leader Evaluation 

Systems Study. I am writing regarding the upcoming district interview. I have attached a copy of 

the interview questions for your review. I encourage you to review the questions beforehand in 

case you need to consult a colleague regarding a question prior to the interview. 

In preparation for the interview, I’d like to acquire some documentation regarding your systems 

for evaluating teachers and principals that are in effect in 2012-13. In particular, I am requesting 

that you send me blank copies of the forms you use for rating teacher and principal performance. 

If you have a separate form for classroom observations, please send that as well. In addition, I 

am requesting any written guidance that you provide to those who complete the ratings. 

Guidance may include, but is not limited to, evaluator handbooks, manuals, or memos regarding 

how to rate instruction. Gathering these documents is an important part of our data collection 

process and will help us ensure that we are fully prepared for our interview with you. 

I appreciate your assistance in helping us collect this information. Please send us this information 

at your earliest convenience. Electronic versions of these forms can be mailed to 

awayne@air.org, or sent via standard mail to my address, below. 

Sincerely,

Andrew Wayne, Ph.D.

Project Director, TLES Study

American Institutes for Research

1000 Thomas Jefferson Street, NW

Washington, D.C. 20007
DISTRICT INTERVIEW PROTOCOL SPRING 2013

3

DISTRICT INTERVIEW PROTOCOL

I. Respondent Background Information

The questions in this section will be asked of all respondents to verify that each respondent has 

the information necessary to answer the interview questions. In setting up the interview, the 

interviewer will work with each district to identify respondent(s) with the necessary knowledge to 

respond to our questions and allocate questions to respondent(s) accordingly.

First, we would like to discuss your specific responsibilities related to the topics we are going to 

ask about.

A. Information related to district role in teacher and principal evaluation

1. Please describe your position in the district and your role in the following 

areas, if any:

a. Teacher evaluation

b. Teacher transfer and staffing

c. Teacher professional development

d. Principal evaluation and professional development

e. Implementation of activities associated with TLES

II. Questions Related to Teachers and Teacher Evaluation

The questions in this section will be asked of respondents who are knowledgeable about policies 

and practices related to teachers and teacher evaluation.

Now, I’d like to focus on current policies and practices related to teachers. I will be asking 

specifically about policies that are in effect for the 2012–13 school year.

B. Teacher evaluation policies and practices

1. Ratings of teacher performance

a. According to your district’s human resources policies, are all 

teachers supposed to receive an annual performance rating? Does 

the frequency differ depending on the type of teacher (e.g., novice 

teachers, teachers who received a satisfactory rating in the past)? If 

such ratings are not given annually, how often do teachers receive 

an overall performance rating? 
DISTRICT INTERVIEW PROTOCOL SPRING 2013

4

b. Is the overall rating derived by completing one of the forms that 

you provided? [Determine which form collected ahead of the 

interview is the correct form]

c. What sources of information are used to determine the overall 

performance rating? Are the ratings based in part on:

i. Observations of classroom practice? [If yes, ask 

Question A.2 below]

ii. Analyses of the achievement test scores of the 

teacher’s students? [If yes, ask Question A.3 below]

iii. Administrator’s judgments about other aspects of 

performance (e.g., communication with parents). [If 

yes, obtain a brief description]

iv. Other sources (e.g., portfolios). [If yes, obtain a 

brief description]

d. Are there rules for how information from these sources should be

combined to create the overall performance rating?

2. Details on classroom observations as a source of information for teacher 

performance ratings

a. To help determine performance ratings for teachers, how often are 

teachers required to be observed in their classroom? Does the 

answer differ depending on the type of teacher (e.g., novice 

teachers, teachers who received a satisfactory rating in the past)?

i. Are there rules about how long observations 

typically need to last?

ii. Are there rules about whether observations should 

be announced in advance?

b. Who conducts classroom observations of teachers (e.g., principals, 

assistant principals)?

c. Are the criteria for classroom observations those that appear on the 

overall evaluation form we discussed earlier, or is there a separate 

form used for classroom observations? [If separate, confirm 

understanding of which of the forms collected ahead of the 

interview is the form for classroom observations]

d. Are those who observe given flexibility to use a different form or 

different set of criteria? If so, please explain.
DISTRICT INTERVIEW PROTOCOL SPRING 2013

5

e. Does the district require that observers be trained in how to 

conduct classroom observations and/or how to provide feedback 

based on an observation?

i. Who is responsible for providing the training to 

classroom observers?

ii. Approximately how many hours of training are 

spent on how to rate teacher practice through 

observation?

iii. Approximately how many hours of training are 

spent on how to provide feedback?

iv. What, if any, assessment do observers need to pass 

before they conduct classroom observations for 

purposes of rating teachers?

3. [If applicable] Details on student achievement as a source of information for 

teacher ratings

a. In taking student achievement into account to determine the annual 

performance ratings for teachers, are there requirements about 

which types of student assessment data may be used? [If yes]

Which of the following may be used [Note yes or no for each]:

i. State mandated tests

ii. District mandated tests

iii. Final examination grades

iv. Other (please specify) 

b. [If not already addressed] For teachers in untested grades or 

subjects, is there a different policy regarding how student 

achievement is to be taken into account? If so, please briefly 

describe the policy.

c. In analyzing the student achievement information, is it your policy

to focus on the level of student achievement at the end of the year

or the growth in achievement over the course of a year? If growth 

is considered, how is growth determined?

4. Performance feedback for teachers

a. How often are teachers supposed to receive feedback on their 

performance? For example, are teachers given feedback after each 

observation of their teaching? Are there other occasions when 

teachers are supposed to receive feedback?
DISTRICT INTERVIEW PROTOCOL SPRING 2013

6

b. For these feedback occasions, I’d like to know whether teachers 

are supposed to receive (i) oral feedback, (ii) ratings, and/or (iii) a 

written narrative of some kind.

c. After an observation, is there a time frame within which feedback 

must be given?

5. Use of teacher performance information

a. Can teachers earn additional pay based on their performance? 

b. Are teachers required to set professional growth goals? In what 

way, if any, is teacher performance taken into account in goal 

setting?

c. Are certain teachers required to participate in professional 

development based on their performance? 

d. In what way, if any, is teacher performance taken into account in 

determining whether teachers earn tenure in your district? 

e. Are some teachers on contracts that have to be renewed? If so, in 

what way, if any, is teacher performance taken into account in 

decisions regarding contract renewal.

f. Were any teachers identified for layoffs in the last 12 months? If 

so, in what way, if any, was teacher performance taken into 

account in decisions regarding layoffs.

g. Were any teachers terminated for performance reasons in the last 

12 months?

h. In addition to the uses of individual teacher performance 

information, which were just mentioned, does the district use 

teacher performance information that has been aggregated (e.g., 

school-level summary, district-level summary)? For what purposes 

is such information used (e.g., for planning district-wide 

professional development)?

C. Teacher mobility policies and practices

Next, I would like to ask you about policies related to teacher mobility in your district. We 

would like to understand the process by which teachers change schools. 

1. First, I have some questions about teachers who seek to transfer from one 

school to another within the district—that is, voluntary transfers.

a. Which teachers are eligible to voluntarily transfer? How many

years of experience (in the district) do teachers need before they

can voluntarily transfer?
DISTRICT INTERVIEW PROTOCOL SPRING 2013

7

b. Do certain types of voluntary transfers receive priority over others 

in filling vacancies (e.g., because of seniority)? If so, which 

teachers receive priority?

c. Can voluntary transfers fill positions that are currently held by 

novice teachers or other types of teachers? 

d. Do principals have any discretion regarding voluntary transfers?

2. I also have some questions about teachers who are involuntarily transferred 

between schools. Under what circumstances can a teacher be involuntarily 

transferred?

a. When school enrollments change, in what ways do principals have

discretion to select the teachers who will be involuntarily

transferred? 

b. Are there other circumstances under which a teacher can be 

transferred involuntarily? What role does the principal play in 

identifying who will be transferred under those circumstances?

3. Were any of the policies you just described new in 2012–13? [If so:] Could 

you say a little about the timeline for the implementation of that policy? I’m 

trying to get a sense of whether it was in full effect throughout the 2012–13 

school year.

D. Implementation of TLES components for teacher evaluation

Now, I’d like to turn to the evaluation system components related to teacher evaluation that have 

been implemented in the pilot schools only—the schools that were randomly assigned to receive 

services using the study’s resources. My first questions are about the classroom observation 

system, and then I have some questions about the student growth system.

1. Approach to implementing TLES components

a. After the initial summer training for those who conduct classroom 

observations, we had the principals and DSOs convene for 

additional support meetings approximately once per month. Have 

you done anything outside these meetings to help principals and 

DSOs successfully conduct classroom observations and feedback 

sessions?

b. Have you encountered any challenges with the performance of 

principals in observing teachers and providing feedback? What 

about DSOs? How did you address those challenges?

 

This item and some additional items on this protocol are shaded to indicate that the burden for the items 

was not counted in the burden estimate. The respondent for these items will be the individual in the 

district who is paid by the study to assist with study coordination activities.
DISTRICT INTERVIEW PROTOCOL SPRING 2013

8

c. We provided some parameters for scheduling the observations. 

Have you encountered any challenges in ensuring that the 

observations were conducted in the specified time windows. What 

strategies were used to overcome any challenges? [Ask with 

respect to (i) principals and (ii) DSOs]

d. Have you personally used the [CLASS/FFT] reporting system, or 

directed others to use it on your behalf?

i. Please indicate if you used the reporting system for

management of the implementation of the 

classroom observations. Did you use it for tracking 

purposes, to ensure that observations were talking 

place? Did you use it to monitor the quality with 

which observation reports were being prepared by 

observers?

ii. Please indicate if you used the reporting system to 

access performance information. Did you use it to

access individuals’ performance information? Did 

you use it to access performance information for 

particular groups of teachers (e.g., all teachers in a 

particular school).

iii. Were there any challenges that limited your use of 

the [CLASS/FFT] reporting system? Were there 

challenges in getting access to the reporting system, 

or were there challenges in understanding the 

reports that the system provides? How did you 

overcome these challenges?

iv. Can you identify any suggested improvements to

the [CLASS/FFT] reporting system?

e. Have you personally used the student growth reporting system, or 

directed another person to use it on your behalf?

i. Please indicate how you used the system. Did you 

use it to access individuals’ performance 

information? Did you use it to access performance 

information for particular groups of teachers (e.g., 

all reading teachers)? Did you use it to access 

student growth information for individual schools? 

ii. Were there any challenges that limited your use of 

the student growth reporting system? Were there 

challenges in getting access to the reporting system, 

or were there challenges in understanding the 
DISTRICT INTERVIEW PROTOCOL SPRING 2013

9

reports that the system provides? How did you 

overcome these challenges?

iii. Can you identify any suggested improvements to 

the student growth reporting system?

f. Was the webinar provided shortly after the release of the student 

growth results sufficient, or did you perceive a need for additional 

support?

2. Approach to integration of TLES and current evaluation system practices

a. In your implementation plan, you indicated that the district 

[WOULD/WOULD NOT] use TLES observations in place of the 

observations required under the district’s current policy. Has this 

decision given rise to any challenges for the district, and how were 

those challenges overcome?

b. In your implementation plan, you indicated that TLES 

performance information [SHOULD/COULD/SHOULD NOT] be 

taken into account in giving performance ratings to teachers. Has 

this decision given rise to any challenges for the district, and how 

were those challenges overcome?

c. To clarify, could performance information from TLES be used in

(i) tenure review, (ii) contract renewal, or (iii) termination 

proceedings?

3. Implementation of additional, local uses of TLES system

In reviewing your implementation plan, I see that you have planned several activities to make 

use of the TLES system for teacher evaluation. For study purposes, we are very interested in 

knowing which of these activities were implemented and when. We’d also like to know about the 

challenges you have encountered and the solutions that you have developed to overcome those 

challenges.

[The interviewer will refer to the implementation plan as updated after the spring 2013 

implementation planning visit. Because there is no spring 2014 implementation planning visit, 

the interviewer for the spring 2014 district interview will need to begin by asking about the 

status of each activity (i.e., implemented vs. not implemented) before proceeding with the 

questions.]

a. [For each activity that has reportedly already been implemented] I 

see you have already implemented [ACTIVITY].

i. Could you briefly describe the timeline for 

implementing this activity? I’m trying to get a sense 

of whether it was in full effect throughout the 2012–

13 school year. [Determine when the activity was 
DISTRICT INTERVIEW PROTOCOL SPRING 2013

10

initiated and the extent of implementation achieved 

since initiation. For example, indicate how many 

pilot schools implemented the activity and what 

proportion of teachers in those schools were likely 

affected.]

ii. In implementing this activity, did you encounter any

challenges, and how did you overcome those 

challenges?

b. [For each activity that has reportedly NOT yet been implemented]

I see you have not yet implemented [ACTIVITY].

i. Could you say briefly why this activity has not yet 

been initiated? Have you encountered any specific 

challenges?
DISTRICT INTERVIEW PROTOCOL SPRING 2013

11

III. Questions Related to Principals and Principal Evaluation

These questions will be asked of respondents who are knowledgeable about policies and 

practices related to principals and principal evaluation.

Now, I’d like to focus on current policies and practices related to principals. I will be asking 

specifically about policies that are in effect for the 2012–13 school year.

E. Principal evaluation policies and practices

1. Ratings of principal performance

a. According to your district’s human resources policies, are all 

principals supposed to receive an annual performance rating? Does 

the frequency differ depending on the type of principal (e.g.,

novices, those who received a satisfactory rating in the past)? If 

such ratings are not given annually, how often do principals 

receive an overall performance rating? 

b. Is the overall rating derived by completing one of the forms that 

you provided? [Determine which form collected ahead of the 

interview is the correct form]

c. What sources of information are used to determine an overall 

performance rating? For each source, please indicate how often it 

is required to be used. 

i. [If not already discussed]Are principals required to 

complete a self-assessment as part of the principal 

evaluation process?

ii. [If not already discussed]In what way, if any, is input from 

teachers supposed to be taken into account to determine 

performance ratings for principals?

2. [If applicable] Details on student achievement as a source of information

for principal performance ratings

a. In taking student achievement into account to determine 

performance ratings for principals, are there requirements about 

which types of student assessment data may be used? [If yes]

Which of the following may be used [Note yes or no for each]:

i. State mandated tests

ii. District mandated tests

iii. Final examination grades
DISTRICT INTERVIEW PROTOCOL SPRING 2013

12

iv. Other (please specify) 

b. In analyzing the student achievement information, is it your policy

to focus on the level of student achievement at the end of the year 

or the growth in achievement over the course of a year? If growth 

is considered, how is growth determined?

3. Performance feedback for principals

a. How often are principals supposed to receive feedback on their 

performance from their supervisor? For each occasion on which 

feedback is expected, I’d like to know whether the feedback is 

supposed to be based on the form identified earlier, or any other 

form. I’d also like to know whether the feedback is supposed to 

include (i) oral feedback, (ii) ratings, and/or (iii) a written narrative 

of some kind. 

b. Were any principals terminated for performance reasons in the last 

12 months?

c. In addition to uses of individual principal performance 

information, does the district use principal performance 

information that has been aggregated (e.g., district-level 

summary)? For what purposes is such information used (e.g., for 

planning district-wide coaching of principals)?

F. Implementation of TLES components for principal evaluation

Now, I’d like to turn to the evaluation system components related to principal evaluation that 

have been implemented in the pilot schools only, using the study’s resources.

1. Approach to implementing TLES components

a. First I have some questions about your experiences with 

administering the VAL-ED survey

i. In administering the survey, did you encounter any

concerns about the survey from the intended 

respondents? Please describe any concerns you 

encountered, and the type of respondent who 

expressed the concern (e.g., elementary school 

teachers). If you tried to address the concern, how 

did you do so, and were you successful?

ii. Did you have any challenges in getting the intended 

respondents to respond and to submit their 

responses within the intended timeline? How did 

you address those challenges?
DISTRICT INTERVIEW PROTOCOL SPRING 2013

13

iii. Did you think the respondent training was 

adequate? Did you do anything to supplement the 

respondent training?

b. My next questions are about feedback given to principals after 

each VAL-ED administration.

i. Did you encounter any challenges in the feedback 

sessions with principals? If so, what were the 

challenges, and how did you overcome those 

challenges?

ii. Did you think the supervisor training was adequate? 

Did you do anything to supplement the supervisor 

training?

c. I also have some questions about the VAL-ED reporting system.

Did you personally use the reporting system, or direct others to use 

it on your behalf? 

i. Please indicate if the reporting system was used to 

access performance information. Was it used to 

access individuals’ performance information? Was 

it used to access performance information for all 

principals collectively?

ii. Were there any challenges that limited your use of 

the VAL-ED reporting system? Were there 

challenges in getting access to the reporting system, 

or were there challenges in understanding the 

reports that the system provides? How did you 

overcome these challenges?

iii. Can you identify any suggested improvements to 

the VAL-ED reporting system?

2. Approach to integration of TLES and current evaluation system practices

a. In your implementation plan, you indicated that TLES 

performance information [SHOULD/COULD/SHOULD NOT] be 

taken into account in giving performance ratings to principals. Has 

this decision given rise to any challenges for the district? If so, 

how were those challenges overcome?

b. To clarify, could performance information from TLES be used in 

ways that had consequences for principals (e.g., promotion, 

termination, etc.)? Please explain.

3. Implementation of additional, local uses of TLES system
DISTRICT INTERVIEW PROTOCOL SPRING 2013

14

In reviewing your implementation plan, I see that you have planned several activities to make 

use of the TLES system for principal evaluation. For purposes of the study, we are very 

interested in knowing which of these activities were implemented and when. We’d also like to 

know about the challenges you have encountered, and solutions that you have developed to 

overcome those challenges.

[The interviewer will refer to the implementation plan as updated after the spring 2013 

implementation planning visit. Because there is no spring 2014 implementation planning visit, 

the interviewer for the spring 2014 district interview will need to begin by asking about the 

status of each activity (i.e., implemented vs. not implemented) before proceeding with the 

questions.]

a. [For each activity that has reportedly already been implemented.]

I see you have already implemented [ACTIVITY].

i. Could you briefly describe the timeline for 

implementing this activity? I’m trying to get a sense 

of whether it was in full effect throughout the 2012–

13 school year. [Determine when the activity was 

initiated and the extent of implementation achieved 

since initiation. For example, indicate how many 

pilot schools implemented the activity and what 

proportion of teachers in those schools are likely 

affected]

ii. In implementing this activity, have you encountered 

any challenges, and how have you overcome those 

challenges?

b. [For each activity that has reportedly NOT yet been implemented]

I see you have not yet implemented [ACTIVITY].

i. Could you say briefly why this activity has not yet 

been initiated? Have you encountered any specific 

challenges?

IV. Use of Time

These questions will be asked of respondents who are knowledgeable about time spent by district 

staff in support of the student growth component of the study. 

G. Use of time to support student growth component of study

One purpose of our interviews is to determine the total amount of time spent by district staff in 

working with the American Institutes for Research (AIR) to implement the student growth 

component of TLES. Please answer the following questions with respect to the most recently 

completed round of student growth reports.
DISTRICT INTERVIEW PROTOCOL SPRING 2013

15

1. After the district agreed on the covariates to be included in the value-added 

model, how many central office staff members were involved in working with 

AIR to implement the student growth component of TLES? If applicable, 

please divide the staff into those considered “regular” and “senior.” How 

many regular staff were involved, and how many senior staff were involved?

2. I’m interested in determining the amount of time that regular and senior staff 

spent after the district agreed on the covariates to be included in the valueadded model. For each of the following, please provide an estimate of the total 

amount of staff time spent by regular staff and the total time spent by senior

staff.

a. Discussing data requests with AIR

b. Preparing and submitting data to AIR

c. Troubleshooting and resubmitting information as needed