SUPPORTING	STATEMENT A:

REQUEST	FOR	CLEARANCE	OF	INFORMATION	

COLLECTION	FORMS	FOR

“IMPACTS	OF	A	DETAILED	CHECKLIST	ON	

FORMATIVE	FEEDBACK	TO	TEACHERS”

August 2014

Submitted to: Submitted by:

U.S. Department of Education SEDL

Institute of Education Sciences 4700 Mueller Blvd.

555 New Jersey Ave. NW, Rm. 308 Austin, TX 78723

Washington, DC 20208 Phone: (800) 476-6861

4700 Mueller Blvd. Austin, TX 78723

800-476-6861

www.relsouthwest.org

This publication was prepared for the Institute of Education Sciences (IES) under contract ED-IES-12-C-00012 by 

Regional Educational Laboratory Southwest, administered by SEDL. The content of the publication does not 

necessarily reflect the views or policies of IES or the U.S. Department of Education, nor does mention of trade names, 

commercial products, or organizations imply endorsement by the U.S. government. The publication is in the public 

domain. Authorization to reproduce in whole or in part for educational purposes is granted.
Contents

OVERVIEW ................................................................................................................................... 5

Impact Research Questions ......................................................................................................... 7

Implementation Fidelity Research Questions.............................................................................. 7

Exploratory Research Questions ................................................................................................. 7

A. JUSTIFICATION...................................................................................................................... 9

1. Circumstances Necessitating the Data Collection.................................................................. 9

2. How, by Whom, and for What Purpose the Information Is to Be Used............................... 17

3. Use of Information Technology ........................................................................................... 18

4. Efforts to Avoid Duplication of Effort ................................................................................. 19

5. Sensitivity to Burden on Small Entities................................................................................ 19

6. Consequences of Less Frequent Collection and Obstacles to Burden Reduction ................ 19

7. Circumstances Requiring Special Information Collection ................................................... 20

8. Federal Register Announcement and Consultation Outside the Agency ............................. 20

a. Federal Register Announcement ........................................................................................ 20

b. Consultations Outside the Agency ..................................................................................... 20

c. Unresolved Issues............................................................................................................... 21

9. Payments or Gifts to Respondents........................................................................................ 22

10. Assurances of Confidentiality ............................................................................................ 22

11. Justification for Sensitive Questions .................................................................................. 24

12. Estimates of Hour Burden .................................................................................................. 24

Burden for Study Recruitment ............................................................................................... 26

Burden for Extant Data Collection......................................................................................... 27

Burden for Survey Data Collection........................................................................................ 27

13. Estimated Total Annual Cost Burden to Respondents or Recordkeepers.......................... 27

Total Cost to Respondents...................................................................................................... 27

14. Estimated Annualized Cost to the Federal Government .................................................... 28

15. Program Changes or Adjustments...................................................................................... 28

16. Plans for Tabulation, Publication, and Schedule for Project.............................................. 28
a. Tabulation Plans................................................................................................................. 28

b. Publication Plans................................................................................................................ 29

c. Project Time Schedule........................................................................................................ 30

17. Display of Expiration Date ................................................................................................. 31

18. Exceptions to the Certification Requirement...................................................................... 31

REFERENCES ............................................................................................................................. 32

APPENDIX A – Teacher and Principal Survey Instruments........................................................ 34

Spring 2015 School Leader Survey............................................Error! Bookmark not defined.

Spring 2016 School Leader Survey............................................Error! Bookmark not defined.

Spring 2015 Teacher Survey ......................................................Error! Bookmark not defined.

Spring 2016 Teacher Survey ......................................................Error! Bookmark not defined.

APPENDIX B – Principal and Teacher Recruitment Materials................................................... 95

Wave 1 Principal Recruitment Script– SPRING 2015.............................................................. 95

Wave 1 Teacher Recruitment Script– – SPRING 2015 .............Error! Bookmark not defined.

Wave 2 Principal Recruitment Script– – SPRING 2016............Error! Bookmark not defined.

Wave 2 Teacher Recruitment Script– – SPRING 2015 .............Error! Bookmark not defined.

Wave 1 Principal Recruitment Script Follow-up .......................Error! Bookmark not defined.

Wave 1 Teacher Recruitment Script Follow-up .........................Error! Bookmark not defined.

APPENDIX C – Secondary Data Elements................................................................................ 111

Teacher and School Administrative Data Variables List ........................................................ 111

NM TEACH Reflect Data Variables List...................................Error! Bookmark not defined.

Principal and Teacher Contact Information................................Error! Bookmark not defined.

APPENDIX D – Technical Working Group Suggestions.......................................................... 117

APPENDIX E – Educational Sciences Reform Act ................................................................... 119

APPENDIX F – IRB Approval................................................................................................... 120

APPENDIX G –Study Participants’ Consent ............................................................................. 124

Principal Wave 1 and Wave 2 Consent ......................................Error! Bookmark not defined.

Teacher Wave 1 and Wave 2 Consent..................................................................................... 128

APPENDIX H - Confidentiality Form and Affidavits................................................................ 132

5

OVERVIEW

The U.S. Department of Education (ED) requests clearance for the recruitment materials and 

data collection protocols under the OMB clearance agreement (OMB Number [IES to complete])

for activities related to the Regional Educational Laboratory Program (REL). ED, in consultation 

with SEDL, is planning a clustered randomized evaluation in New Mexico to test the 

effectiveness of materials intended to improve the feedback that principals provide in one-on-one 

conferences to their teachers about their classroom instruction. The study includes an impact 

analysis and an implementation analysis. 

New Mexico Public Education Department (NM PED) staff have identified the topic of principal 

feedback to teachers as an area where New Mexico needs assistance. In particular, NM PED 

staff have received comments from principals that they do not feel adequately prepared to 

provide teachers with feedback about their instructional practices. The NM PED has committed 

to ensure that every student has equitable access to an effective principal and teacher every day

they are in school. Although NM PED recognizes that the conferences are likely an area in the 

overall evaluation system in need of improvement, it has limited resources and time to focus on 

the post-observation conference step in the evaluation cycle. Therefore, any potential 

interventions to address this issue must be easy to implement and require few resources.

This impact study will examine whether an enhanced feedback guide (Conversation Protocol), 

relative to business-as-usual guidance to principals and teachers, achieves the following:

1. improves the structure and content of the principal-teacher feedback conversation, 

2. improves perceptions that the feedback delivered to teachers is useful 

3. better targets guidance to teachers’ regarding professional development that is tied to their 

formal classroom observation scores

4. improves quality of teacher instruction as measured by subsequent formal observation ratings 

on the state’s tool, called NM TEACH, and 

5. increases student achievement and state standardized tests.

To examine these outcomes, ED contractors will invite all public school principals of charter and 

non-charter schools in the state to participate in the study. Blocking by school district, a 

randomly selected half of principals who agree to participate (along with teachers in their school 

who also agree to participate in the study) will be assigned to the treatment group. The remaining 

half of principals (along with teachers in their school who also agree to participate in the study) 

will be assigned to the control group. 

In summary, the study will compare principals and teachers in the treatment and control 

conditions, which are detailed in Table 1. 
6

Table 1. Treatment and Control Conditions

Treatment 

group

Control 

group

Principals

NM PED-sponsored professional development for principals 

offered during the summer, of which 1.5-2 hours were 

devoted to feedback to teachers. Training conducted in 

regional in-person professional development sessions led by

the state’s contractor, Southern Regional Education Board 

(SREB). All principals in the state are required to participate 

as a part of their job requirements.

 

List of documents to bring to each feedback conversation 

24-item checklist to use during each feedback conversation. 

5-minute video in which principal testifies about her

experience using the checklist.



Business-as-usual guide to remind principals about “5 stages 

of feedback” described in NM PED-sponsored principal 

professional development to use for feedback conversations



Teachers

Business-as-usual guide: A document the includes a 

reminder to teachers about their right to feedback within 10 

calendar days of formal observation under the NM TEACH 

Educator Effectiveness System, and a link to the NM PED 

provided FAQ on teacher evaluation 

(http://ped.state.nm.us/ped/NMTeach_FAQ.html)



List of documents to bring to each feedback conversation 

24-item checklist to use during each feedback conversation 

5-minute video in which teacher testifies about her 

experience using the checklist



ED contractors will request that the treatment group principals and control group principals do 

three things: (1) disseminate the relevant guide (and a link to the video, where relevant) to all 

school leaders and to all teachers in his/her building, (2) use the guide and view the video, where 

relevant (along with all other schools leaders in the building who observe teachers) with 100% of 

the teachers that they formally observe during school year 2015-2016, and (3) complete a 

baseline survey in fall 2015 and a follow-up survey in spring 2016. ED contractors will request 

treatment group teachers and control group teachers to do two things: (1) use the guide they

received from ED contractors (and view the video, where relevant) in the feedback conversations 
7

they have with their school leader(s), and (2) complete a baseline survey in fall 2015 and a 

follow-up survey in spring 2016. 

Impact	Research	Questions

The evaluation will examine the impact of providing a post-classroom observation feedback 

checklist and a video to principals and teachers on principal, teacher, and student outcomes. To 

do so, this evaluation will use a cluster RCT design with random assignment at the school level 

to address the following impact and implementation questions. 

Does provision of the feedback checklist and video relative to the business as usual guidance: 

1) improve the content and structure of post-observation conference according to principals and 

teachers? 

2) improve principals’ and teachers’ perceptions of the utility of post-observation conference? 

3) increase the amount of time it takes to complete the post-observation conference?

4) cause principals to recommend and teachers to take professional development that is aligned 

to needs identified in the formal observations?

5) improve the quality of teachers’ subsequent instructional practices as measured by principals’ 

formal classroom observation scores?

Implementation	Fidelity	Research	Questions

ED contractors will also examine the implementation of the Feedback Checklist relative to the 

business-as-usual guidance by analyzing responses to survey questions regarding the type of 

guidance used. The implementation portion of the study will address the following research 

questions:

6) How extensively do principals and teachers in the treatment and control groups report using 

the form of guidance they were assigned? 

7) How extensively do principals and teachers in the treatment and control groups report using 

the form of guidance they were not assigned (i.e. cross-overs)?

Exploratory	Research	Questions

ED contractors will also perform exploratory analyses examining the association of the enhanced 

feedback protocol on subgroups of teachers and principals. In addition, we will also examine 

whether provision of the treatment guide improves subsequent student performance on state 

standardized tests. Although the design of the study allows for causal analysis of these questions, 

for reasons of limited sample size within New Mexico and limited time for the treatment or 

control condition to influence behavior and student achievement, we will likely have insufficient 

statistical power to detect differences. Nevertheless, these questions have high policy relevance 

for NM PED, so we include them to provide insights for follow-on studies and for NM PED’s 

improvement of subsequent iterations of NM TEACH observations and feedback. The

exploratory research questions are:
8

8) Does the provision of the checklist to principals and teachers increase student achievement 

relative to business-as-usual guidance regarding feedback? 

9) Is there suggestive evidence that the effects of providing the checklist varies according to the 

following subgroups of principals and to the following subgroups of teachers:

a) extensive users of the checklist – i.e., teachers who have received three versus two 

formal observations and principals who have used the checklist in schools that have a 

greater than average number of teachers within the given district

b) personal beliefs and attitudes about NMTEACH evaluation system

c) accountability pressure (as measured by prior year’s school accountability grade)

d) those principals who have received training on the NMTEACH Observation Rubric

e) those who work in schools that teachers rate as collaborative and supportive

f) those with greater qualifications, including more years of experience and certification 

type

g) those who teach core subjects versus non-core subjects

h) those who work in elementary versus middle or high schools

i) teachers in the top and bottom quartile of the NM TEACH summative effectiveness 

score distribution

j) principals in the top and bottom quartile of the NM TEACH School Leader 

effectiveness score distribution

Beyond subgroup analyses, ED contractors will also gather information about the perceptions 

users have of the treatment checklist and the control guide.

10) What are principals’ and teachers’ perceptions of the checklist and videos versus business-asusual feedback guidance? For example, do they report that it is easy to use, burdensome, 

formulaic, lengthy, useful for creation of professional development plans, and helpful in 

causing teachers to commit to instructional changes? 

11) What are the main reasons why principals report not using the form of guidance they

received? checklist or the business-as-usual guidance? 

The project consists of data collection from NM PED and participating principals and teachers 

who work in New Mexico public schools. Specifically, in this OMB clearance package, ED is 

requesting clearance for the following data collection approaches:

 Recruitment materials for participating principals and teachers

 Extant administrative records data collections from NM PED

 Two waves of web-based surveys of principals and teachers in treatment and control schools.

ED believes that the data collections for which clearance is being requested represent the bare 

minimum necessary to assess the impact and implementation of the Conversation Protocol.
9

A.		JUSTIFICATION

1.		Circumstances	Necessitating	the	Data	Collection

New Mexico is committed to ensure that every student has equitable access to an effective 

principal and teacher. As a part of this mission, the NM PED has committed to “implementing a 

rigorous, uniform [classroom] observation protocol, providing immediate constructive feedback 

[to teachers], using meaningful student data, and other multiple measures [that] will provide 

valuable information to aid the personal development and growth of each teacher and principal.” 

NM PED undertook the task of reforming their teacher evaluation system starting in 2011. In 

2012-2013, New Mexico piloted a new teacher evaluation system called NM TEACH that is 

mandated in all public schools starting in the 2013-2014 school year. Under NM TEACH, 

teachers are evaluated each school year based on three primary components: 

1. evidence of student achievement improvement (such as value added scores on state and end 

of course exams),

2. average scores on the state-developed teacher observation rubric (NM TEACH), and

3. locally selected measures from a menu of options approved by the state (e.g., teacher 

attendance or student surveys).

These components are combined into a summative evaluation score, with 50% of the summative 

evaluation score based on improved student achievement, 25% based on the observation of 

teacher practice, and the final 25% based on the locally selected measures. Regarding the second 

component of teacher evaluations, NM PED requires that a principal or a designated school 

leader observe his or her teachers formally two or three times per year (with 20 minute 

observations), and informally throughout the school year (with several short “walk-throughs”

lasting 3-5 minutes each). For formal observations, the school leader must rate the teacher using 

the NM TEACH Observation Rubric—a single rubric that applies to all grade levels and subjects 

taught. The combined scores from the 2-3 required formal observations comprise the second 

element of teachers’ ratings. 

As the teacher evaluation cycle is conceived, a critical step to improving instruction through the 

revised teacher evaluation system is the required conference between the teacher and the school 

leader that takes places within 10 days after each of the formal classroom observations. In this 

conference the school leader reviews his or her classroom observation rubric scoring with the 

teacher and makes recommendations for improvement and professional development. Note that 

this study will only recruit one type of school leader—principals—and we therefore only refer to 

principals when describing the conduct of formal observations and feedback below.1

 

1 The recruitment does not include all types of school leaders such as assistant principals or deans because it is a 

school-based decision which school leaders, if any, besides principals formally observe and rate teachers using the 

NM TEACH Observation Rubric. Therefore, among school leaders, ED contractors will recruit only principals into 

the study, and then ask participating principals to disseminate the guide they receive (whether the treatment or 

control guide) to all relevant school leaders in the building who also conduct formal classroom observations.
10

In the research literature, much attention has been paid to the development of the classroom 

formal observation rubrics and the extent to which teacher observations align with other 

measures of student learning.2 However, NM PED, like the broader field, has paid much less 

attention to the next step in the annual cycle of the teacher evaluation: the feedback conversation 

in which the principal conveys to the teacher the results of the scored observation. There is a 

dearth of scientifically rigorous research about effective feedback, and there are few resources 

dedicated to teaching principals how to convey the results of the classroom observations in a 

manner that helps teachers improve their instruction and leads to improved student outcomes. 

The teacher feedback process is perhaps the least developed link in new teacher evaluation 

systems. The majority of academic attention has been devoted to the mechanics of the value 

added scores and the relative weight of multiple measures (Mihaly et al. 2013; Bill and Melinda 

Gates Foundation, 2012; McCaffrey, Lockwood, Koretz, Louis, and Hamilton, 2004), as well as 

the development of observation rubrics and their alignment with other measures of student 

learning (Bill and Melinda Gates Foundation, 2012; Pianta and Hamre, 2009; Hill, Kapitula and 

Umland, 2010). Yet the quality and clarity of the feedback is the primary means by which 

teacher evaluation systems would allow teachers to recognize the aspect of their practice that 

needs improvement and could encourage teachers to seek out appropriate professional 

development and to improve teachers’ instruction. It is therefore a critical point in the teacher 

evaluation cycle to study.

The research literature provides some limited evidence that teachers improve their instruction 

when they receive professional learning opportunities that are ongoing and closely connected to 

teachers’ curriculum and instruction (Supovitz and Turner, 2000; Correnti, 2007; Correnti and 

Rowan, 2007), as well as those that are aligned with what school districts value (Penuel et al., 

2007; Garet et al., 2001). Given that teacher-principal conferences are meant to occur throughout 

the year and directly reflect what teachers are doing in their classrooms, those conferences could 

thus be an important mechanism for encouraging both teacher growth and directing teachers to 

appropriate professional development. Most recently, Hill (2009) provides evidence that 

professional development remains largely ineffectual for improving practice, often coming in the 

form of short-term, low-quality training that teachers find difficult to apply in their classrooms. If 

tightly linked to teachers’ observed classroom practices, recommended professional development 

could become increasingly relevant to teachers’ instructional practices. (Little, 1993; Ball and 

Cohen, 1999; Wilson and Berne, 1999).

To date, only two studies have examined the feedback given to teachers by principals, and both 

are based on the evaluation system in Chicago Public Schools. The University of Chicago 

 

2

See, for example, Mihaly et al., 2013; Bill and Melinda Gates Foundation, 2012; Pianta and Hamre, 2009; and Hill, 

Kapitula and Umland, 2010.
11

Consortium on Chicago School Research (CCSR) found that Chicago Public Schools’ 

expectations for principal-teacher conferences pre- and post-observation did not align with 

principal and teachers’ actual practices (Sartain et al., 2011). In that CCSR study, only 10% of 

the questions principals asked of teachers reflected high expectations and required deep 

reflection about instructional practice and 65% of questions required limited response and 

focused on affirmation of the principal’s views. Additionally, principals tended to talk for 75% 

of the conference, and teachers only 25%. Overall, Chicago Public School principals reported a 

need for guidance on how to provide actionable feedback, which is precisely what this study

aims to do for New Mexico principals. A more recent study by CCSR (Sporte et al., 2013) of the 

same teacher evaluation system found that administrators list the provision of useful feedback to 

teachers as an area where principals need professional development. 

The theories employed in our intervention comport with the main lessons from a broad literature 

providing guidance on the most effective methods and features for delivering job performance 

feedback and for conducting performance appraisals. In the field of clinical supervision, 

Heckman-Stone (2003) provides some guidance for the proper structure of performance 

evaluation and feedback based on observations from supervision of students in the clinical 

practices. In providing feedback, she recommends that feedback standards should be agreed 

upon and understood by both parties and be provided regularly to build a strong relationship and 

avoid misunderstanding. Additionally, positive feedback should be provided prior to negative 

evaluations, and that any feedback should be properly contextualized into specific skill areas so 

that a supervisee is clear on what skills are being assessed and how they are being assessed. The 

standard professional development that the New Mexico PED offers in the summer to all 

principals about NM TEACH covers some, but not all, of these principles. The business-as-usual 

guidance to principals does not spell out ways to offer positive feedback before negative

feedback, does not contextualize feedback so that the supervise is clear on what skills are being 

assessed (i.e., no reference to the NM Observation Rubric), and does not include a discussion of 

how teachers are being assessed.

In both health and education research, several simple, low-cost interventions such as the one 

studied here have proven effective. Most relevant to this study, the distribution of a one-page 

checklist of tasks before surgery with common-sense items on it such as nurses and surgeons 

should introduce themselves before surgery and correctly time the use of antiseptics had the 

extremely large impact of reducing surgical complications by 36% and deaths by 47% (Haynes 

et al., 2009 in Yeager & Walton, 2011). The author of the surgery checklist, Atul Gawande, 

claims that the surprisingly large effects of the checklist is due to health care professionals 

simply (and erroneously) assuming that surgeons and nurses were doing all the items on the 

checklist without verifying that fact. New Mexico PED staff with whom the SW REL

researchers have spoken profess a similar assumption that principals and teachers will carry out 

common-sense steps in feedback conversations (e.g., review in detail the ratings on the 
12

observations on NM TEACH Observation Rubric, offer both positive and negative feedback, 

mutually develop a set of next steps for the teacher where the professional development 

explicitly ties to the scores on the Observation Rubric) but admit that they have no way to verify

that assumption. The limited research on feedback within these new teacher evaluation systems 

like NM TEACH suggest that feedback conversations do not adhere to the seemingly commonsense guidelines provided by NM PED-sponsored professional development or by the Carnegie 

checklist. 

Yeager and Walton (2011) review six education studies that have had substantial, lasting impacts 

on student achievement even though the interventions are short, provide information-only, do not 

teach academic content but instead seek to influence students’ beliefs and attitudes about school. 

The authors deduce two lessons from the interventions in the six studies: (1) the interventions 

use students’ perspectives to influence student behavior, and (2) they use “stealth tactics” like 

requiring fewer hours of study participants time to receive the session, not stigmatizing students, 

and not being didactic. Our study builds off of this work on psychological interventions by

examining a low-cost, brief intervention tailored to New Mexico that could theoretically improve 

the quality of feedback for teachers.

Given the critical role of principal feedback in teacher evaluation systems and the limited 

research about it, the study proposed here is a cluster randomized controlled trial evaluation to 

estimate the impact of providing principals and their teachers with a detailed checklist and video 

suggesting how to structure the formative observation feedback conversation—as compared to a 

reminder to principals about the 5 steps for feedback outlined in the required state-sponsored 

professional development session —on the content and structure of the post-observation 

conference, the perceived quality of the feedback itself, the quality and take-up of professional 

development for teachers, teacher practice as measured by subsequent formal classroom 

observation scores, and, by extension, student achievement. The combination of the 

Conversation checklist and the video are intended to be a low-cost intervention—an intervention 

that primarily consists of information—that capacity-constrained state departments of education 

could theoretically implement with success. 

As stated in the overview, the study proposed here is a cluster randomized controlled trial 

evaluation to estimate the impact of providing principals and their teachers with about a checklist 

to follow during the feedback conversations regarding teachers’ instruction relative to the 

minimal guidance offered currently to principals about effective feedback. Blocking by school 

district, a randomly selected half of principals who agree to participate in the study (along with 

teachers in their school who also agree to participate in the study) will receive the Conversation 

Protocol, which contains an introduction about the importance of feedback, a feedback

conversation checklist to be used during the post-observation conference, and a link to a video 

testimonial in which a principal or a teacher describe their experience using the checklist. The 
13

checklist is a customized version of the Teacher Feedback Conversation Protocol provided in 

“Strategies for Enhancing the Impact of Post-Observation Feedback for Teachers,” developed by

coauthors at the Carnegie Foundation for the Enhancement of Teaching (Myung and Martinez, 

2013). While this version of the Conversation protocol has not been used in New Mexico, The 

Vision Network, located in Delaware, has used the Carnegie version of the checklist with 

principals in the 2013-2014 school year and will again use it in the 2014-2015 school year. The 

designer of the checklist, Myung, also reports that other schools have piloted the use of the 

checklist, but we are not aware of its widespread usage as an official tool adopted by districts or 

state education agencies.

To conduct a placebo randomized controlled trial, ED contractors will also provide a businessas-usual guide to control group principals and control group teachers participating in the study.

The control guide for principals will list the five steps for feedback discussed in the summer 

2014 professional development attended by NM principals. This will mimic “business-as-usual” 

for principals in New Mexico, which is professional development only provided to principals and 

not to teachers about feedback. The control group principals and teachers will not receive a link 

to the video nor will they receive the checklist. Meanwhile, to mimic “business-as-usual” for 

teachers, control group teachers will receive a short PDF with content from the state NM PED 

website reminding them of their right to a post-observation feedback conversation within 10 

calendar days of the observation.

The treatment guide (called the Conversation Protocol) will go beyond business as usual by

providing a brief introduction explaining the importance of a feedback, providing a video 

testimonial, and, most importantly, providing a detailed checklist for the feedback session. 

Checklists have been shown to have large impacts on practice in other settings (Haynes et al., 

2009). ED contractors have applied the following four lessons from other effective simple, lowcost, information-only interventions in designing this study:

1. For the treatment group, systematize the information provided to bridge both the 

temporal gap between in-person professional development and feedback conversations 

and the conceptual gap between theory and practice by offering an easy-to-follow 

checklist for both principals and teachers to break down the feedback conversation 

into discrete steps for use before and during the feedback session. 

2. For the treatment group, present a video (one video for principals; and second video 

for teachers) about the checklist from a principal’s (teacher’s) perspective to minimize 

their resistance to the message, rather than have professional development consultants 

or state or district staff persons tell principals and teachers what to do.

3. Avoid stigmatizing either the treatment or control group by framing the study as trying 

out two alternatives for improving feedback to teachers. 

4. Keep the information brief to respect principals’ and teachers’ busy schedules, and 

because interventions requiring fewer hours of time have proven more effective than 
14

longer lasting interventions in several relevant education studies (Yeager & Walton, 

2011).

The Conversation Protocol conforms to the schedule of formal observations and the required 

elements of NM TEACH Observation Rubric and is structured based on the Carnegie Protocol 

with the following designated sections:

A. Open warmly and get down to business. (Thanks for meeting with me. Could you 

remind me of the lesson goal? Let’s clarify what we want to accomplish.)

B. Focus on what’s going well. (What worked? In addition, here’s what I saw.)

C. Identify challenges facing the teacher. (What were some things you feel could 

have done better? It sounds like the challenges are ....)

D. Generate ideas for addressing the challenges. (Let’s approach these one by one...)

E. Prioritize next steps.

F. End positively (Was this conversation helpful? Thanks for your time and 

insights.)

The theory of change, presented in Figure 1, emphasizes the links in the chain that might be 

expected to connect principals’ feedback with changes to teacher instruction and, subsequently, 

to student learning. We envision that the process starts with a principal’s formal observation of a 

teacher’s classroom lesson. Any expected impacts of that observation on teacher instruction and 

student learning would then depend upon whether the teacher and principal used the guide both 

to reflect on the instructional strengths and challenges in the lesson and discuss those strengths 

and challenges in a way that led to actionable and useful feedback to the teacher during the 

teacher-principal conference. Then, subsequent links in the chain depend upon the teachers’ 

positive perceptions of the teacher-principal conference and a desire to seek out the 

recommended professional development (PD), the quality of the PD itself and the extent to 

which the teacher is able to utilize the PD to improve his/her instruction.

Figure 1 – Theory of Change
15

ED contractors propose to collect both primary and secondary data to understand the impact of 

providing the Conversation Protocol to principals and teachers on principals and teachers’ 

perceptions about the quality of feedback, teachers’ professional development, teachers’ 

instructional quality as measured by classroom observation scores and student achievement. The 

primary data will consist of a principal and a teacher Web-based baseline and follow-up survey

(for four total survey instruments). The secondary data consists of formal observation scores by 

school leaders of teacher practice as well as principal, teacher and school characteristics and 

student achievement scores collected by NM PED. 
16

Table 2 shows the timeline for all three data collection activities described in this OMB package.

Table 2. Data Collection Timeline

Data Collection Purpose

Requesting 

OMB 

Clearance?

May / 

June

2015

May/ 

June

2016

Extant data for 

recruitment from 

New Mexico Public 

Education 

Department

 Email addresses and first and last 

name for principals and teachers 

to recruit them into study via 

email (by April 1, 2015)

Yes X

Extant 

administrative 

school and student 

data from New 

Mexico Public 

Education 

Department 

 Teachers’ NM TEACH 

Observation Rubric scores (RQ5)

 Student achievement data from 

spring state standardized test 

(RQ 9)

 Description of study schools, 

principal qualifications, teacher 

qualifications, and student 

population (RQ10)

Yes X X

Principal surveys  Perceptions of post-observation 

conferences (RQ1, 2)

 Behaviors in post-observation 

conferences with teachers (RQ3, 

4, 7, 8, 10)

 Use of checklist and guidance 

(RQ 11, 12)

Yes X X

Teacher surveys  Perceptions of post-observation 

conferences (RQ1, 2)

 Behaviors in post-observation 

conferences with teachers (RQ3, 

4, 7, 8, 10)

 Use of checklist and guidance 

(RQ 11, 12)

Yes X X

See Appendix A for the teacher and principal survey instruments, and Appendix B for the 

Recruitment Materials. Appendix C contains the secondary data elements to be provided by NM 

PED to ED Contractors. Appendix D summarizes the suggestions received ED’s contractors for 

this study and the responses to those suggestions. The data collection involved in this study is 
17

authorized by the Educational Sciences Reform Act (ESRA) of 2002. Please see Appendix E for 

the ESRA. Appendix F provides the IRB approval by Human Subjects Protection Committee at 

the research organization carrying out the study. Appendix G provides the written consent forms 

for study participants.

2.		How,	by	Whom,	and	for	What	Purpose	the	Information	Is	to	Be	Used

ED’s contractor for REL Southwest will analyze the data to be collected through this study using 

statistical models and procedures that are preapproved by the Institute of Education Sciences 

(IES). The contractor will then summarize the findings in a report that will undergo review for 

quality and relevance by the National Center for Education Evaluation and Regional Assistance’s 

(NCEE’s) external review contractor. After the report has undergone IES review, findings will 

be presented to members of the NM PED (the primary audience) and published through IES for 

educators and education researchers (secondary audience).

The primary audience for this study is NM PED, and other state and local agencies considering 

ways to improve feedback from principals to teachers. The results from all of the research 

questions will be used by New Mexico to decide whether to adopt the checklist and video as a 

mandatory or recommended element of teacher evaluation. The study will also offer detailed 

information to NM PED about teachers’ use of professional development (PD), their perceptions 

of it, the PD topics they pursue, and PD recommended to them. NM PED currently has no way to 

gather this information, because PD is arranged for and determined by local school districts in 

the state. In addition, the study will provide NM PED with detailed feedback from principals and 

teachers about the state’s teacher evaluation system, supports for it (or lack thereof), and 

suggestions for improvement. Answers to the exploratory and implementation research questions 

could help NM PED fill gaps in their knowledge regarding district and school practices in 

professional development and teacher feedback. 

More generally, the results of the study will provide concrete evidence to state policymakers 

regarding the utility of low-cost, easy to implement conversation checklists, scripts, and 

testimonial videos if they can encourage teachers both to seek out appropriate professional 

development opportunities and improve their instruction. Should this New Mexico Guide 

improve teachers or principals’ description of the structure and content of the feedback 

conversation and perceptions of the feedback process or subsequent classroom observation 

scores, it suggests that other localities may benefit from developing customized checklists built 

around their evaluation system but incorporating the core theories of effective performance 

feedback. If the checklist and video do not influence then it suggests that the checklist and video 

do not add value to the pre-existing professional development about classroom observation, and 

an examination of more intensive interventions is warranted. 
18

To provide this information to the primary audience and the secondary audience, ED’s contractor 

must collect the four forms of data described in Table 2. Namely, the contractor must collect 

extant data from NM PED that includes:

 teacher and principal contact information for ED’s contractor to recruit participants into 

the study;

 principals’ NM TEACH Observation Rubric ratings of teachers’ classroom to determine 

if classroom ratings improve following distribution of the treatment and control guide

(research question 5);

 student-level achievement data to determine if student achievement improves in 

treatment and control schools following distribution of the guide (research question 9); 

and

 school-level, teacher-level, and principal-level characteristics to determine if those

characteristics moderate take-up and perceptions of the treatment and the control group 

guide (research question 10).

We estimate that NM PED will devote a total of 20 hours to provide ED’s contractors with 

contact information and then another 105 hours from NM PED for school-, teacher-, and 

principal-level data. 

The contractor must also administer a principal survey and a teacher survey at two time 

points to determine if there were changes in study participants’:

 Perceptions of the post-observation conferences following receipt of the treatment and 

control guide (research questions 1 and 2).

 Behaviors in the post-observation conferences following receipt of the treatment and 

control guide (research question 3 and 4).

 Perceptions and use of the guidance they received (research questions 6, 7, 8, 11, and 12).

The surveys will take up to 30 minutes to complete, and they will be administered to all study

participants in spring 2015 and again in spring 2016. For the purposes of this OMB package, it is

assumed that 600 principals and 3,000 teachers will complete the spring 2015 and the spring 

2016 surveys. 

3.		Use	of	Information	Technology

The data collection will be conducted in a manner that utilizes technology to reduce respondent 

burden and improve accuracy. Where feasible, REL SW researchers will collect all possible data 

from administrative sources rather than through primary data collection. For example, electronic 

school records on teachers, principals and schools will be gathered from NM PED and 

transferred directly to ED contractors using secure file transfer procedures, and thus reduce the 

number of questions posed to teachers and principals in their surveys. Clear instructions will be 
19

provided on the method of data transfer from NM PED to ED Contractors. 

For data that can only be obtained directly from principals and teachers, ED contractors will 

employ MMICTM (Multimode Interviewing Capability), a comprehensive information system 

developed by RAND, to administer the surveys.3 MMIC is used to manage the whole data 

collection process from questionnaire design, sample management, and fieldwork monitoring.

Using MMIC, ED contractors will email to study participants a link to online surveys. To reduce 

the burden on respondents, the MMIC software is flexible, and allows survey respondents to 

participate using a multitude of devices like computers, PDAs, and Smart phones, and to switch 

between devices while completing the survey. When requested, questionnaires will be 

transmitted to and from the respondent by fax. A telephone number to a staffed MMIC help desk 

and electronic mail address are included in the questionnaire if anyone has questions. These 

procedures are designed to minimize the survey burden on respondents. 

4.		Efforts	to	Avoid Duplication of	Effort

In an effort to avoid duplication of effort, this study will use extant administrative records where 

possible to understand the impact of the Conversation Protocol. As described in Table 2, ED’s 

contractors will collect principals’ rating scores of teachers on NM TEACH Observation Rubrics 

from secondary data sources collected by NM PED, rather than ask principals to provide this 

information in surveys. As a part of this same secondary data request, ED’s contractors will also 

collect school-level characteristics such as size, level (elementary, middle, high), accountability 

status, teacher-level characteristics such as degree earned, race, gender, and job title, principallevel characteristics such as degree earned, race, gender, and job title, and student-level 

characteristics like student achievement to minimize the length of surveys administered directly 

to principals and teachers. The only data collected that will be unique to this study are the webbased principal and teacher survey data, which are also described in Table 2.

5.		Sensitivity	to	Burden	on	Small	Entities

The use of administrative records will reduce the burden on school educators by ensuring that 

only the minimum amount of original data is requested from schools in order to meet the 

objectives of this study. Aside from surveys emailed directly to participating principals and 

teachers, ED’s contractors will not contact schools to request additional data.

6.		Consequences	of	Less	Frequent	Collection	and	Obstacles	to	Burden	

Reduction

The Education Science Reform Act of 2002 states that the central mission and primary function 

 

3

For more information, see http://www.rand.org/labor/mmic.html
20

of the regional education laboratories is to support applied research and provide technical 

assistance to state and local education agencies within their region (ESRA, Part D, section 

174[f]). If the proposed data were not collected, REL Southwest would not be fulfilling its 

central mission to serve the states in the region and provide support for evidence-based research.

The research questions and the intervention contemplated in this study respond to the concerns 

raised by NM PED, which is a constituent member of the REL Southwest. If the proposed data 

were not collected, NM PED and, by extension, other SEAs would not know whether a low-cost 

informational intervention such as this one has an impact on the quality of principal feedback to 

teachers, and by extension academic outcomes.

This is a one-time study (i.e., not recurring) and therefore periodicity is not addressed.

7.		Circumstances	Requiring	Special	Information	Collection

There are no special circumstances.

8.		Federal	Register	Announcement	and	Consultation	Outside	the	Agency

a.	Federal	Register	Announcement

A 60-day Federal Register Notice was published on [DATE TO BE COMPLETED BY IES]. A

30-day notice was published on [DATE TO BE COMPLETED BY IES].

b.	Consultations	Outside	the	Agency

The evaluation team has consulted the Technical Working Group (TWG) of researchers with 

expertise in large-scale randomization studies and impact evaluation that was formed to review 

studies under the Southwest REL. We have consulted the TWG on the overall study design, data 

collection and survey instruments. Major recommendations from the TWG are included in 

Appendix D. 

Members of the TWG include:

Dan Goldhaber, Ph.D.

Director, CALDER (National Center for Analysis of Longitudinal Data in Education Research)

Vice President, American Institutes for Research (AIR) 

Director, Center for Education Data & Research (CEDR), University of Washington Bothell

Co-Editor, Education Finance and Policy

3876 Bridge Way N, Suite 201

Seattle, WA 98103

Ph: 206-547-1562

Fax: 206-547-1641
21

E-mail: dgoldhab@uw.edu

Geoffrey Borman, Ph.D.

Professor of Education, University of Wisconsin—Madison

Deputy Director of the University of Wisconsin's Predoctoral Interdisciplinary Research 

Training Program

Senior Researcher, Consortium for Policy Research in Education.

348 Education Building

1000 Bascom Mall

Madison, WI 53706-1326

Ph: 608-263-3688

Fax: 608-265-3135

E-mail: gborman@education.wisc.edu

Johannes M. (Hans) Bos, Ph.D.

Vice President and Program Director, International Development, Evaluation, and Research 

(IDER) Program

American Institutes for Research

2800 Campus Drive, Suite 200

San Mateo, CA 94403

Ph: 650-843-8100

Fax: 650-843-8200

E-mail: jbos@air.org

W. Steven Barnett, Ph.D.

Board of Governors Professor and Director of the National Institute for Early Education 

Research

Rutgers University

73 Easton Avenue

New Brunswick, NJ 08901

Ph: 848-932-4350 x23132

Fax: 732-932-4360

E-mail: sbarnett@nieer.org

c.	Unresolved	Issues

There are no unresolved issues.
22

9.		Payments	or	Gifts	to	Respondents

ED contractors propose to provide teachers and principals a $25 gift certificate upon completion 

of each of two 30-minute surveys (baseline survey before the intervention, and follow-up survey

after the intervention), for a total of $50 per person. These amounts were set based on the hourly 

wage rate of principals ($44.23/hour) and teachers ($32.52/hour) and the estimated length of the 

survey. Incentives will be distributed electronically (i.e., a link to a gift card) after respondents 

complete the surveys.

The only other data for the study will be obtained from NM PED, which has already agreed in 

concept to provide such data and will not be compensated for the time required to draw together 

the data files for transmission to ED’s contractor. As a constituent member of the REL

Southwest, NM PED identified this topic as of interest of them, and thus the study is being done 

at their request.

10.		Assurances	of	Confidentiality

The data collection efforts that are the focus of this clearance package will be conducted in 

accordance with all relevant federal regulations and requirements. The Southwest REL will be 

following the new policies and procedures required by the Education Sciences Reform Act of 

2002, Title I, Part E, Section 183 requires “All collection, maintenance, use, and wide 

dissemination of data by the Institute” to “conform with the requirements of section 552 of title 

5, United States Code, the confidentiality standards of subsection (c) of this section, and sections 

444 and 445 of the General Education Provision Act (20 U.S.C. 1232g, 1232h).” These citations 

refer to the Privacy Act, the Family Educational Rights and Privacy Act, and the Protection of 

Pupil Rights Amendment.

All survey responses will be kept strictly confidential, and will only be used for the purpose of 

the study. No one at the school, district, or the state will have access to survey responses that 

include respondents’ names, school names, or other information that could potentially be used to 

identify individuals or schools. The project has been approved by RAND’s Human Subjects 

Protection Committee, which serves as RAND’s Institutional Review Board (IRB00000051) to 

review research involving human subjects. RAND is registered with the Office of Human 

Research Protection (OHRP) as a research institution (IORG0000034). RAND's Federal wide

Assurance for the Protection of Human Subjects (FWA00003425, effective until July 1, 2018) 

serves as our assurance of compliance with federal regulations.

In addition, for student information, the data collection efforts will ensure that all individually

identifiable information about students, their academic achievements, their families and 

information with respect to individual schools, shall remain confidential in accordance with 

section 552a of Title 5, United States Code, the confidentiality standards of subsection (c) of this 
23

section, and sections 444 and 445 of the General Education Provision Act. The study will also 

adhere to requirements of subsection (d) of section 183 prohibiting disclosure of individually

identifiable information as well as making the publishing or inappropriate communication of 

individually identifiable information by employees or staff a felony.

ED contractors will protect the confidentiality of all information collected for the study and will 

use it for research purposes only. No information that identifies any study participant will be 

released publicly. Information from participating institutions and respondents will be presented 

at aggregate levels in reports. Information on respondents will be linked to their institution type 

(e.g., elementary schools vs. middle schools) but not to any individually identifiable information. 

No individually identifiable information will be maintained by the study team upon study

completion.

Respondents will be assured that all information identifying them and the school will be kept 

confidential. All members of the study team have obtained their certification on the use of 

human subjects in research. The following safeguards are routinely employed at RAND, the 

contractor that will execute this study, to carry out confidentiality assurances:

 All employees at ED’s contractor working on this project will sign a confidentiality pledge 

emphasizing its importance and describing their obligations under it (please see Appendix H 

for the confidentiality pledge). 

 All research projects that have access to identifiable private or proprietary data need to have 

a Data Safeguarding Plan reviewed and approved by RAND’s Human Subjects Protection 

Committee. The Data Safeguarding Plan includes information on who is responsible for data 

safeguarding, the types of sensitive information to be transferred and stored, the mode of data 

transfer, client and respondent agreements, disclosure risks, audit and monitoring plans, and 

the procedures to be employed for data safeguarding.

 Any electronic transmission and sharing of individually identifiable data will be encrypted. 

This procedure will prevent anyone without permission to access and enter the data system

 Access to the data shall be limited to the minimum number of individuals necessary to 

achieve the approved purpose and to those individuals on a need-to-know basis only.

 Identifiable data will be stored in a locked container when not in use. We will store original 

and derivative data files only on disks (e.g. servers, local hard disks) that are not routinely 

backed up. We will keep all hardcopy materials containing sensitive data in a locked file 

cabinet when not in use. 

 When no longer needed, we will discard sensitive output in a shredder or sensitive-waste 

container. We will destroy all individual linkages to data after a respondent ceases 

participation in the project. 

Also, the REL study team has submitted to the NCEE security officer a list of the names of all 

people who will have access to respondents and data. The contractor, on behalf of ED, will track 
24

new staff and staff who have left the study and ensure that signatures will be obtained or 

clearances revoked, as necessary.

ED’s contractor will make certain that all data are held in strict confidentiality, as just described, 

and that in no instance will responses or data be made available except in in aggregate statistical 

form. The following statement will appear on all letters to respondents on data collection:

Per the policies and procedures required by the Education Sciences Reform Act of 2002, 

Title I, Part E, Section 183, responses to this data collection will be used only for 

statistical purposes. The reports prepared for this study will summarize findings across the 

sample and will not associate responses with a specific district, school, or individual. The 

contractor will not provide information that identifies you or your district to anyone 

outside the study team, except as required by law. Any willful disclosure of such 

information for nonstatistical purposes, without the informed consent of the respondent, is 

a class E felony.

11.		Justification	for	Sensitive	Questions

Teachers completing the survey will be asked about the frequency, structure and content of the 

post-observation feedback conference, the type of professional development courses they were 

recommended to take and completed, the quality of the professional development courses offered 

by the school district, their opinions about the effectiveness of the school leadership and the 

teacher evaluation system. Teachers will also be asked more sensitive questions, such as to rate 

their own performance and to rate the quality of the school leadership. These questions are 

sensitive because teachers would risk embarrassment or damage to their reputation or 

employability were their answers to be revealed. As noted above, all responses will be kept 

confidential and the data safeguarded. The questions regarding teachers’ own ratings of 

performance are important to anchor the principal as a strict or lenient rater, and the questions 

regarding the quality of school leadership are an important control variables to establish the

school environment. Principals completing the survey will be asked about the structure and 

content of the post-observation feedback conference, the type and quality of professional 

development offered by the school district, the training they received on the observation rubric, 

and their opinions about the teacher evaluation system. Principals will also be asked to answer 

similar sensitive questions such as to rate the quality of his/her teachers. While all responses will 

be kept confidential, these questions are important to establish the environment of the school.

In addition, participants will be informed that their responses are voluntary, and they may decline 

to answer any question.

12.		Estimates	of	Hour Burden

There are three components for which ED’s contractor has calculated hours of burden for this 

clearance package: recruitment activities, extant data provided by the state to ED’s contractor, 
25

and survey data collected from study participants by ED’s contractor. Table 3 shows the hourly 

burden overall and for all three components. The total burden associated with this study, across 

three study years, is 5,352.75 hours, with an annualized burden of 1,784.25 hours over three 

years. The recruitment burden is 1,107.75 hours, the extant data collection burden is 105 hours, 

and the survey data collection burden is 4,140 hours. The annualized number of responses is 

10,800 (for a total of 31,743 across all three years).

Table 3. Total Estimated Hourly Burden

Instrument

Person 

Incurring 

Burden

Number 

of 

responden

ts

Response

s per 

Responde

nt

Total 

Respons

es

Hours 

per 

Respons

e

Total 

Burden 

Hours

Recruitment

Principal and Teacher 

Contact Information 

(name, email)

State Data 

Manager 1 3 3 10 30

First Principal contact 

(e-mail)

Principal 855 1 855 0.05 42.75

Round 1 Follow-up for 

nonresponding 

Principal

Principal 500 1 500 0.05 25

Round 2 Follow-up for 

nonresponding 

Principal

Principal 400 1 400 0.05 20

Round 3Follow-up for 

nonresponding 

Principal

Principal 300 1 300 0.05 15

First Teacher contact 

(e-mail)

Teacher 6,000 1 6,000 0.05 300

Round 1 Follow-up for 

nonresponding 

Teacher

Teacher 5,000 1 5,000 0.05 250

Round 2 Follow-up for 

nonresponding 

Teacher

Teacher 4,500 1 4,500 0.05 225

Round 3Follow-up for 

nonresponding 

Teacher

Teacher 4,000 1 4,000 0.05 200

Subtotal --- 21,556 --- 21,558

10.4

(across 

respons

es)

1,107.75

Extant Data 

Collection
26

Instrument

Person 

Incurring 

Burden

Number 

of 

responden

ts

Response

s per 

Responde

nt

Total 

Respons

es

Hours 

per 

Respons

e

Total 

Burden 

Hours

Teacher and School 

Administrative Data

State Data 

Manager 1 3 3 15 45

Teacher Observation 

Score Data

State Data 

Manager 1 3 3 20 60

Subtotal --- 1 --- 6

35 

(across 

respons

es)

105

Survey Data 

Collection

Principal Consent Principal 600 1 600 0.15 90

Teacher Consent Teacher 3,000 1 3,000 0.15 450

Wave 1 Principal 

Survey Principal 600 1 600 0.5 300

Wave 2 Principal 

Survey Principal 600 1 600 0.5 300

Wave 1 Teacher 

Survey Teacher 3,000 1 3,000 0.5 1,500

Wave 2 Teacher 

Survey Teacher 3,000 1 3,000 0.5 1,500

Subtotal --- 10,800 --- 10,800 2.3 4,140

Totals --- 32,357 --- 32,364

47.7

(across 

respons

es)

5,352.75

Burden	for	Study	Recruitment

Our recruitment strategy will focus on both New Mexico public school principals and New 

Mexico public school teachers. ED’s contractor will conduct recruitment in a top-down 

approach, with principal recruitment preceding teacher recruitment. Our burden table is a 

conservative estimate of the highest potential burden with this recruitment approach, but ED’s 

contractor anticipates that fewer districts and schools will actually be involved in recruitment 

activities. 

First, ED’s contractor will collect contact information for principals and teachers from the state 

data manager at NM PED, with a second collection 2 months later to ensure that the information 

is up to date, at a total burden of 20 hours for NM PED. Principals will be contacted by email, 

with an estimated burden of 3 minutes to read and respond to the email (0.05 hours). We will recontact non-responding principals up to three times via email and then twice by letter mailed to 

their school. The target sample size for initial contact with principals is 855 principals with an 
27

estimated response rate of 70 percent (600 principals). Once the principals have been enrolled in 

the study, we will then contact up to 10 teachers in the school by email, with an estimated burden 

of three minutes per teacher per email to read and respond to the email. We will re-contact nonresponding teachers up to three times via email and then twice by letter mailed to their school. 

The target sample size for initial contact with teachers is at most 6,000 teachers, with an 

estimated sample of at most 3,000 teachers consenting to participate in the study.

Burden	for	Extant	Data	Collection

The total estimated burden for NM PED to compile and transmit secondary data to ED’s 

contractors is 105 hours. This calculation assumes one data manager works a collective total of 

2.625 weeks on compiling the data request. 

Burden	for	Survey	Data	Collection

The total estimated burden for survey data collection for the impact and implementation study is 

4,140 hours. Principals and teachers will each be consented into the study, and will be surveyed 

two times, once at baseline, and a second time one year later. 

13.		Estimated	Total	Annual	Cost	Burden	to	Respondents	or	Recordkeepers

Total	Cost	to	Respondents

The total cost to respondents for the two components of this study—recruitment activities and 

survey data collection—is provided in Table 4. The total respondent cost associated with this 

study is approximately $184,590. The annualized cost for each year of the three-year study is 

$61,530. The recruitment cost is $37,152; the respondent cost for the data collection is $147,438.

Table 4. Total Cost to Respondents

Instrument

Person 

Incurring 

Burden

Total 

Burden 

Hours

Hourly 

Wage 

Rate

Total 

Respondent 

Cost

Recruitment

Principal and Teacher Contact 

Information (name, email)

State Data 

Manager 20 $45 $900

First Principal contact (e-mail) Principal 42.75 $44.23 $1,891

Round 1 Follow-up for nonresponding 

Principal Principal 25 $44.23 $1,106

Round 2 Follow-up for nonresponding 

Principal Principal 20 $44.23 $885

Round 3Follow-up for nonresponding 

Principal Principal 15 $44.23 $663

First Teacher contact (e-mail) Teacher 300 $32.52 $9,756

Round 1 Follow-up for nonresponding Teacher 250 $32.52 $8,130
28

Instrument

Person 

Incurring 

Burden

Total 

Burden 

Hours

Hourly 

Wage 

Rate

Total 

Respondent 

Cost

Teacher

Round 2 Follow-up for nonresponding 

Teacher Teacher 225 $32.52 $7,317

Round 3Follow-up for nonresponding 

Teacher Teacher 200 $32.52 $6,504

Subtotal --- 1,097.75 --- $37,152

Survey Data Collection

Principal Consent Principal 90 $44.23 $3,981

Teacher Consent Teacher 450 $32.52 $14,634

Wave 1 Principal Survey Principal 300 $44.23 $13,269

Wave 2 Principal Survey Principal 300 $44.23 $13,269

Wave 1 Teacher Survey Teacher 1,500 $32.52 $48,780

Wave 2 Teacher Survey Teacher 1,500 $32.52 $48,780

Teacher and School Administrative Data State Data 

Manager 45 $45 $2,025

Teacher Observation Score Data State Data 

Manager 60 $45 $2,700

Subtotal --- 4,245 $147,438

Totals --- 5,342.75 $184,590

Note: The hourly wage rates used to calculate total respondent cost for principals are based on the salary schedule of 

Albuquerque High school principals with more than 10 years of experience (http://www.aps.edu/humanresources/salary-schedules/salaries/spe-salary-schedule). For teachers, the salary is based on the salary schedule of 

Albuquerque Level III teachers with a doctorate (http://www.aps.edu/human-resources/salary-schedules/salaries/at3-

salary-schedule). For the data manager hourly wage rates are based on ED’s contractor’s knowledge of New 

Mexico wage rates.

14.		Estimated	Annualized	Cost	to	the	Federal	Government

The total cost to the federal government for work conducted over all four years is $1,133,000 and 

the estimated annualized cost to the federal government for each year of the study is $283,250.

15.		Program	Changes	or	Adjustments

This is a new study.

16.		Plans	for	Tabulation,	Publication,	and	Schedule	for	Project

a.	Tabulation	Plans

All results for REL rigorous studies will be made available to the public through peer-reviewed 

evaluation reports that are published by IES. The datasets from these rigorous studies will be 
29

turned over to the REL’s IES project officer. 

After the study report is finalized, ED’s contractor will prepare restricted-use data files in 

accordance with NCES standards. These files will contain all the primary survey data collected 

for the study with all personal identifiers removed. Thorough documentation will be provided for 

each data file, including a detailed codebook and explanations of the unit of observation, 

weights, and methods for handling missing data. These data will become IES restricted-use data 

sets requiring a user’s license that is applied for through the same process as NCES restricted-use 

data sets. Even the ED contractor would be required to obtain a restricted-use license to conduct 

any work with the data beyond the original evaluation.

b.	Publication Plans

All results for REL studies are made available to the public through peer-reviewed reports that 

are published by IES. The data sets from these studies will be turned over to the REL’s IES 

project officer. These data may become IES restricted-use data sets requiring a user’s license that 

is applied for through the same process as National Center for Education Statistics restricted-use 

data sets (see http://nces.ed.gov/pubs96/96860rev.pdf for procedures related to obtaining and 

using restricted-use data sets). The REL contractor also would be required to obtain a restricteduse license to conduct any work with the data beyond the original report.

The answer to RQ1 – RQ4 will indicate whether the treatment guide relative to the control guide 

improved or otherwise changed principals’ and teachers’ perceptions and execution of postobservation conferences. To analyze the impact of the feedback protocol on teacher and principal 

outcomes, we will examine differences in outcomes between teachers or principals who were in 

schools that were randomly assigned to receive the treatment guide with their counterparts in the 

control group who were assigned to receive the control guide. The data for the evaluation is 

hierarchical, with students and teachers nested within school (or principals), nested within 

districts. Because units within a group are not statistically independent, ED’s contractors will 

use hierarchical linear modeling to account for the statistical dependence of the error terms. 

Randomization will occur at the principal/school level within district, and ED’s contractors will 

stratify the random assignment by school level. ED’s contractors will estimate the intent-to-treat 

effect in a two-level model for principals and a three-level model for teachers.

Using similar modeling techniques, the answer to RQ5 will indicate whether the treatment guide, 

relative to the control guide, improved principals’ subsequent ratings of teachers’ classroom 

instructional techniques. For each outcome in the impact analysis (RQ1-5), ED’s contractor will 

first construct the outcome measure by using exploratory factor analysis, taking the measure 

directly from the survey or using the measure from Observation Rubric administrative data. 

ED’s contractor will then estimate six regressions, one for each of the following outcomes:
30

 Good Practices Index (measure of post-observation feedback conference content and 

structure)

 Feedback Conversation Perceptions Index (measure of the perceived quality of feedback)

 Average number of minutes it takes to complete the post-observation conference

 PD is tailored to needs identified in formal classroom observation 

 Fraction of targeted Professional Development that was recommended by the principal 

and taken by the teacher 

 Domain level Observation Rubric score

 Observation proficiency level

Analysis for RQ6 and RQ7 will compare all principals and all teachers in the treatment group to 

their counterparts in the control group. These questions will assess the fidelity of 

implementation, the enacted contrast in service between the control and treatment group, and 

will help NM PED understand the likely proportion of users who would take up a guide where 

NM PED to issue one to school staff. 

Analysis for RQ8 through RQ9 will go beyond the hypothesized first-order impacts of guidance 

on post-observations conferences and classroom instruction to understand what factors, if any, 

moderate the use of the guidance (RQ 9) and whether the feedback guidance impacts student 

achievement, which is the ultimate outcome of interest. However, RQ 8 and 9 are framed as 

exploratory because ED’s contractors anticipate that sample size limitations and a single school 

year of implementation may be insufficient to detect statistically significant effects at the 

principal- and teacher-subgroup level and on student achievement. These research questions will 

employ the same statistical model specifications for the impact research questions, with the 

introduction of interaction terms to assess differences in responses of principals and teachers 

according to their membership in the subgroup of interest.

Finally, descriptive analyses for RQ10 and 11 will provide NM PED feedback about how the two 

types of guidance might be improved in future use and barriers to its use. This will allow NM 

PED, and potentially other capacity-constrained State Education Agencies, to adopt a revised 

version of feedback guidance and professional development for users of the guidance that will 

further enhance feedback and subsequent classroom instruction.

No responses or data will be reported for individual staff members, students, or schools. 

Reported data will contain no fewer than four cases per reported table cell to protect 

confidentiality and mask individually identifiable data.

c.	Project Time	Schedule

The timeline for the activities in this project, including data collection, analyses and reporting are 
31

in Table 5. 

Table 5 – Project Time Schedule

Documentation of Institutional Review Board approval May 1, 2014

Draft Office of Management and Budget (OMB) package August 15, 2014

Submit 60 day FRN TBD

Submit 30 day FRN TBD

Draft proposal approved by ED March 12, 2014

Final proposal approved by ED August 15, 2014

Expected OMB Clearance Date April 1, 2015

Start Teacher Recruitment April 15, 2015

Start Principal Recruitment April 15, 2015

Complete Teacher Recruitment and Collect Wave 1 Survey August 7, 2015

Complete Principal Recruitment and Collect Wave 1 Survey August 7, 2015

Complete random Assignment August 15, 2015

Distribute Conversation Protocol to Principals at Treatment 

Schools and Principal Control Guide to Principals at Control 

Schools

August 15, 2015

Distribute Conversation Protocol to Teachers at Treatment 

Schools and Teacher Control Guide to Teachers at Control 

Schools

1

st day of School, 

Fall 2015

Collect First round of NM TEACH Observation rubric scores December 1, 2015

Collect Second round of NM TEACH Observation rubric scores February 1, 2016

Collect Wave 2 Teacher Survey May 1, 2016

Collect Wave 2 Principal Survey May 1, 2016

Collect spring 2016 student achievement data from NM PED June 15, 2016

Complete Data Analyses December 31, 2016

Draft Making Impact Report March 1,2017

Revised Making Impact Report May 1,2017

Draft Stated Briefly Report July 1, 2017

Final Making Impact Report approved by ED September 1,2017

Revised Stated Briefly Report September 1,2017

Restricted Use Files October 15, 2017

Final Stated Briefly Report approved by ED November 30, 2017

17.		Display	of	Expiration	Date

Approval not to display the expiration date for OMB approval is not requested.

18.		Exceptions	to	the	Certification	Requirement

No exceptions to the certificate statement are being sought.
32

REFERENCES

Ball, D.L. and Cohen, D.K. (1999). Developing practice, developing practitioners: Toward a 

practice-based theory of professional education. In L.D. Hammond and G. Sykes (Eds.), 

Teaching as the learning profession: Handbook of policy and practice, pp. 3-32. San 

Francisco: Jossey-Bass.

Bill and Melinda Gates Foundation. (2012). Gathering feedback for teaching: Combining highquality observations with student surveys and achievement gains. Seattle, WA: Author. 

Available: 

http://www.metproject.org/downloads/MET_Gathering_Feedback_Practioner_Brief.pdf

Correnti, R. (2007). An empirical investigation of professional development effects on literacy

instruction using daily logs. Educational Evaluation and Policy Analysis, 29(4): p. 262-

295.

Correnti, R. and Rowan, B. (2007). Opening up the black box: Literacy instruction in schools 

participating in three comprehensive school reform programs. American Educational 

Research Journal, 44(2), p. 298-338.

Garet, M.S., Porter, A.C., Desimone, L., Birman, B.F. and Yoon, K.S. (2001). What makes 

professional development effective? Results from a national sample of teachers. 

American Journal of Educational Research, 38(4): p. 915-945. 

Haynes, A. B., Weiser, T. G., Berry, W. R., Lipsitz, S. R., Breizat, A. H. S., Dellinger, E. P., ... 

& Gawande, A. A. (2009). A surgical safety checklist to reduce morbidity and mortality 

in a global population. New England Journal of Medicine, 360(5), 491-499.

Heckman-Stone, C. (2003). Trainee Preferences for Feedback and Evaluation in Clinical 

Supervision. Clinical Supervisor, 22(1), 21-34.

Hill, H.C. (2009). Fixing teacher professional development. The Phi Delta Kappan, 90(7): 470-

476.

Hill, H.C., L. Kapitula, and K. Umland. (2011). A validity argument approach to evaluating 

teacher value-added scores. American Educational Research Journal, 48(3): p. 794-831.

Little, J.W. (1993). Teachers’ professional development in a climate of educational reform. 

Educational Evaluation and Policy Analysis, 15(2): 129-151

McCaffrey, D.F., Lockwood, J.R., Koretz, D., Louis, T.A. and Hamilton, L. (2004). Models for 

value-added modeling of teacher effects. Journal of Educational and Behavioral 

Statistics, 29(1): 67-101.

Mihaly, K., McCaffrey, D. F., Staiger, D. O., and Lockwood, J. R. (2013). A composite 

estimator of effective teaching. Seattle, WA: Bill and Melinda Gates Foundation.

Myung, J. and Martinez, K. (2013). Strategies for enhancing the impact of post-observation 

feedback for teachers. Stanford, CA: Carnegie Foundation for the Advancement of 

Teaching.

Pianta, R.C. and Hamre, B.K. (2009). Conceptualization, measurement, and improvement of 

classroom processes: Standardized observation can leverage capacity. Educational 
33

Researcher, 38(2): pp. 109-119.

Penuel, W.R., Fishman, B.J., Yamaguchi, R. and Gallagher, L.P. (2007). What makes 

professional development effective? Strategies that foster curriculum implementation. 

American Journal of Educational Research, 44(4): p. 921-958. 

Sartain, L., Stoelinga, S.R., and Brown, E.R. (2011). Rethinking teacher evaluation in Chicago: 

Lessons learned from classroom observations, principal-teacher conferences, and district 

implementation. Chicago: Consortium on Chicago School Research.

Sporte, S.E., Stevens, W.D., Healey, K., Jiang, J. and H. Hart. 2013. Teacher evaluation in 

practice: Implementing Chicago’s REACH Students. Chicago: Consortium on Chicago 

School Research.

Supovitz, J.A. and Turner, H.M. (2000). The effects of professional development on science 

teaching practices and classroom culture. Journal of Research in Science Teaching, 

37(2), p. 963-980. 

Wilson, S.M., and Berne, J. (1999). Teacher learning and the acquisition of professional 

knowledge: An examination of research on contemporary professional development. 

Review of Research in Education, 24: 173-209.

Yeager, D. S., & Walton, G. M. (2011). Social-psychological interventions in education They’re 

not magic. Review of Educational Research, 81(2), 267-301.
34

APPENDIX A	– Teacher	and	Principal	Survey	Instruments

Spring	2015	School	Leader	Survey

[[Version: On-line administration. Wave 1.]] 

Understanding	Your	Participation	in	the	Study

Welcome to the spring 2015 survey for school leaders about the feedback you give to teachers’ 

regarding their instruction as a part of the NM TEACH Educator Effectiveness System. 

Study Purpose. With funding from the federal Department of Education as a part of the 

Southwest Regional Educational Lab, this study is intended to learn about the feedback 

principals give to teachers under the NM TEACH Educator Effectiveness System, and teachers’ 

perceptions of that feedback. For more information about the study, please see {REL Southwest 

website}. In {month, year} REL Southwest researchers asked all New Mexico’s public school 

principals to participate in a study about evaluation feedback to teachers. Results will be used 

to provide New Mexico Public Education Department feedback regarding how to improve the 

NM TEACH Educator Effectiveness System training for principals and teachers.

Survey Purpose. This survey is intended to gather information on the provision of feedback to 

teachers after formal observations by school leaders. REL Southwest researchers will ask study 

participants to fill out an on-line, 30-minute survey once in spring 2015 and again in spring 

2016. Survey results will not be used to evaluate you, other teachers, or school leaders. 

Procedures. Online completion of the survey should take approximately 30 minutes. By using 

your personal survey link provided in the invitation email, you will be able to complete the 

survey in multiple sessions; responses will be saved as you enter them. However, once you click 

“submit” at the very end of the survey, you will no longer be able to return.

Confidentiality. REL Southwest will keep all survey responses confidential and will only use 

them for the purpose of the study. Our reports will contain summaries, and responses will 

never be presented in any way that would permit readers to identify you or your school. No 

one at your school, district, or the state will have access to survey responses that include 

respondents’ names, school names, or other information that could potentially be used to 

identify individuals or schools.

Participants. At the same time that we are emailing you, REL Southwest is also inviting all New 

Mexico public school school-based principals to participate in this study. Up to 10 teachers per 

school are also being surveyed. 

Benefits. Your completion of this survey gives you the opportunity to reflect on and share your 

perspective about the teacher evaluation system. You will be providing valuable information 

that may help New Mexico Public Education Department improve the NM TEACH Educator 

Effectiveness System. 
35

Compensation. Within two weeks of completion of this online survey, you will receive via email 

a link to your $25 online gift card for completing the survey. We will provide another $25 

online gift card upon completion of the spring 2016 survey to each participant.

Risks. The risks associated with your participation in this study are minimal. 

Voluntary Participation. Your participation in this survey is completely voluntary. You can 

decide not to participate or to discontinue your participation at any time. A few survey 

questions, marked with a red asterisk, are required for routing to later questions.

Questions? If you have any questions about this survey or the study as a whole, please feel free 

to call us toll-free at {XXX}. You can also contact {XX}, the survey coordinator, at {email} or by 

phone at {XX}. If you have concerns or questions about your rights as a participant, contact the 

REL Southwest at SEDL (which is responsible for the protection of project participants): {email}, 

toll free at {XX}, or c/o IRB, {address}.

[Required question] Informed Consent. By clicking the box below, you are indicating that you 

have read and understood the information provided to you about your participation in this 

survey.

 I have read and understood the information and choose to participate.

Thank you very much for your help in this important study!

---------------------------------------[NEW SURVEY PAGE]---------------------------------------------

Using	This	Survey

Here are some tips that may help you navigate and complete this survey.

Returning to previous pages. Be sure to use the survey “go back to previous page” button at 

the bottom of the page. Do not use your browser’s back button—it will not work.

Returning to the survey itself. If you would like to complete the survey in multiple sessions—or 

if you accidentally get disconnected from the survey—you can return to your previous 

responses by clicking on the link that was in your invitation email. This will take you to where 

you left off (through the last full page you completed). You can keep returning until you click on 

the “submit” button on the final page of the survey.

Saving your responses. Your responses on any given survey page are saved when you click on 

either button at the bottom of the page (“save and go to next page” or “go back to previous 

page”). 
36

Adding optional comments. Toward the bottom of each survey page is a space where you can 

write in any comments you may have pertaining to the questions on the page. This is 

completely optional.

----------------------------------------[NEW SURVEY PAGE]-----------------------------------------------

Your	Position	and	Job	Responsibilities

[q1]Which of the following best describes your position as a school leader this school year 

(2014-2015)? 

 If you are not a school leader, you may have received this survey in error. Please reply to 

the invitation email or call {XXX}.

 principal 

 vice principal/assistant principal

 director

 dean

 other: _______________________

[q2x]Including this school year (2014-2015), how many years have you been a school 

administrator? (Fill in each box.)

 “School administrator” includes principal, assistant principal, and other equivalent titles.

[q2xa]Years as a school administrator in your current school

[q2xb]Years as a school administrator in your current district 

(including current school)

[q2xc]Years as a school administrator, total, in any location 

(including other districts and current district)

[q3]In addition to your administrative responsibilities, do you currently (2014-2015) have 

official teaching responsibilities (that is, formally instruct K-12 students as the teacher of record 

in one or more classes)?

 Yes

 No

[q4]Do you play a role in filling teaching positions at your school (for example, recruiting, 

interviewing, and/or hiring)? 

 Yes

 No
37

----------------------------------------[NEW SURVEY PAGE]-----------------------------------------------

Your	Perceptions	of	the	Quality	of	Your	School’s	Teaching	Staff	This	Year

[q5] Indicate your level of agreement with the following statement: “Overall, I was satisfied 

with the quality of the whole teaching staff in my school this year (2014-2015).”

[insert slider graphic here. tick marks for disagree strongly, disagree somewhat, agree 

somewhat, agree strongly] 

[q6x] Use the slider to answer each item below. During this school year (2014-2015), how 

many teachers in your school...

None or 

almost 

none A few About half Most

All or nearly 

all

[q6a] Are highly effective 

instructional leaders.

[insert slider graphic here]

[q6b] Are struggling and/or 

ineffective instructional leaders. 

[insert slider graphic here]

[q6c] Had a good grasp of the 

subject matter they teach

[insert slider graphic here]

[q6d] Engaged in collaborative 

conversations about teaching and 

learning on a weekly or biweekly 

basis 

[insert slider graphic here]

-----------------------------------------------------[NEW SURVEY PAGE]----------------------------------

Your	Practices	When	Observing	Teachers	This	Year	(SY	2014-2015)	for	Evaluation	

Purposes	

[q7] Thinking about the entire school year 2014-2015, please estimate the number of teachers 

you will formally evaluate yourself using the NM TEACH Observation Rubric {link}. 

[enter number here that ranges from 0 – 50] 

[q8] Thinking about the entire school year 2014-2015, how many other school leaders besides 

yourself will formally evaluate teachers using the NM TEACH Observation Rubric {link}? 
38

[enter number here that ranges from 0 – 5] 

[q9] Thinking about the entire school year 2014-2015, how often did you personally formally 

observe teachers in your school building using the NM TEACH Observation Rubric {link}? 

[q8] Thinking about the entire school year 2014-2015, how often did you personally formally 

observe teachers in your school building using the New Mexico Observation Rubric {link}? 

 I or another school leader formally observed (for 15+ minutes using the NM TEACH

Observation Rubric) all teachers 3 or more times.

 I or another school leader formally observed (for 15+ minutes using the NM TEACH

Observation Rubric) all teacher 2 times. 

 I or another school leader formally observed (for 15+ minutes using the NM TEACH

Observation Rubric) some teachers 2 times and other teachers 3 times or more.

 Other: ________________________________

[q10] Use the slider to estimate how long post-observation conferences typically took you 

during school year 2014-2015. Think only of conferences after formal observations; do not 

include informal feedback from classroom walk-throughs. 

[insert slider graphic here. Tick marks for 0 minutes to 60 minutes with 10 minute intervals]

[q11] Thinking about the entire school year 2014-2015, how frequently did you provide 

informal feedback about teaching and learning to individual teachers—e.g. walk-throughs, 

wows and wonders. 

 At least once a week. 

 At least once a month.

 At least once per semester.

 Other: ___________________

 Optional comment here:

[q12x] Thinking about all your formal observations from school year 2014-2015, use the slider 

to indicate how much you agree or disagree with the following statements about your postformal observation conferences with teachers. 

Disagree 

Strongly

Disagree 

Somewhat

Agree 

Somewhat

Agree 

Strongly

[q12a] The feedback session itself, separate 

from professional development I may have 

recommended, helped teachers improve 

instruction.

[insert slider graphic here]

[q12b] In each conference, the teacher [insert slider graphic here]
39

brought documents to the conference,

documents such as the lesson plan, PDP.

[q12c] In each conference, I brought 

documents to conference such as the 

teacher’s report card and the most recent 

PDP.

[insert slider graphic here]

[q12d] In each conference, I identified at 

least one positive practice that the teacher 

does well.

[insert slider graphic here]

[q12e] In each conference, I identified at 

least one challenge facing the teacher.

[insert slider graphic here]

[q12f] In each conference, I used my scores

on the NM TEACH Observation Rubric for that 

teacher to determine which instructional 

practices to praise or critique with the 

teacher.

[insert slider graphic here]

[q12g] In each conference, I used my scores

on the NM TEACH Observation Rubric for that 

teacher to determine what professional 

development to recommend to the teacher. 

[insert slider graphic here]

[q12h] In each conference, I ended the 

conference on a positive note.

[insert slider graphic here]

[q12i]Each conference followed a predictable 

format.

[insert slider graphic here]

[q12j] In each conference, the teacher and I 

mutually developed next steps for their

instruction.

[insert slider graphic here]

[q12k] In each conference provided specific

feedback to teachers about their 

performance.

[insert slider graphic here]

[q12l] In each conference I provided 

actionable feedback to teachers about their 

performance.

[insert slider graphic here]

[q12m] In each conference, teachers 

committed to specific set of next steps to 

improve their instruction. 

[insert slider graphic here]

[q12n] In each conference, I spoke for 

almost all of the time during the conference.

[insert slider graphic here]

[q12o] There was a high level of 

collaboration in feedback conferences.

[insert slider graphic here]

[q12p] There was a high level of conflict in [insert slider graphic here]
40

feedback conferences. 

[q12q] It was hard to provide negative 

feedback to teachers about their 

performance.

[insert slider graphic here]

[q12r] The large majority of teachers seemed 

to trust and accept my feedback.

[insert slider graphic here]

[q12s] I feel positive about the feedback I 

gave teachers in conferences.

[insert slider graphic here]

[q12t] I enjoyed most of the post-observation 

feedback conferences.

[insert slider graphic here]

[q12u] I provided all teachers I reviewed with 

a written or online summary of the 

observation with my comments on it. 

[insert slider graphic here]

-----------------------------------------------------[NEW SURVEY PAGE]----------------------------------

Professional	Development	Offered	to Teachers	This	Year	(2014-2015)

The questions in this section ask about professional development for teachers.

 Professional development includes (but is not limited to) workshops, inservices, 

coaching/mentoring, collaborative planning/study, institutes, professional conferences, 

online courses, self-directed learning, and so forth.

[q13x] Use the slider to rate how much you agree or disagree with each of the following 

statements about the professional development experiences of teachers at your school this 

year (2014-2015, including summer 2014).

 Consider professional development offered by the district and other external 

sources as well as professional development offered by the school.

The professional development options for teachers this year....

Don’t know Disagree 

Strongly

Disagree 

Somewhat

Agree 

Somewhat

Agree 

Strongly

[q13a] Were mostly developed by

school leaders or teachers at my

school.

 [insert slider graphic here]

[q13b] Were mostly job-embedded 

rather than one-time only sessions.  [insert slider graphic here]

[q13c]Were varied and numerous 

enough to allow me to customize my  [insert slider graphic here]
41

PD recommendations to teachers 

based on their performance on specific 

domains of the NM TEACH 

Observation Rubric.{link}

[q13d]Helped teachers improve on 

elements measured by the NM TEACH 

Observation Rubric. 

 [insert slider graphic here]

[q13e]Helped teachers improve 

other elements of the NM TEACH 

evaluation besides the NM TEACH 

Observation Rubric (e.g., student 

achievement or “other” multiple 

measure).

 [insert slider graphic here]

[q13f]Were well aligned with the 

Common Core State Standards and/or 

curriculum based on these standards.

 [insert slider graphic here]

[q13g]Have been convenient enough 

to promote teachers’ participation.  [insert slider graphic here]

[q13h]Were sufficiently resourced 

(for example, substitute coverage, 

funding to cover expenses, stipends) 

to allow teachers to participate in the 

professional development they need 

to do their jobs successfully.

 [insert slider graphic here]

[q13i]District-provided professional 

development offerings (workshops, 

inservices, online courses, etc.) have 

enough slots and availability to 

accommodate all teachers who wish 

or need to participate. 

 [insert slider graphic here]

[q13j]Teachers have had easy access 

to a catalog of professional 

development opportunities aligned 

with the NM TEACH Observation 

Rubric.

 [insert slider graphic here]

[q14x] Thinking about the professional development that teachers at your school have 

participated in this year (2014-2015, including summer 2014), use the slider to indicate how 

useful do think each of the following has been for improving teacher effectiveness.

Not 

Applicable 

Not at all 

useful

Slightly 

useful

Moderately 

useful Very useful
42

[q14a]Workshops or inservices for 

teachers at your school only 

(typically on-site)

 [insert slider graphic here]

[q14b]Workshops, inservices, 

institutes, or conferences organized 

by your district for teachers from 

multiple schools

 [insert slider graphic here]

[q14c]Online professional 

development offered by or through 

your district

 [insert slider graphic here]

[q14d]District- or state-sponsored 

induction program, mentoring, or 

other professional development 

activities for new or beginning 

teachers

 [insert slider graphic here]

[q14e]Instructional coaching 

(provided by school-based coaches 

or district coaches)

 [insert slider graphic here]

[q14f]Professional development 

led by me or a school leader such as 

study groups or school-wide staff 

meetings

 [insert slider graphic here]

[q14g]School-based teacher 

collaboration (grade-level or 

subject-area teams, professional 

learning communities, study groups, 

etc.)

 [insert slider graphic here]

[q14h]Videos of sample lessons  [insert slider graphic here]

-----------------------------------------------------[NEW SURVEY PAGE]----------------------------------

Training	You	Received	About	NM	TEACH	to	Date

[q15x]What types of training have you received to date regarding the NM TEACH Educator 

Effectiveness system? (Check all that apply.)

 [q15a][exclusive option]No professional development received

 [q15b]Information on the NM TEACH Educator Effectiveness system(overall purpose, 

specific components, consequences for teachers, relationship to professional 

development, and so forth)

 [q15c]How to use the NM TEACH Observation Rubric
43

 [q15d]How to provide feedback to individual teachers after formal classroom 

observations 

 [q15e]How to enter scores from NM TEACH Observation Rubric into Reflect, the 

Teachscape software

 [q15f]How to use Reflect, the Teachscape software 

 [q15g]How to interpret a teachers’ NM TEACH Summative Evaluation Score

 [q15h]How to give NM TEACH Observation Rubric ratings consistent with those given 

by other raters

 [q15i]How to understand and make use of NM TEACH Summative Evaluation Score to 

help improve teaching at my school

 [q15j]How to direct teachers to professional development based on their NM TEACH 

Summative Evaluation Score

 [q15k]Other:

[q16] [if 15b-k ==yes] Who sponsored the training you attended about NM TEACH Educator 

Effectiveness system? Mark as many as apply.

 State Public Education Department 

 My school district

 Other: __________________

 Don’t know 

[q17] [if 15b-k ==yes] In total, how many minutes of training did you receive this year (2014-

2015, including summer 2014) about how to provide feedback to teachers.

 0 minutes

 15 minutes

 30 minutes

 60 minutes

 120 minutes

 240 minutes

 Other:

[q18] [if 15d==yes] Thinking about the training on how to conduct post-observation 

conferences with teachers, indicate how useful you think the training was. 

 Not at all useful

 Slightly useful

 Moderately useful

 Very useful

[q19] I understand how the overall teacher ratings work in the NM teacher evaluation system 
44

(NM TEACH).

 Disagree strongly

 Disagree somewhat

 Agree somewhat

 Agree Strongly

-----------------------------------------------------[NEW SURVEY PAGE]----------------------------------

Your	Personal	Views	of	NM	TEACH	Educator	Effectiveness	System

[q20x]Indicate how much you agree or disagree with each of the following statements about 

the NM TEACH Educator Effectiveness System from the school year 2014-2015. 

Disagree 

Strongly

Disagree 

Somewhat

Agree 

Somewhat

Agree 

Strongly

[q20a]The NM TEACH Educator Effectiveness 

System does a good job distinguishing 

effective from ineffective teachers.

[insert slider graphic here]

[q20b]The NM TEACH Educator Effectiveness 

System is fair to all teachers, regardless of 

their personal characteristics or those of the 

students they teach.

[insert slider graphic here]

[q20c]The NM TEACH Educator Effectiveness 

System ignores important aspects of teacher 

performance.

[insert slider graphic here]

[q20d]The NM TEACH Educator Effectiveness 

System requires so much of my time that it 

interferes with other important activities.

[insert slider graphic here]

[q20e]I have a clear idea of what the NM 

TEACH Educator Effectiveness System views 

as “good instruction.”

[insert slider graphic here]

[q20f]The NM TEACH Educator Effectiveness 

System has helped me pinpoint specific things 

I can do to help improve my teachers’ 

performance.

[insert slider graphic here]

[q20g]In the NM TEACH Educator 

Effectiveness System, teachers who do not 

score “Effective” or better often leave 

teaching voluntarily.

[insert slider graphic here]
45

[q21x]Indicate how much you agree or disagree with each of the following statements about 

the NM TEACH Observation Rubric. 

Disagree 

Strongly

Disagree 

Somewhat

Agree 

Somewhat

Agree 

Strongly

[q21a]The 6 elements of Domain 1 (Planning 

and Preparation) of the NM TEACH 

Observation Rubric {link} are a useful way to 

measure the quality of teachers’ instructional 

practices. 

[insert slider graphic here]

[q21b] The 5 elements of Domain 2 (Creating 

an Environment for Learning) of the NM 

TEACH Observation Rubric {link} are a useful 

way to measure the quality of teachers’ 

instructional practices.

[insert slider graphic here]

[q21c]The 5 elements of Domain 3 (Teaching 

for Learning) of the NM TEACH Observation 

Rubric {link} are a useful way to measure the 

quality of teachers’ instructional practices.

[insert slider graphic here]

[q21d] The 4 elements of Domain 4 

(Professionalism) of the NM TEACH 

Observation Rubric {link} are a useful way to 

measure the quality of teachers’ instructional 

practices.

[insert slider graphic here]

[q21e] Using the NM TEACH Observation 

Rubric {link} helps me to provide constructive, 

specific feedback to teachers to improve their 

instruction. 

[insert slider graphic here]

-----------------------------------------------------[NEW SURVEY PAGE]----------------------------------

Teacher	Retention	and	Improvement

.

[q22x]Over the past year (since approximately May 1, 2014), how many teachers at your 

school...

None (0 

teachers) 1 teacher 2 teachers 3 teachers More than 3 teachers

Don’t 

know
46

[q22a] Moved from effective to 

highly effective on the NM TEACH

Summative Evaluation.

     

[q22b]Were put on an 

professional growth plan or 

entered probationary status 

     

[q22c] Were dismissed (that is, 

had their district employment as a 

teacher terminated) due to 

receiving a low effectiveness rating 

on the NM TEACH Summative 

Evaluation 

     

[q22d]Left teaching voluntarily

after receiving a low effectiveness 

rating on the NM TEACH

Summative Evaluation 

     

---------------------------------------[NEW SURVEY PAGE]---------------------------------------------

Optional	Comments	from	You

The following four questions are optional if you have any feedback you wish to share.

[q23] Do you have any comments about the NM TEACH Observation Rubric? 

[q24] Do you have any comments about professional development available to teachers in 

your district?

[q25] Do you have any comments about training either your district or NM PED have offered 

about the NM TEACH teacher evaluation system?

[q26] Do you have any feedback about this survey? 
47

---------------------------------------[NEW SURVEY PAGE]---------------------------------------------

Thank	You!

Thank you so much for your time to complete this survey. 

Within the next 2 weeks, you will receive an email message with instructions about how to 

claim your gift card. This email message will come from {iCard (services@icardgiftcard.com)} 

and will be sent to the same email address at which you received the survey invitation, unless 

you indicate otherwise here:

 Yes, please send my gift card information to the same email address to which you sent 

the survey invitation.

 Please send my gift card information to a different email address, which I am providing 

here: _______________

 No, I do not wish to receive a gift card email from {iCard}.

---------------------------------------[NEW SURVEY PAGE]---------------------------------------------

Thanks	Again

You have completed the survey and may now close it.
48

Spring	2016	School	Leader	Survey

[[Version: On-line administration. Wave 2.]] 

Understanding	Your	Participation	in	the	Study

Welcome to the spring 2016 survey for school leaders about the feedback you give to teachers’ 

regarding their instruction as a part of the NM TEACH Educator Effectiveness System. This is a 

follow-on to our fall 2015 survey to learn about the feedback you have provided to teachers 

this school year.

Study Purpose. With funding from the federal Department of Education as a part of the 

Southwest Regional Educational Lab, this study is intended to learn about the feedback 

principals give to teachers under the NM TEACH Educator Effectiveness System, and teachers’ 

perceptions of that feedback. For more information about the study, please see {REL Southwest 

website}. In {month, year} REL Southwest researchers asked all New Mexico’s public school 

principals to participate in a study about evaluation feedback to teachers. Results will be used 

to provide New Mexico Public Education Department feedback regarding how to improve the 

NM TEACH Educator Effectiveness System training for principals and teachers.

Survey Purpose. This survey is intended to gather information on the provision of feedback 

after formal observations by school leaders to teachers about their performance. Survey results 

will not be used to evaluate you, other teachers, or school leaders. 

Procedures. Online completion of the survey should take approximately 30 minutes. By using 

your personal survey link provided in the invitation email, you will be able to complete the 

survey in multiple sessions; responses will be saved as you enter them. However, once you click 

“submit” at the very end of the survey, you will no longer be able to return.

Confidentiality. REL Southwest will keep all survey responses confidential and will only use 

them for the purpose of the study. Our reports will contain summaries, and responses will 

never be presented in any way that would permit readers to identify you or your school. No 

one at your school, district, or the state will have access to survey responses that include 

respondents’ names, school names, or other information that could potentially be used to 

identify individuals or schools.

Participants. REL Southwest originally invited all New Mexico public school school-based 

principals to participate in this study. Only those principals who completed surveys in spring 

2015 are being sent this survey now in spring 2016. 

Benefits. Your completion of this survey gives you the opportunity to reflect on and share your 

perspective about the teacher evaluation system. You will be providing valuable information 

that may help New Mexico Public Education Department improve the NM TEACH Educator 

Effectiveness System.
49

Compensation. Within two weeks of completion of the online survey, you will receive via email 

a link to your $25 online gift card for completing the survey.

Risks. The risks associated with your participation in this study are minimal. 

Voluntary Participation. Your participation in this survey is completely voluntary. You can 

decide not to participate or to discontinue your participation at any time. A few survey 

questions, marked with a red asterisk, are required for routing to later questions.

Questions? If you have any questions about this survey or the study as a whole, please feel free 

to call us toll-free at {XXX}. You can also contact {XX}, the survey coordinator, at {email} or by 

phone at {XX}. If you have concerns or questions about your rights as a participant, contact the 

REL Southwest at SEDL (which is responsible for the protection of project participants): {email}, 

toll free at {XX}, or c/o IRB, {address}.

[Required question] Informed Consent. By clicking the box below, you are indicating that you 

have read and understood the information provided to you about your participation in this 

survey.

 I have read and understood the information and choose to participate. 

Thank you very much for your help in this important study!

---------------------------------------[NEW SURVEY PAGE]---------------------------------------------

Using	This	Survey

Here are some tips that may help you navigate and complete this survey.

Returning to previous pages. Be sure to use the survey “go back to previous page” button at 

the bottom of the page. Do not use your browser’s back button—it will not work.

Returning to the survey itself. If you would like to complete the survey in multiple sessions—or 

if you accidentally get disconnected from the survey—you can return to your previous 

responses by clicking on the link that was in your invitation email. This will take you to where 

you left off (through the last full page you completed). You can keep returning until you click on 

the “submit” button on the final page of the survey.

Saving your responses. Your responses on any given survey page are saved when you click on 

either button at the bottom of the page (“save and go to next page” or “go back to previous 

page”). 

Adding optional comments. Toward the bottom of each survey page is a space where you can 

write in any comments you may have pertaining to the questions on the page. This is 
50

completely optional.

----------------------------------------[NEW SURVEY PAGE]-----------------------------------------------

Your	Position	and	Job	Responsibilities

We would like to update our information about your position.

[q1] Which of the following best describes your position as a school leader this school year 

(2015-2016)? 

 If you are not a school leader, you may have received this survey in error. Please reply to 

the invitation email or call {XXX}.

 principal 

 vice principal/assistant principal

 director

 dean

 other: _______________________

[q2x]Including this school year (2015-2016), how many years have you been a school 

administrator? (Fill in each box.)

 “School administrator” includes principal, assistant principal, and other equivalent titles.

[q2xa]Years as a school administrator in your current school

[q2xb]Years as a school administrator in your current district 

(including current school)

[q2xc]Years as a school administrator, total, in any location 

(including other districts and current district)

[q3]In addition to your administrative responsibilities, do you currently (2015-2016) have 

official teaching responsibilities (that is, formally instruct K-12 students as the teacher of record 

in one or more classes)?

 Yes

 No

[q4]Do you play a role in filling teaching positions at your school (for example, recruiting, 

interviewing, and/or hiring)? 

 Yes

 No
51

----------------------------------------[NEW SURVEY PAGE]-----------------------------------------------

Your	Perceptions	of	the	Quality	of	Your	School’s	Teaching	Staff	This	Year

[q5] Use the slider to indicate your level of agreement with the following statement: “Overall, 

I was satisfied with the quality of the whole teaching staff in my school this year (2015-2016).”

[insert slider graphic here. Tick marks at disagree strongly, disagree somewhat, agree 

somewhat, agree strongly.]

[q6x] Use the slider to answer the following items. During this school year (2015-2016), how 

many teachers in your school... 

None or 

almost 

none A few About half Most

All or nearly 

all

[q6a] Are highly effective 

instructional leaders.

[insert slider graphic here]

[q6b] Are struggling and/or 

ineffective instructional leaders. 

[insert slider graphic here]

[q6c] Had a good grasp of the 

subject matter they teach

[insert slider graphic here]

[q6d] Engaged in collaborative

conversations about teaching and 

learning on a weekly or biweekly 

basis

[insert slider graphic here]

-----------------------------------------------------[NEW SURVEY PAGE]----------------------------------

Your	Practices	When	Observing	Teachers	This	Year	(SY	2015-2016)	for	Evaluation	

Purposes	

[q7] Thinking about the entire school year 2014-2015, please estimate the number of teachers 

you will formally evaluate yourself using the NM TEACH Observation Rubric {link}. 

[enter number here that ranges from 0 – 50] 

[q8] Thinking about the entire school year 2014-2015, how many other school leaders besides 

yourself will formally evaluate teachers using the NM TEACH Observation Rubric {link}? 

[enter number here that ranges from 0 – 5] 

[q9] Thinking about the entire school year 2015-2016, how often did you personally formally 

observe teachers in your school building using the NM TEACH Observation Rubric {link}? 
52

 I or another school leader formally observed (for 15+ minutes using the NM TEACH 

Observation Rubric) all teachers 3 or more times.

 I or another school leader formally observed (for 15+ minutes using the NM TEACH 

Observation Rubric) all teacher 2 times. 

 I or another school leader formally observed (for 15+ minutes using the NM TEACH 

Observation Rubric) some teachers 2 times and other teachers 3 times or more.

 Other: ________________________________

[q10] Estimate how long post-observation conferences typically took you during school year 

2014-2015. Think only of conferences after formal observations; do not include informal 

feedback from classroom walk-throughs. 

[insert slider graphic here. Tick marks for 0 minutes to 60 minutes with 10 minute intervals

[q11] Thinking about the entire school year 2014-2015, how frequently did you provide 

informal feedback about teaching and learning to individual teachers—e.g. walk-throughs, 

wows and wonders. 

 At least once a week. 

 At least once a month.

 At least once per semester.

 Other: ___________________

 Optional comment here:

[q12x] Thinking about all your formal observations from school year 2015-2016 using the NM 

TEACH Observation Rubric, use the slider to indicate how much you agree or disagree with the 

following statements about your post- formal observation conferences with teachers. 

Disagree 

Strongly

Disagree 

Somewhat

Agree 

Somewhat

Agree 

Strongly

[q12a] The feedback session itself, separate 

from professional development I may have 

recommended, helped teachers improve 

instruction.

[insert slider graphic here]

[q12b] In each conference, the teacher 

brought documents to the conference,

documents such as the lesson plan, PDP.

[insert slider graphic here]

[q12c] In each conference, I brought 

documents to conference such as the 

teacher’s report card and the most recent 

PDP.

[insert slider graphic here]

[q12d] In each conference, I identified at [insert slider graphic here]
53

least one positive practice that the teacher 

does well.

[q12e] In each conference, I identified at 

least one challenge facing the teacher.

[insert slider graphic here]

[q12f] In each conference, I used my scores

on the NM TEACH Observation Rubric for that 

teacher to determine which instructional 

practices to praise or criticize with the 

teacher.

[insert slider graphic here]

[q12g] In each conference, I used my scores

on the NM TEACH Observation Rubric for that 

teacher to determine what professional 

development to recommend to the teacher. 

[insert slider graphic here]

[q12h] In each conference, I ended the 

conference on a positive note.

[insert slider graphic here]

[q12i]Each conference followed a predictable 

format.

[insert slider graphic here]

[q12j] In each conference, the teacher and I 

mutually developed next steps for their 

instruction.

[insert slider graphic here]

[q12k] In each conference I provided specific

feedback to teachers about their

performance.

[insert slider graphic here]

[q12l] In each conference I provided 

actionable feedback to teachers about their 

performance.

[insert slider graphic here]

[q12m] In each conference, teachers 

committed to specific set of next steps to 

improve their instruction. 

[insert slider graphic here]

[q12n] In each conference, I spoke for

almost all of the time during the conference. 

[insert slider graphic here]

[q12o] There was a high level of 

collaboration in feedback conferences.

[insert slider graphic here]

[q12p] There was a high level of conflict in 

feedback conferences. 

[insert slider graphic here]

[q12q] It was hard to provide negative 

feedback to teachers about their 

performance.

[insert slider graphic here]

[q12r] The large majority of teachers seemed 

to trust and accept my feedback.

[insert slider graphic here]

[q12s] I feel positive about the feedback I [insert slider graphic here]
54

gave teachers in conferences.

[q12t] I enjoyed most of the post-observation 

feedback conferences.

[insert slider graphic here]

[q12u] I provided all teachers I reviewed with 

a written or online summary of the 

observation with my comments on it. 

[insert slider graphic here]

[q13a] Have you ever received the “Checklist for New Mexico Principals’ Provision of 

Feedback to Teachers” , a guide to conducting formal feedback sessions? The first page of the 

Conversation Checklist looks like this: 

 Yes

 No

[q13b] Have you ever used the “Checklist for New Mexico Principals’ Provision of Feedback to 

Teachers” when conducting the formal feedback conference in the 2015-2016 school year? 

 Yes

 No

[q13c] [If q11b = yes)] With how many of your teachers, if any, have you used the 

“Conversation Checklist”?
55

 None

 A few 

 About half

 Most

 All or nearly all 

[q13d] [If treatment=1| (if treatment = 0 & q13a = yes)] Did you view the testimonial 

video that was distributed along with the Conversation Checklist?

 Yes

 No

 I don’t know

[q13e] Have you ever received the “Guidance for New Mexico Principals’ About Provision of 

Feedback to Teachers” document? It looks like this: 

 Yes

 No

[q13f] If q13e = yes)] With how many of your teachers, if any, have you used the advice 

included in the “Recommended Practices” document?

 None

 A few 

 About half

 Most

 All or nearly all
56



[q13g][If q13c = none] Please select all of the reasons you have not used the Conversation 

Checklist.

 [q13ga]I have never seen the Conversation Checklist. 

 [q13gb]I haven’t yet used the Conversation Checklist, but I plan to. 

 [q13gc]I got the Conversation Checklist too late for me to use in teacher conferences.

 [q13gd]The Conversation Checklist looks burdensome—e.g., too long, too hard. 

 [q13ge]I don’t need it; I already have a way of holding feedback conversations with 

teachers that works well.

 [q13gf]I’ve read the Conversation Checklist and am using some concepts, but I’m not 

following the checklist.

 [q13gg]Other: ______________________________________________________

[q13h][If q13c ~= all & q13c ~= none] Which type of teachers did you use the 

“Conversation Checklist” with? Select all that apply.

 [q13ha] Only highly effective teachers

 [q13h b] Only satisfactory teachers

 [q13h c] Only struggling teachers

 [q13h d] Only new teachers

 [q13h e] Only experienced teachers

 [q13h h] Other: _________________________________________________________

[q14a] [if 13C ~= none] Use the slider to estimate how long post-observation conferences 

typically took you during school year 2015-2016 when you used the “Conversation Checklist.” 

Think only of conferences after formal observations; do not include informal feedback from 

classroom walk-throughs. 

[insert slider graphic here. Tick marks for 0 minutes, 15 min, 30 min., 45 min., 60 min, 75 min, 

90 min]

[q14b] [if q13f~= none] Use the slider to estimate how long post-observation conferences 

typically took you during school year 2015-2016 after you reviewed the “Guidance for New 

Mexico Principals’ About Provision of Feedback to Teachers”. Think only of conferences after 

formal observations; do not include informal feedback from classroom walk-throughs. 

[insert slider graphic here. Tick marks for 0 minutes, 15 min, 30 min., 45 min., 60 min, 75 min, 
57

90 min]

[q15x] If 13C ~= none ] Use the slider to indicate how much you agree or disagree with the 

following statements about your opinions about the Conversation Checklist this year (school 

year 2015-2016). 

Disagree 

Strongly

Disagree 

Somewhat

Agree 

Somewhat

Agree 

Strongly

[q15a] The Conversation Checklist is easy to 

use.

[insert slider graphic here]

[q15b] The Conversation Checklist takes too 

much time to use.

[insert slider graphic here]

[q15c] The Conversation Checklist makes the 

feedback conversation feel formulaic. 

[insert slider graphic here]

[q15d] The Conversation Checklist provided 

me and teachers a helpful structure for 

feedback. 

[insert slider graphic here]

[q15e] I provided more critical feedback to 

teachers than I had in prior conferences where 

I didn’t use the Conversation Checklist.

[insert slider graphic here]

[q15f] The Conversation Checklist helps 

teachers commit to specific set of next steps

to improve their instruction.

[insert slider graphic here]

[q15g] I use the Conversation Checklist as the 

basis to create teachers’ professional 

development plans (PDPs).

[insert slider graphic here]

[q15h] I use the Conversation Checklist as the 

basis to create professional growth plans

(PGPs) for struggling teachers. 

[insert slider graphic here]

-----------------------------------------------------[NEW SURVEY PAGE]----------------------------------

Professional	Development	Offered	to Teachers	This	Year	(2015-2016)

The questions in this section ask about professional development for teachers.

 Professional development includes (but is not limited to) workshops, inservices, 

coaching/mentoring, collaborative planning/study, institutes, professional conferences, 

online courses, self-directed learning, and so forth.

[q16x] Use the slider to indicate how much you agree or disagree with each of the following 

statements about the professional development experiences of teachers at your school this 

year (2015-2016, including summer 2015).
58

 Consider professional development offered by the district and other external 

sources as well as professional development offered by the school.

The professional development options for teachers this year....

Don’t know Disagree 

Strongly

Disagree 

Somewhat

Agree 

Somewhat

Agree 

Strongly

[q16a] Were mostly developed by

school leaders or teachers at my

school.

 [insert slider graphic here]

[q16b] Were mostly job-embedded 

rather than one-time only sessions.  [insert slider graphic here]

[q16c]Were varied and numerous 

enough to allow me to customize my

PD recommendations to teachers 

based on their performance on specific 

domains of the NM TEACH 

Observation Rubric.{link}

 [insert slider graphic here]

[q16d]Helped teachers improve on 

elements measured by the NM TEACH 

Observation Rubric. 

 [insert slider graphic here]

[q16e]Helped teachers improve 

other elements of the NM TEACH 

evaluation besides the NM TEACH 

Observation Rubric (e.g., student 

achievement or “other” multiple 

measure).

 [insert slider graphic here]

[q16f]Were well aligned with the 

Common Core State Standards and/or 

curriculum based on these standards.

 [insert slider graphic here]

[q16g]Have been convenient enough 

to promote teachers’ participation.  [insert slider graphic here]

[q16h]Were sufficiently resourced 

(for example, substitute coverage, 

funding to cover expenses, stipends) 

to allow teachers to participate in the 

professional development they need 

to do their jobs successfully.

 [insert slider graphic here]

[q16i]District-provided professional 

development offerings (workshops, 

inservices, online courses, etc.) have 

enough slots and availability to 

accommodate all teachers who wish 

 [insert slider graphic here]
59

or need to participate. 

[q16j]Teachers have had easy access 

to a catalog of professional 

development opportunities aligned 

with the NM TEACH Observation 

Rubric.

 [insert slider graphic here]

[q17x] Thinking about the professional development that teachers at your school have 

participated in this year (2015-2016, including summer 2015), use the slider to indicate how 

useful you think each of the following has been for improving teacher effectiveness.

Not 

Applicable 

Not at all 

useful

Slightly 

useful

Moderately 

useful Very useful

[q17a]Workshops or inservices for 

teachers at your school only 

(typically on-site)

 [insert slider graphic here]

[q17b]Workshops, inservices, 

institutes, or conferences organized 

by your district for teachers from 

multiple schools

 [insert slider graphic here]

[q17c]Online professional 

development offered by or through 

your district

 [insert slider graphic here]

[q17d]District- or state-sponsored 

induction program, mentoring, or 

other professional development 

activities for new or beginning 

teachers

 [insert slider graphic here]

[q17e]Instructional coaching 

(provided by school-based coaches 

or district coaches)

 [insert slider graphic here]

[q17f]Professional development 

led by me or a school leader such as 

study groups or school-wide staff 

meetings

 [insert slider graphic here]

[q17g]School-based teacher 

collaboration (grade-level or 

subject-area teams, professional 

learning communities, study groups, 

etc.)

 [insert slider graphic here]

[q17h]Videos of sample lessons  [insert slider graphic here]
60

-----------------------------------------------------[NEW SURVEY PAGE]----------------------------------

Training	You	Received	About	NM	TEACH	to	Date

[q18x]What types of professional development have you received to date regarding the 

teacher evaluation system, NM TEACH? (Check all that apply.)

 [q18a][exclusive option]No professional development received

 [q18b]Information on the NM TEACH Educator Effectiveness system(overall purpose, 

specific components, consequences for teachers, relationship to professional 

development, and so forth)

 [q18c]How to use the NM TEACH Observation Rubric

 [q18d]How to provide feedback to individual teachers after formal classroom 

observations 

 [q18e]How to enter scores from NM TEACH Observation Rubric into Reflect, the 

Teachscape software

 [q18f]How to use Reflect, the Teachscape software 

 [q18g]How to interpret a teachers’ NM TEACH Summative Evaluation Score

 [q18h]How to give NM TEACH Observation Rubric ratings consistent with those given 

by other raters

 [q18i]How to understand and make use of NM TEACH Summative Evaluation Score to 

help improve teaching at my school

 [q18j]How to direct teachers to professional development based on their NM TEACH 

Summative Evaluation Score

 [q18k]Other:

[q19] [if 18b-k ==yes] Who sponsored the training you attended about NM TEACH Educator 

Effectiveness system? Mark as many as apply.

 State Public Education Department 

 My school district

 Other: __________________

 Don’t know 

[q20] [if 18b-k ==yes] In total, how many minutes of training did you receive this year (2014-

2015, including summer 2014) about how to provide feedback to teachers.

 0 minutes

 15 minutes

 30 minutes

 60 minutes

 120 minutes

 240 minutes
61

 Other: 

[q21] [if 18d==yes] Thinking about the training on how to conduct post-observation 

conferences with teachers, use the slider to indicate how useful you think the training was. 

Not at all useful Slightly useful Moderately useful Very useful

[insert slider graphic here]

[q22x] Use the slider to indicate how much you agree or disagree with each of the following 

statements about your understanding of NM TEACH.

Disagree 

strongly

Disagree 

somewhat

Agree 

somewhat

Agree 

strongly

[q22a]I understand how the overall teacher 

ratings work in the NM teacher evaluation 

system (NM TEACH). 

[insert slider graphic here]

-----------------------------------------------------[NEW SURVEY PAGE]----------------------------------

Your	Personal	Views	of	NM	TEACH	

[q23x]Use the slider to indicate how much you agree or disagree with each of the following 

statements about the teacher effectiveness ratings that became available in school year 2015-

2016. 

Disagree 

Strongly

Disagree 

Somewhat

Agree 

Somewhat

Agree 

Strongly

[q23a]The NM TEACH Educator Effectiveness 

System does a good job distinguishing 

effective from ineffective teachers.

[insert slider graphic here]

[q23b]The NM TEACH Educator Effectiveness 

System is fair to all teachers, regardless of 

their personal characteristics or those of the 

students they teach.

[insert slider graphic here]

[q23c]The NM TEACH Educator Effectiveness 

System ignores important aspects of teacher 

performance.

[insert slider graphic here]

[q23d]The NM TEACH Educator Effectiveness 

System requires so much of my time that it 

[insert slider graphic here]
62

interferes with other important activities.

[q23e]I have a clear idea of what the NM 

TEACH Educator Effectiveness System views as 

“good instruction.”

[insert slider graphic here]

[q23g]The NM TEACH Educator Effectiveness 

System has helped me pinpoint specific things 

I can do to help improve my teachers’ 

performance.

[insert slider graphic here]

[q23h]In the current NM TEACH Educator 

Effectiveness System, teachers who do poorly 

on their evaluation often leave teaching 

voluntarily.

[insert slider graphic here]

[q24x]Use the slider to rate how much you agree or disagree with each of the following 

statements about the NM TEACH Observation Rubric. 

Disagree 

Strongly

Disagree 

Somewhat

Agree 

Somewhat

Agree 

Strongly

[q24a]The 6 elements of Domain 1 (Planning 

and Preparation) of the NM TEACH 

Observation Rubric {link} are a useful way to 

measure the quality of teachers’ instructional 

practices. 

[insert slider graphic here]

[q24b] The 5 elements of Domain 2 (Creating 

an Environment for Learning) of the NM 

TEACH Observation Rubric {link} are a useful 

way to measure the quality of teachers’ 

instructional practices.

[insert slider graphic here]

[q24c]The 5 elements of Domain 3 (Teaching 

for Learning) of the NM TEACH Observation 

Rubric {link} are a useful way to measure the 

quality of teachers’ instructional practices.

[insert slider graphic here]

[q24d] The 4 elements of Domain 4 

(Professionalism) of the NM TEACH 

Observation Rubric {link}are a useful way to 

measure the quality of teachers’ instructional 

practices.

[insert slider graphic here]

[q24e] Using the NM TEACH Observation 

Rubric {link} helps me to provide constructive, 

specific feedback to teachers to improve their 

instruction. 

[insert slider graphic here]
63

-----------------------------------------------------[NEW SURVEY PAGE]----------------------------------

Teacher	Retention	and	Improvement

[q25x]Over the past year (since approximately May 1, 2015), how many teachers at your 

school...

None 

(0 

teachers)

1 

teacher

2 

teachers

3 

teachers

More 

than 3 

teachers

Don’t 

know

[q25a] Moved from effective to 

highly effective on the NM 

TEACH Summative Evaluation.

     

[q25b]Were put on an 

professional growth plan or 

entered probationary status 

     

[q25c] Were dismissed (that is, 

had their district employment as a 

teacher terminated) due to 

receiving a low effectiveness rating 

on the NM TEACH Summative 

Evaluation

     

[q25d]Left teaching voluntarily 

after receiving a low effectiveness 

rating on the NM TEACH

Summative Evaluation

     

---------------------------------------[NEW SURVEY PAGE]---------------------------------------------

Optional	Comments	from	You

The following four questions are optional if you have any feedback you wish to share.

[q26] [if treatment = 1] Do you have any suggestions to help us improve the PrincipalTeacher Post-Observation Conversation Checklist?

[q27] Do you have any comments about professional development available to teachers in 

your district?

[q28] Do you have any comments about the NM TEACH Observation Rubric? 

[q29] Do you have any comments about training either your district or NM PED have offered 

about the NM TEACH teacher evaluation system?
64

---------------------------------------[NEW SURVEY PAGE]---------------------------------------------

Thank	You!

Thank you so much for your time to complete this survey. 

Within the next 2 weeks, you will receive an email message with instructions about how to 

claim your gift card. This email message will come from {iCard (services@icardgiftcard.com)} 

and will be sent to the same email address at which you received the survey invitation, unless 

you indicate otherwise here:

 Yes, please send my gift card information to the same email address to which you sent the 

survey invitation.

 Please send my gift card information to a different email address, which I am providing 

here: _______________

 No, I do not wish to receive a gift card email from {iCard}.

---------------------------------------[NEW SURVEY PAGE]---------------------------------------------

Thanks	Again

You have completed the survey and may now close it.
65

Spring	2015	Teacher	Survey

[[Version: Teachers. On-line administration. Wave 1.]]

Understanding	Your	Participation	in	the	Study

Welcome to the spring 2015 survey for school leaders about the feedback you give to teachers’ 

regarding their instruction as a part of the NM TEACH Educator Effectiveness System. 

Study Purpose. With funding from the federal Department of Education as a part of the 

Southwest Regional Educational Lab, this study is intended to learn about the feedback 

principals give to teachers under the NM TEACH Educator Effectiveness System, and teachers’ 

perceptions of that feedback. For more information about the study, please see {REL Southwest 

website}. In {month, year} REL Southwest researchers asked all New Mexico’s public school 

principals to participate in a study about evaluation feedback to teachers. Results will be used 

to provide New Mexico Public Education Department feedback regarding how to improve the 

NM TEACH Educator Effectiveness System training for principals and teachers.

Survey Purpose. This survey is intended to gather information on the provision of feedback to 

teachers after formal observations by school leaders. REL Southwest researchers will link your 

survey responses to basic employment information, such as race, ethnicity and years of 

experience that REL Southwest researchers will obtain from NM PED. Survey results will not be 

used to evaluate you, other teachers, or school leaders. REL Southwest researchers will ask 

study participants to fill out an on-line, 30-minute survey once in spring 2015 and again in 

spring 2016.

Procedures. Online completion of the survey should take approximately 30 minutes. By using 

your personal survey link provided in the invitation email, you will be able to complete the 

survey in multiple sessions; responses will be saved as you enter them. However, once you click 

“submit” at the very end of the survey, you will no longer be able to return.

Confidentiality. REL Southwest will keep all survey responses confidential and will only use 

them for the purpose of the study. Our reports will contain summaries, and responses will 

never be presented in any way that would permit readers to identify you or your school. No 

one at your school, district, or the state will have access to survey responses that include 

respondents’ names, school names, or other information that could potentially be used to 

identify individuals or schools.

Participants In each of the participating New Mexico public schools, REL Southwest has invited 

the principal to participate in this study Up to 10 teachers in your school were selected at 

random to be surveyed. 

Benefits. Your completion of this survey gives you the opportunity to reflect on and share your 

perspective about the teacher evaluation system. You will be providing valuable information 

that may help New Mexico Public Education Department improve the NM TEACH Educator 
66

Effectiveness System. 

Compensation. Within two weeks of completion of this online survey, you will receive via email 

a link to your $25 online gift card for completing the survey. We will provide another $25 

online gift card upon completion of the spring 2016 survey to each participant.

Risks. The risks associated with your participation in this study are minimal. 

Voluntary Participation. Your participation in this survey is completely voluntary. You can 

decide not to participate or to discontinue your participation at any time. A few survey 

questions, marked with a red asterisk, are required for routing to later questions.

Questions? If you have any questions about this survey or the study as a whole, please feel free 

to call us toll-free at {XXX}. You can also contact {XX}, the survey coordinator, at {email} or by 

phone at {XX}. If you have concerns or questions about your rights as a participant, contact the 

REL Southwest at SEDL (which is responsible for the protection of project participants): {email}, 

toll free at {XX}, or c/o IRB, {address}.

[Required question] Informed Consent. By clicking the box below, you are indicating that you 

have read and understood the information provided to you about your participation in this 

survey.

 I have read and understood the information and choose to participate.. 

Thank you very much for your help in this important study!

---------------------------------------[NEW SURVEY PAGE]---------------------------------------------

Using	This	Survey

Here are some tips that may help you navigate and complete this survey.

Returning to previous pages. Be sure to use the survey “go back to previous page” button at 

the bottom of the page. Do not use your browser’s back button—it will not work.

Returning to the survey itself. If you would like to complete the survey in multiple sessions—or 

if you accidentally get disconnected from the survey—you can return to your previous 

responses by clicking on the link that was in your invitation email. This will take you to where 

you left off (through the last full page you completed). You can keep returning until you click on 

the “submit” button on the final page of the survey.

Saving your responses. Your responses on any given survey page are saved when you click on 

either button at the bottom of the page (“save and go to next page” or “go back to previous 

page”). 
67

Adding optional comments. Toward the bottom of each survey page is a space where you can 

write in any comments you may have pertaining to the questions on the page. This is 

completely optional.

----------------------------------------[INTERNAL CODES]-----------------------------------------------

[[Treatment code (treatment): REL Southwest will pre-identify via an internal code that will not 

be listed on the survey whether teacher respondent works at a school that belongs to the 

treatment group or the control group. treatment = 1 indicates teacher is in the treatment 

group. treatment = 0 = control group. Unless otherwise noted, survey questions will appear for 

both groups of teachers.]]

----------------------------------------[NEW SURVEY PAGE]-----------------------------------------------

Your	Position	and	Job	Responsibilities

[required] [q1] Which of the following best describes your current position as a teacher of K12 students? (Check one.)

 If you are not a teacher of K-12 students, or do not have teaching responsibilities in 

2014-2015, you have received this survey in error. Please let us know by replying to 

the invitation email or by calling {number}.

 Regular education teacher

 Special education teacher

 Other kind of teacher: ___________________________________

[q2x] What subject area(s) are you currently teaching ? (Check all that apply.)

 [q2a] English/language arts/reading/writing

 [q2b] Mathematics

 [q2d] Science

 [q2d] Social Studies

 [q2e] Foreign language(s)

 [q2f] Visual or performing arts (art, music, etc.)

 [q2g] Physical education/health education

 [q2h] Career/technical education

 [q2i] Other subject area(s): [include fill-in box]

[q3] Which one of the following three groups of teachers do you belong to? These groups are 

defined by New Mexico’s teacher evaluation system, NM TEACH. 
68

 Group A: tested subject, tested grade

 Group B: grade 3 – 12 subject not linkable to NM’s standard based assessments

 Group C: grade K-2 teacher

[q4]Which of the following best describes your current teaching arrangement? (Check one.)

 I teach a single group of students all or most of the day in multiple subject areas. 

(Traditional elementary arrangement; sometimes called “self-contained.”)

 I teach several classes of different students during the day in a particular subject (for 

example, math) or perhaps two subjects (for example, you teach some math classes and 

some science classes). (Traditional secondary arrangement; sometimes called “subjectspecific” or “departmentalized”; at elementary level, sometimes called “subject matter 

specialist.” Also typical arrangement for physical education, art, music, etc.)

 I mainly teach selected students released from (or in) their regular classes in specific 

skills or to address specific needs (for example, special education, reading, English as a 

second language, gifted and talented.) (Sometimes called “pull out”, “resource,” or 

“push in” instruction.)

 I am one of two or more teachers who are jointly responsible for teaching the same 

subject(s) to a group of students (for example, in the same classroom), all or most of the 

time and/or in a majority of classes. (Sometimes called “co-teaching” or “job share.”)

 Other (please describe): _______________________________________

[q5] Including this school year (2014-2015), how many years have you been teaching, total,

regardless of location?

Total years teaching: [fill in number: must be whole number less than 60]

[q6] Including this school year (2014-2015), how many years have you been teaching in your 

current school?

Years in current school: [fill in number: must be whole number less than 60]

[q7] Which of the following best describes your current position as a teacher of K-12 students? 

(Check all that apply)

 Spanish bilingual teacher

 Native American bilingual teacher

 Other bilingual teacher: ___________________________________

 Not a bilingual teacher
69

----------------------------------------[NEW SURVEY PAGE]-----------------------------------------------

Your	Experiences	This	Year	(SY	2014-2015)	of	Formal	Observations	for	Evaluation	

Purposes	

[* required question] [q8] Thinking back over this whole school year 2014-2015, how many 

times were you formally observed by a school leader? School leader is defined as a principal, 

assistant principal, dean, or coach charged with authority to conduct formal observations of 

20+ minutes using the NM TEACH Observation Rubric {link to rubric}. 

 3 or more times

 2 times

 1 time

 0 times

 Other: ________________________________

[q9] [If q8 = 1, 2, or 3+ times] Use the slider to estimate how long the post-observation 

conferences typically took. Think only of conferences after formal observations; do not include 

informal feedback you might have received from classroom walk-throughs. 

[insert slider graphic here. Tick marks for 0 min, 15 min, 30 min, 45 min, 60 min, 75 min, 90 

min.]

[q10] How frequently did you receive informal feedback this year from a school leader—for 

example, “wow or wonder” statements after brief walk-throughs. 

 At least once a week 

 At least once a month

 At least once per semester or trimester

 Never

 Other: ___________________

[q11x] [If q8 = 1, 2, or 3+ times] Use the slider to indicate the extent to which you agree or 

disagree with the following statements about all of the conferences you had with a school 

leader following his or her formal observation(s) of your class this year (school year 2014-

2015). 

Disagree 

Strongly

Disagree 

Somewhat

Agree 

Somewhat

Agree 

Strongly

[q11a] The feedback session itself, separate

from professional development I may have 
70

been recommended to take, helped me 

improve my instruction.

[q11b] In each conference, I brought 

documents to the conference, documents 

such as the lesson plan, and PDP.

[insert slider here]

[q11c] In each conference, the school leader

brought documents to conference, such as my 

summative score and the most recent PDP.

[insert slider here]

[q11d]In each conference the school leader 

identified at least one positive practice that I 

do well.

[insert slider here]

[q11e] In each conference the school leader 

identified at least one challenge facing me. 

[insert slider here]

[q11f]In each conference, the school leader 

used my scores on the NM TEACH 

Observation Rubric to determine which 

instructional practices to praise or critique.

[insert slider here]

[q11g] In each conference, the school leader 

used my scores on the NM TEACH 

Observation Rubric to determine what 

professional development to recommend to 

the teacher.

[insert slider here]

[q11h] In each conference, the school leader

ended the conference on a positive note.

[insert slider here]

[q11i]Each conference followed a predictable 

format.

[insert slider here]

[q11j]In each conference ,y school leader and 

I mutually developed next steps for my 

instruction.

[insert slider here]

[q11k]I walked away with a clear 

understanding of my school leader’s 

feedback.

[insert slider here]

[q11l]My school leader listened to what I had 

to say during the conferences.

[insert slider here]

[q11m]The observations were done to me, 

rather than for me. 

[insert slider here]

[q11n]I received specific feedback about my 

performance. 

[insert slider here]
71

[q11o] I received actionable feedback about 

my performance.

[insert slider here]

[q11p]During the conferences, I committed 

to a specific set of next steps to improve my 

instruction. 

[insert slider here]

[q11q] ] In each conference, the school 

leader spoke for almost all of the time. 

[insert slider here]

[q11r] There was a high level of conflict in 

the conference. 

[insert slider here]

[q11s] In each conference I obtained 

recommendations for professional 

development tailored to my specific needs.

[insert slider here]

[q11t] I trusted and accepted the feedback I 

received. 

[insert slider here]

[q11u]I feel positive about the feedback I got 

during the conferences. 

[insert slider here]

[q11v] I enjoyed most of the postobservation feedback conferences.

[q11w] I was always provided with a written 

or online summary of the observation and my 

school leader’s comments on it. 

[insert slider graphic here]

[q12x] Please indicate whether you were ever rated this school year by your school leader as 

needing to improve upon or else excelling in the following observation items from NM TEACH, 

whether you received a recommendation to take up professional development (PD), and how you 

rate yourself on the item. Mark all that apply.

NM TEACH 

Observation Item

I was rated at 

least once as 

excelling on 

this item

I was rated at 

least once as 

needing to 

improve on 

this item

I was 

recommended 

to take PD 

because of 

this item

I don’t 

know how 

I was rated

How would 

you rate 

yourself 

this school 

year on this 

item? 

2A: Creating an 

environment of respect 

and rapport

[Ineffective, 

Minimally

Effective, 

Effective, 

Highly 

Effective, 

Exemplary

I don’t 
72

know]

2B: Organizing 

Physical Space

2C: Establishing a 

culture for learning

2D: Managing 

classroom 

procedures

2E: Managing Student 

Behavior

3A: Communicating 

with Students

3B: Using questioning 

and discussion 

techniques

3C: Engaging students 

in learning

3D: Assessment in 

Instruction

3E: Demonstrating 

flexibility and 

responsiveness

-----------------------------------------------------[NEW SURVEY PAGE]----------------------------------

Professional	Development	You	Received	This	Year	(SY	2014-2015)

The questions on this page ask about all of the professional development you have participated 

in during the 2014-2015 school year, including summer 2014.

 Professional development includes (but is not limited to) workshops, inservices, 

coaching/mentoring, collaborative planning/study, courses, institutes, professional 

conferences, self-directed learning, and so forth.

 Consider mandatory/required professional development as well as voluntary 

professional development.

[q13x] Please indicate all of the topics below that one or more school leader recommend you 

take. In addition, please also include which topics were covered in professional development 

you actually participated in this year (2014-2015, including summer 2014), and topics that you 

would have liked to have participated in professional development, but it was not available to 

you. Select as many topics as apply.

A	school	

leader	

I	took	the	

PD	online

I	received	

the	PD	in	 h
73

recommen

ded	I	take	

this	PD

person t

[q13a] Classroom management   

[q13b] Professionalism and communication in the classroom   

[q13c] Special education and inclusion   

[q13d] Standards-based lesson plans   

[q13e] Common Core Standards   

[q13f] Assessment (includes continuous assessment, portfolio 

assessment, Common Core aligned assessments)   

[q13g] Data-driven instruction   

[q13h] Early childhood   

[q13i] Instructional strategies or content in the content area(s) you 

teach—e.g., Mathematics; or Literacy PD   

[q13j] Response to Intervention   

[q13k] Teachscape Reflect training   

[q13l] Other NM TEACH training   

[q13m] Sheltering Instruction for Language development   

[q13n] Language Acquisition Design   

[q13o] Scaffolding instruction   

[q13p] Other: _______________________________   

[q13q] Other: _______________________________   

[q13r] Other: _______________________________   

[q14x] Thinking about the professional development you have participated in this year (2014-

2015, including summer 2014), use the slider to indicate how much you agree or disagree with 

each of the following statements. 

My professional development experiences this year...

Disagree	

Strongly

Disagree	

Somewhat

Agree	

Somewhat

Agree	

Strongly

[q14a] Were mostly developed by school 

leaders or teachers at my school.

[insert slider here]

[q14b] Were mostly job-embedded rather 

than one-time only sessions.

[insert slider here]
74

[q14a] Were varied and numerous enough to 

allow the school leader to customize PD 

recommendations based on my performance on 

specific domains of the NM TEACH 

Observation Rubric.{link}

[insert slider here]

[q14b] Have been useful for helping me 

improve the achievement of my students

[insert slider here]

[q14c]Have been useful for improving my 

instruction

[insert slider here]

[q14d]Have taken more time than they were 

worth

[insert slider here]

[q14e]Have been well aligned with the 

Common Core State Standards and/or 

curriculum based on these standards

[insert slider here]

[q15x]Use the slider to indicate how much you agree or disagree with each of the following 

statements about the support you have received related to your professional development 

experiences this year (SY 2014-2015, including summer 2014). 

Disagree	

Strongly

Disagree	

Somewhat

Agree	

Somewhat

Agree	

Strongly

[q15a] I had easy access to a list of the 

possible professional development courses or 

offerings I could take. 

[insert slider graphic here]

[q15c]My school leader(s) has encouraged 

and supported my participation in professional 

development.

[insert slider here]

[q15d]Sufficient resources (for example, 

substitute coverage, funding to cover 

expenses, stipends) have been available to 

allow me to participate in the professional 

development I need to teach effectively.

[insert slider here]

[q15e]I have had sufficient flexibility in my 

schedule to pursue the professional 

development opportunities recommended for 

me.

[insert slider here]

-----------------------------------------------------[NEW SURVEY PAGE]----------------------------------

Your	Perceptions	of	the	Quality	of	Your	School’s	Leadership	Last	Year
75

[q16] Rate your level of agreement with the following statement: “Overall, I was satisfied with 

the quality of my school’s leaders (principals, assistant principals, deans, department chairs) 

this year (school year 2014-2015).”

 Disagree strongly

 Disagree somewhat

 Agree somewhat

 Agree strongly 

[q17x] Thinking about this year (2014-2015), use the slider to indicate how much you agree or 

disagree with the following statements. “The school leaders who conduct formal observations 

of my classroom ... 

Disagree	

Strongly

Disagree	

Somewhat

Agree	

Somewhat

Agree	

Strongly

[q17a] Understand the subject 

matter content that I teach.

[insert slider here]

[q17b] Understand what skills are 

needed for teachers to foster 

meaningful student learning. 

[insert slider here]

[q17c] Foster an atmosphere of 

trust and mutual respect in my 

school.

[insert slider here]

[q17d] Understand the 3 rating 

components of the NM TEACH 

Summative Evaluation Report

(student achievement, observations, 

multiple measures). 

[insert slider here]

[q17e] Help to improve my 

teaching. 

[insert slider here]

[q17f] Are instructional leaders. [insert slider here]

-----------------------------------------------------[NEW SURVEY PAGE]----------------------------------

Your	Personal	Views	of	NM	TEACH	Educator	Effectiveness	System

[q18x]Use the slider to rate how much you agree or disagree with each of the following 

statements about the NM TEACH Educator Effectiveness System from school year 2014-2015. 

Disagree 

Strongly

Disagree 

Somewhat

Agree 

Somewhat

Agree 

Strongly

[q18a]The NM TEACH Educator [insert slider here]
76

Effectiveness System does a good job 

distinguishing effective from ineffective 

teachers.

[q18b]The NM TEACH Educator 

Effectiveness System is fair to all teachers, 

regardless of their personal characteristics or 

those of the students they teach.

[insert slider here]

[q18c]The NM TEACH Educator Effectiveness 

System ignores important aspects of teacher 

performance.

[insert slider here]

[q18d]The NM TEACH Educator 

Effectiveness System has helped me pinpoint 

specific things I can do to help improve my 

performance.

[insert slider here]

[q18e] I understand how the overall teacher 

ratings work in the NM TEACH Educator 

Effectiveness System.

[insert slider graphic here]

-----------------------------------------------------[NEW SURVEY PAGE]----------------------------------

Your	NM	TEACH	Summative	Evaluation	Score	

[q19]Do you know what your level of Educator Effectiveness (Ineffective, Minimally Effective, 

Effective, Highly Effective, Exemplary) was for the school year 2014-2015? 

 Yes

 No

[q20] [if q19=yes] How did you learn your level of Educator Effectiveness for the school year 

2014-2015? Select all that apply.

 I received it via email from New Mexico Public Education Department

 I logged into my teacher portal online

 A school leader told me 

 I received it via my mailbox at school 

 Other: ______________________

 Other: ______________________

 I don’t recall

[q21] [if q19=yes] Did a school leader ever discuss with you how to improve your Educator 
77

Effectiveness score? 

 Yes, we had a conference

 Yes, in passing

 No

 I don’t recall 

[q22] [if q19=yes] Did a school leader ever discuss with you how to interpret your student 

achievement growth score? 

 Yes, we had a conference

 Yes, in passing

 No

 Not Applicable

 I don’t recall 

---------------------------------------[NEW SURVEY PAGE]---------------------------------------------

Optional	Comments	from	You

The following four questions are optional if you have any feedback you wish to share. If you do 

not, proceed to the next and final page.

[q23] Do you have any comments about the NM TEACH Observation Rubric? 

[q24] Do you have any comments about professional development available to teachers in 

your district?

[q25] Do you have any comments about the NM TEACH Educator Effectiveness System?

[q26] Do you have any feedback about this survey? 

---------------------------------------[NEW SURVEY PAGE]---------------------------------------------

Thank	You!

Thank you so much for your time to complete this survey. 
78

Within the next 2 weeks, you will receive an email message with instructions about how to 

claim your gift card. This email message will come from {iCard (services@icardgiftcard.com)} 

and will be sent to the same email address at which you received the survey invitation, unless 

you indicate otherwise here:

 Yes, please send my gift card information to the same email address to which you sent the 

survey invitation.

 Please send my gift card information to a different email address, which I am providing 

here: _______________

 No, I do not wish to receive a gift card email from {iCard}.

---------------------------------------[NEW SURVEY PAGE]---------------------------------------------

Thanks	Again

You have completed the survey and may now close it.
79

Spring	2016	Teacher	Survey

[[Version: Teachers. On-line administration. Wave 2.]]

Understanding	Your	Participation	in	the	Study

Welcome to the spring 2016 survey for teachers about feedback you receive about your 

instruction as a part of the NM TEACH Educator Effectiveness System. This is a follow-on to our 

spring 2015 survey to learn about the feedback you received from your school leaders last year.

Study Purpose. With funding from the federal Department of Education as a part of the 

Southwest Regional Educational Lab, this study is intended to learn about the feedback 

principals give to teachers under the NM TEACH Educator Effectiveness System, and teachers’ 

perceptions of that feedback For more information about the study, please see {REL Southwest 

website}. In {month, year} REL Southwest researchers asked all New Mexico’s public school 

principals to participate in a study about evaluation feedback to teachers. Results will be used 

to provide New Mexico Public Education Department feedback regarding how to improve the 

NM TEACH Educator Effectiveness System training for principals and teachers.

Survey Purpose. This survey is intended to gather information on the provision of feedback 

after formal observations by school leaders to teachers about their performance. REL 

Southwest researchers will link your survey responses to basic employment information, such 

as race, ethnicity and years of experience that REL Southwest researchers will obtain from NM 

PED. Survey results will not be used to evaluate you, other teachers, or school leaders. 

Procedures. Online completion of the survey should take approximately 30 minutes. By using 

your personal survey link provided in the invitation email, you will be able to complete the 

survey in multiple sessions; responses will be saved as you enter them. However, once you click 

“submit” at the very end of the survey, you will no longer be able to return.

Confidentiality. REL Southwest will keep all survey responses confidential and will only use 

them for the purpose of the study. Our reports will contain summaries, and responses will 

never be presented in any way that would permit readers to identify you or your school. No 

one at your school, district, or the state will have access to survey responses that include 

respondents’ names, school names, or other information that could potentially be used to 

identify individuals or schools.

Participants. In each of the participating New Mexico public schools, REL Southwest has 

invited the principal to participate in this study. Up to 10 teachers in your school were selected at 

random to be surveyed. Only those teachers who completed surveys in spring 2015 are being 

sent this survey now in spring 2016.

Benefits. Your completion of this survey gives you the opportunity to reflect on and share your 

perspective about the teacher evaluation system. You will be providing valuable information 

that may help New Mexico Public Education Department improve the NM TEACH Educator 
80

Effectiveness System. 

Compensation. Within two weeks of completion of the online survey, you will receive via email 

a link to your $25 online gift card for completing the survey.

Risks. The risks associated with your participation in this study are minimal. 

Voluntary Participation. Your participation in this survey is completely voluntary. You can 

decide not to participate or to discontinue your participation at any time. A few survey 

questions, marked with a red asterisk, are required for routing to later questions.

Questions? If you have any questions about this survey or the study as a whole, please feel free 

to call us toll-free at {XXX}. You can also contact {XX}, the survey coordinator, at {email} or by 

phone at {XX}. If you have concerns or questions about your rights as a participant, contact the 

REL Southwest at SEDL (which is responsible for the protection of project participants): {email}, 

toll free at {XX}, or c/o IRB, {address}.

[Required question] Informed Consent. By clicking the box below, you are indicating that you 

have read and understood the information provided to you about your participation in this 

survey.

 I have read and understood the information and choose to participate. 

Thank you very much for your help in this important study!

---------------------------------------[NEW SURVEY PAGE]---------------------------------------------

Using	This	Survey

Here are some tips that may help you navigate and complete this survey.

Returning to previous pages. Be sure to use the survey “go back to previous page” button at 

the bottom of the page. Do not use your browser’s back button—it will not work.

Returning to the survey itself. If you would like to complete the survey in multiple sessions—or 

if you accidentally get disconnected from the survey—you can return to your previous 

responses by clicking on the link that was in your invitation email. This will take you to where 

you left off (through the last full page you completed). You can keep returning until you click on 

the “submit” button on the final page of the survey.

Saving your responses. Your responses on any given survey page are saved when you click on 

either button at the bottom of the page (“save and go to next page” or “go back to previous 

page”). 
81

Adding optional comments. Toward the bottom of each survey page is a space where you can 

write in any comments you may have pertaining to the questions on the page. This is 

completely optional.

----------------------------------------[INTERNAL CODES]-----------------------------------------------

Treatment code (treatment): REL Southwest will pre-identify via an internal code that will not 

be listed on the survey whether teacher respondent works at a school that belongs to the 

treatment group or the control group. treatment = 1 indicates teacher is in the treatment 

group. treatment = 0 = control group. Unless otherwise noted, survey questions will appear for 

both groups of teachers.

Prior respondent code: REL Southwest will pre-identify those respondents who, from the same 

email address, completed the corresponding item in wave 1 spring 2015 survey. For these 

respondents, prior = 1. 

----------------------------------------[NEW SURVEY PAGE]-----------------------------------------------

Your	Position	and	Job	Responsibilities

We would like to update our information about your position.

[q1] Which of the following best describes your current position as a teacher of K-12 students? 

(Check one.)

 If you are not a teacher of K-12 students, or do not have teaching responsibilities in 

2015-2016, you have received this survey in error. Please let us know by replying to 

the invitation email or by calling {number}.

 Regular education teacher

 Special education teacher

 Other kind of teacher: ___________________________________

[q2x] What subject area(s) are you currently teaching? (Check all that apply.)

 [q2a] English/language arts/reading/writing

 [q2b] Mathematics

 [q2d] Science

 [q2d] Social Studies

 [q2e] Foreign language(s)

 [q2f] Visual or performing arts (art, music, etc.)

 [q2g] Physical education/health education

 [q2h] Career/technical education

 [q2i] Other subject area(s): [include fill-in box]
82

[q3] Which one of the following three groups of teachers do you belong to? These groups are 

defined by New Mexico’s teacher evaluation system, NM TEACH. 

 Group A: tested subject, tested grade

 Group B: grade 3 – 12 subject not linkable to NM’s standard based assessments

 Group C: grade K-2 teacher

[q4]Which of the following best describes your current teaching arrangement? (Check one.)

 I teach a single group of students all or most of the day in multiple subject areas. 

(Traditional elementary arrangement; sometimes called “self-contained.”)

 I teach several classes of different students during the day in a particular subject (for 

example, math) or perhaps two subjects (for example, you teach some math classes and 

some science classes). (Traditional secondary arrangement; sometimes called “subjectspecific” or “departmentalized”; at elementary level, sometimes called “subject matter 

specialist.” Also typical arrangement for physical education, art, music, etc.)

 I mainly teach selected students released from (or in) their regular classes in specific 

skills or to address specific needs (for example, special education, reading, English as a 

second language, gifted and talented.) (Sometimes called “pull out”, “resource,” or 

“push in” instruction.)

 I am one of two or more teachers who are jointly responsible for teaching the same 

subject(s) to a group of students (for example, in the same classroom), all or most of the 

time and/or in a majority of classes. (Sometimes called “co-teaching” or “job share.”)

 Other (please describe): _______________________________________

[q5] [if prior q5 blank] Including this school year (2015-2016), how many years have you 

been teaching, total, regardless of location?

Total years teaching: [fill in number: must be whole number less than 60]

[q6] [if prior q6 blank] Including this school year (2015-2016), how many years have you 

been teaching in your current school?

Years in current school: [fill in number: must be whole number less than 60]

[q7] Which of the following best describes your current position as a teacher of K-12 students? 

(Check all that apply)

 Spanish bilingual teacher

 Native American bilingual teacher

 Other bilingual teacher: ___________________________________
83

 Not a bilingual teacher

----------------------------------------[NEW SURVEY PAGE]-----------------------------------------------

Your	Experiences	This	Year	(SY	2015-2016)	of	Formal	Observations	for	Evaluation	

Purposes	

[* required question ] [q8] Thinking back over this whole school year 2015-2016, how many 

times were you formally observed by a school leader? School leader is defined as a principal, 

assistant principal, dean, or coach charged with authority to conduct formal observations of 

20+ minutes using the NM TEACH Observation Rubric {link to rubric}. 

 3 or more times

 2 times

 1 time

 0 times



[q9] [If q8 = 1, 2, or 3+ times] Use the slider to estimate how long the formal postobservation conferences typically took? Think only of conferences after formal observations; do 

not include informal feedback you might have received from classroom walk-throughs. 

[insert slider graphic here. tick marks for 0 min, 15 min, 30 min, 45 min, 60 min, 75 min, 90 min]

[q10] How frequently did you receive informal feedback this year from a school leader—for 

example, “wow or wonder” statements after brief walk-throughs. 

 At least once a week 

 At least once a month

 At least once per semester or trimester

 Never

 Other: ___________________

[q11x] [If q8 = 1, 2, or 3+ times] Use the slider to indicate how much you agree or disagree 

with the following statements about all of the conferences you had with a school leader 

following his or her formal observation(s) of your class this year (school year 2015-2016). 

Disagree 

Strongly

Disagree 

Somewhat

Agree 

Somewhat

Agree 

Strongly

[q11a] The feedback session itself, separate [insert slider graphic here]
84

from professional development I may have 

been recommended to take, helped me 

improve my instruction.

[q11b] In each conference, I brought 

documents to the conference, documents 

such as the lesson plan, and PDP.

[insert slider here]

[q11c] In each conference, the school leader

brought documents to conference, such as my 

summative score and the most recent PDP.

[insert slider here]

[q11d] In each conference the school leader 

identified at least one positive practice that I 

do well.

[insert slider graphic here]

[q11e]In each conference the school leader 

identified at least one challenge facing me. 

[insert slider graphic here]

[q11f]In each conference, the school leader 

used my scores on the NM TEACH 

Observation Rubric to determine which 

instructional practices to praise or critique.

[insert slider graphic here]

[q11g] In each conference, the school leader 

used my scores on the NM TEACH 

Observation Rubric to determine what 

professional development to recommend to 

the teacher.

[insert slider graphic here]

[q11h] In each conference, the school leader

ended the conference on a positive note.

[insert slider graphic here]

[q11i]Each conference followed a predictable 

format.

[insert slider graphic here]

[q11j]In each conference, my school leader 

and I mutually developed next steps for my 

instruction.

[insert slider graphic here]

.

[q11k]I walked away with a clear 

understanding of my school leader’s 

feedback.

[insert slider graphic here]

[q11l]My school leader listened to what I had 

to say during the conferences.

[insert slider graphic here]

[q11m]The observations were done to me, 

rather than for me. 

[insert slider graphic here]

[q11n]I received specific feedback about my [insert slider graphic here]
85

performance. 

[q11o] I received actionable feedback about 

my performance.

[insert slider graphic here]

[q11p]During the conferences, I committed 

to a specific set of next steps to improve my 

instruction. 

[insert slider graphic here]

[q11q] ] In each conference, the school 

leader spoke for almost all of the time.

[insert slider here]

[q11r] There was a high level of conflict in 

the conference. 

[insert slider graphic here]

[q11s In each conference I obtained 

recommendations for professional 

development tailored to my specific needs.

[insert slider graphic here]

[q11t] I trusted and accepted the feedback I 

received. 

[insert slider graphic here]

[q11u]I feel positive about the feedback I got 

during the conferences. 

[insert slider graphic here]

[q11v] I enjoyed most of the postobservation feedback conferences.

[insert slider graphic here]

[q11w] I was always provided with a written 

or online summary of the observation and my 

school leader’s comments on it. 

[insert slider graphic here]

[q12x] Please indicate whether you were ever rated this school year by your school leader as 

needing to improve upon or else excelling in the following observation items from NM TEACH, 

whether you received a recommendation to take up professional development (PD), and how you 

rate yourself on the item. Mark all that apply.

NM TEACH 

Observation Item

I was rated at 

least once as 

excelling on 

this item

I was rated at 

least once as 

needing to 

improve on 

this item

I was 

recommended 

to take PD 

because of 

this item

I don’t 

know how 

I was rated

How would 

you rate 

yourself 

this school 

year on this 

item? 

2A: Creating an 

environment of respect 

and rapport

[Ineffective, 

Minimally

Effective, 

Effective, 

Highly 
86

Effective, 

Exemplary

I don’t 

know]

2B: Organizing 

Physical Space

2C: Establishing a 

culture for learning

2D: Managing 

classroom 

procedures

2E: Managing Student 

Behavior

3A: Communicating 

with Students

3B: Using questioning 

and discussion 

techniques

3C: Engaging students 

in learning

3D: Assessment in 

Instruction

3E: Demonstrating 

flexibility and 

responsiveness

[q13a][if treatment=1 & q8 > 0] 

Have you ever received the “Checklist for New Mexico Principals’ Provision of Feedback to 

Teachers” , a guide to conducting formal feedback sessions? The first page of the Conversation 

Checklist looks like this: 
87

 Yes

 No

[q13b][if q8 > 0]Has your school leader ever used the “Checklist for New Mexico Principals’ 

Provision of Feedback to Teachers” when conducting the formal feedback conference in the 

2015-2016 school year? 

 Yes

 No

[q13c] [If q13b = yes)] For how many of your feedback sessions did your school leader(s) use 

the “Conversation Checklist”?

 3 or more observations

 2 observations

 1 observation

 None of the observations

 I don’t know

[q13d [If 13a = yes)] Did you view the testimonial video that was distributed along with the 

Conversation Checklist?

 Yes

 No

 I don’t know
88

[q13e][] During the formal feedback sessions, has a school leader ever used the “Guidance for 

New Mexico Principals’ About Provision of Feedback to Teachers” to guide any of your formal 

feedback sessions during the 2015-2016 school year? The first page of the Conversation 

Checklist looks like this: 

 Yes

 No

[q14x] If 13b=yes] Use the slider to indicate how much you agree or disagree with the 

following statements about your opinions about the Conversation Checklist this year (school 

year 2015-2016). 

Disagree 

Strongly

Disagree 

Somewhat

Agree 

Somewhat

Agree 

Strongly

[q14a] The Conversation Checklist was easy 

to use.

[insert slider graphic here]

[q14b] The Conversation Checklist took too 

much time to use.

[insert slider graphic here]

[q14c] The Conversation Checklist made the 

feedback conversation feel formulaic. 

[insert slider graphic here]

[q14d] The Conversation Checklist provided 

me and my school leader(s) a helpful 

structure for feedback. 

[insert slider graphic here]

[q14e] I received more critical feedback

when my school leader used the Conversation 

Checklist than in prior conferences when 

he/she didn’t use the Protocol. 

[insert slider graphic here]
89

[q14f] The Conversation Checklist helped me 

commit to specific set of next steps to 

improve my instruction.

[insert slider graphic here]

-----------------------------------------------------[NEW SURVEY PAGE]----------------------------------

Professional	Development	You	Received	This	Year	(SY	2015-2016)

The questions on this page ask about all of the professional development (PD) you have 

participated in during the 2015-2016 school year. Don’t include any professional development 

you may have participated in prior to your first formal observation this school year. 

 Professional development includes (but is not limited to) workshops, inservices, 

coaching/mentoring, collaborative planning/study, courses, institutes, professional 

conferences, self-directed learning, and so forth.

 Consider mandatory/required professional development as well as voluntary 

professional development.

[q15x] Please indicate all of the topics below that one or more school leader recommend you 

take. In addition, please also include which topics were covered in professional development 

you actually participated in this year (2015-2016). Select as many topics as apply.

A	school	

leader	

recommen

ded	I	take	

this	PD

I	took	the	

PD	online

I	received	

the	PD	in	

person

h

t

[q15a] Classroom management   

[q15b] Professionalism and communication in the classroom   

[q15c] Special education and inclusion   

[q15d] Standards-based lesson plans   

[q15e] Common Core Standards   

[q15f] Assessment (includes continuous assessment, portfolio 

assessment, Common Core aligned assessments)   

[q15g] Data-driven instruction   

[q15h] Early childhood   

[q15i] Instructional strategies or content in the content area(s) you 

teach—e.g., Mathematics; or Literacy PD   

[q15j] Response to Intervention   

[q15k] Teachscape Reflect training   
90

[q15l] Other NM TEACH training   

[q15m] Sheltering Instruction for Language development   

[q15n] Language Acquisition Design   

[q15o] Scaffolding instruction   

[q15m] Other: _______________________________   

[q15n] Other: _______________________________   

[q15o] Other: _______________________________   

[q16x] Thinking about the professional development you have participated in this year (2015-

2016, including summer 2015), use the slider to indicate how much you agree or disagree with 

each of the following statements. 

My professional development experiences this year...

Disagree	

Strongly

Disagree	

Somewhat

Agree	

Somewhat

Agree	

Strongly

[q14a] Were mostly developed by school 

leaders or teachers at my school. [insert slider graphic here]

[q14b] Were mostly job-embedded rather 

than one-time only sessions. [insert slider graphic here]

[q16a] Were varied and numerous enough to 

allow the school leader to customize PD 

recommendations based on my performance on 

specific domains of the NM TEACH 

Observation Rubric.{link}

[insert slider graphic here]

[q16b] Have been useful for helping me 

improve the achievement of my students

[insert slider graphic here]

[q16c]Have been useful for improving my 

instruction

[insert slider graphic here]

[q16d]Have taken more time than they were 

worth

[insert slider graphic here]

[q16e]Have been well aligned with the 

Common Core State Standards and/or 

curriculum based on these standards

[insert slider graphic here]

[q17x]Use the slider to indicate how much you agree or disagree with each of the following 

statements about the support you have received related to your professional development 

experiences this year (SY 2015-2016, including summer 2015). 
91

Disagree	

Strongly

Disagree	

Somewhat

Agree	

Somewhat

Agree	

Strongly

[q17a] I had easy access to a list of the 

possible professional development courses or 

offerings I could take. 

[insert slider graphic here]

[q17b]My school leader(s) has encouraged 

and supported my participation in professional 

development.

[insert slider graphic here]

[q17c]Sufficient resources (for example, 

substitute coverage, funding to cover 

expenses, stipends) have been available to 

allow me to participate in the professional 

development I need to teach effectively.

[insert slider graphic here]

[q17d]I have had sufficient flexibility in my 

schedule to pursue the professional 

development opportunities recommended for 

me.

[insert slider graphic here]

-----------------------------------------------------[NEW SURVEY PAGE]----------------------------------

Your	Perceptions	of	the	Quality	of	Your	School’s	Leadership

[q18] Rate your level of agreement with the following statement: “Overall, I was satisfied with 

the quality of my school’s leaders (principals, assistant principals, deans, department chairs) 

this year (school year 2015-2016).”

 Disagree strongly

 Disagree somewhat

 Agree somewhat

 Agree strongly 

[q19x] Thinking about this year (2015-2016), use the slider to indicate how much you agree or 

disagree with the following statements. “The school leaders who conduct formal observations 

of my classroom ... 

Disagree	

Strongly

Disagree	

Somewhat

Agree	

Somewhat

Agree	

Strongly

[q19a] Understand the subject 

matter content that I teach.

[insert slider graphic here]

[q19b] Understand what skills are 

needed for teachers to foster 

meaningful student learning. 

[insert slider graphic here]

[q19c] Foster an atmosphere of [insert slider graphic here]
92

trust and mutual respect in my 

school.

[q19d] Understand the 3 rating 

components of the NM Teach

Summative Evaluation Report

(student achievement, observations, 

multiple measures). 

[insert slider graphic here]

[q19e] Help to improve my 

teaching. 

[insert slider graphic here]

[q19f] Are instructional leaders. [insert slider graphic here]

-----------------------------------------------------[NEW SURVEY PAGE]----------------------------------

Your	Personal	Views	of	NM	TEACH	Educator	Effectiveness	System

[q20x]Use the slider to rate how much you agree or disagree with each of the following 

statements about the NM TEACH Educator Effectiveness System from school year 2015-2016. 

Disagree 

Strongly

Disagree 

Somewhat

Agree 

Somewhat

Agree 

Strongly

[q20a]The NM TEACH Educator 

Effectiveness System does a good job 

distinguishing effective from ineffective 

teachers.

[insert slider graphic here]

[q20b]The NM TEACH Educator 

Effectiveness System is fair to all teachers, 

regardless of their personal characteristics or 

those of the students they teach.

[insert slider graphic here]

[q20c]The NM TEACH Educator 

Effectiveness System has helped me pinpoint 

specific things I can do to help improve my 

performance.

[insert slider graphic here]

[q20d] I understand how the overall teacher 

ratings work in the NM TEACH Educator 

Effectiveness System.

[insert slider graphic here]

-----------------------------------------------------[NEW SURVEY PAGE]----------------------------------

Your	NM	TEACH	Rating	Summative	Evaluation	

[q21]Do you know what your level of Educator Effectiveness (Ineffective, Minimally Effective, 

Effective, Highly Effective, Exemplary) was for the school year 2015-2016? 
93

 Yes

 No

[q22] [if q21=yes] How did you learn your level of Educator Effectiveness for the school year 

2015-2016? Select all that apply.

 I received it via email from New Mexico Public Education Department

 I logged into my teacher portal online

 A school leader told me 

 I received it via my mailbox at school 

 Other: ______________________

 Other: ______________________

 I don’t recall

[q23] [if q21=yes] Did a school leader ever discuss with you how to improve your Educator 

Effectiveness score? 

 Yes, we had a conference

 Yes, in passing

 No

 I don’t recall 

[q24] [if q21=yes] Did a school leader ever discuss with you how to interpret your student 

achievement growth score? 

 Yes, we had a conference

 Yes, in passing

 No

 Not Applicable

 I don’t recall 

---------------------------------------[NEW SURVEY PAGE]---------------------------------------------

Optional	Comments	from	You

The following four questions are optional if you have any feedback you wish to share. If you do 
94

not, proceed to the next and final page.

[q25] [if 13a=yes] Do you have any suggestions to help us improve the Checklist for New 

Mexico Principals’ Provision of Feedback to Teachers?

[q26] Do you have any comments about professional development available to teachers in 

your district?

[q27] Do you have any comments about the NM TEACH Observation Rubric? 

[q28] Do you have any comments about the NM TEACH teacher evaluation system?

---------------------------------------[NEW SURVEY PAGE]---------------------------------------------

Thank	You!

Thank you so much for your time to complete this survey. 

Within the next 2 weeks, you will receive an email message with instructions about how to 

claim your gift card. This email message will come from {iCard (services@icardgiftcard.com)} 

and will be sent to the same email address at which you received the survey invitation, unless 

you indicate otherwise here:

 Yes, please send my gift card information to the same email address to which you sent the 

survey invitation.

 Please send my gift card information to a different email address, which I am providing 

here: _______________

 No, I do not wish to receive a gift card email from {iCard}.

---------------------------------------[NEW SURVEY PAGE]---------------------------------------------

Thanks	Again

You have completed the survey and may now close it.
95

APPENDIX B – Principal	and	Teacher	Recruitment	Materials

Wave	1	Principal	Recruitment	Script– SPRING	2015

Subject: Invitation to take $25 survey to provide feedback on the NM TEACH teacher evaluation 

process (and receive $25)

title first_name_ppal last_name_ppal

School_name

full_district_name

Dear title last_name_ppal,

The Regional Education Lab Southwest (REL Southwest) is studying principals’ feedback to 

teachers in an effort to improve the teacher evaluation process. Will you please be part of the 

study? 

Participation involves three things:

1. Complete a 30-minute online survey now using the link below (and receive $25).

2. Receive a short guide about providing effective feedback to teachers for your use 

throughout the 2015-2016 school year. You and your teachers will be assigned at random 

to one of two types of guidance.

3. Complete a second 30-minute online survey in spring 2016 (and receive an additional 

$25). 

All New Mexico public school principals are invited to participate in the study.

REL Southwest researchers will keep your survey answers confidential. The survey will not 

be used to evaluate you or your school. Only researchers from the REL Southwest will view your 

answers, and all answers will be aggregated before being included in the public report. The 

survey will give you an opportunity to provide feedback on the evaluation process, and the

results will help NM PED refine NM TEACH, and hopefully help other states improve their 

teacher evaluation systems. 

Click this link to read your rights as a research subject and to take the survey. 

https://survey.rand.org/research/hosting/ced/

Login Code: [MMIC code here]

If you prefer that we use a different email address for you, please send your request to the survey 

coordinator, [MMIC staff person here].
96

Below are some answers to frequently asked questions. If you have any additional questions 

about the survey, please feel free to contact me at [MMIC survey coordinator email address 

here], tel. [MMIC survey coordinator # here]. 

Sincerely,

[MMIC survey coordinator name]

Survey Coordinator

Southwest REL researcher

Frequently Asked Questions

How was I selected for this study? We are asking all New Mexico public school principals to 

complete the survey. 

What do I get for completing the surveys? You will receive a $25 gift card for completing each 

survey. You’ll be able to choose your gift card from more than 200 merchants. To see the full 

list of merchants, go to http://www.icardgiftcard.com/gift-cards/.

What is the purpose of the survey and how will the results be used? The Southwest Regional 

Educational Lab is conducting this study to learn about the feedback principals give to teachers 

under NM TEACH, and teachers’ perceptions of that feedback. The study is funded by the U.S. 

Department of Education. For more information about the study, visit {REL Southwest website}. 

Results will be used to provide New Mexico Public Education Department feedback regarding 

how to improve NM TEACH training for principals and teachers.

Who is conducting the study? Southwest REL researchers are conducting this study, including 

the survey. The researchers are from the RAND Corporation, which is an independent, 

nonprofit, nonpartisan research organization.

Who will get the study results? Only Southwest REL researchers will see your answers to 

survey questions. The survey results will be summarized across all survey respondents, and 

Southwest REL will present summaries of results to NM PED and to the federal Department of 

Education. We adhere to strict codes of confidentiality and will NEVER release individual 

responses in any way that could identify respondents.

Am I required to complete the survey? No, the survey is voluntary. However, we strongly 

encourage you to participate. The survey is an opportunity to share your views of NM TEACH 

and to describe the post-observation feedback process. 
97

How do I complete the survey? You complete the survey online, accessing it from a link shown 

above. This is your own individual link, so please do not share it with anyone or alter it.

Who can I contact if I have more questions or concerns? If you have general questions, you can 

contact [first and last name of survey coordinator]. [He/She] is available by email ([survey email 

account]@rand.org) or by phone (1-866-MMIC- – toll-free). If you have concerns or questions 

about your rights as a participant, contact the REL Southwest at SEDL (which is responsible for 

the protection of project participants): {email}, toll free at {XX}, or c/o IRB, {address}.
98

Wave	1	Teacher	Recruitment	Script– – SPRING	2015

Subject: Invitation to takesurvey to provide feedback about the NM TEACH teacher evaluation 

process (and receive $25)

title first_name_teacher last_name_teacher

School_name

full_district_name

Dear title last_name_teacher,

The Regional Education Lab Southwest (REL Southwest) is studying the quality of the feedback 

you receive after formal observations that are a part of NM TEACH. Will you please be part of 

the study? 

Your participation involves three things:

1. Complete a 30-minute online survey using the link below (and receive $25).

2. Receive a short document about effective feedback sessions between you and your school 

leader for your use throughout the 2015-2016 school year. You and your principal will be 

assigned at random to one of two types of guidance.

3. Complete a second 30-minute online survey in spring 2016 (and receive an additional 

$25). 

REL Southwest has first invited all New Mexico public school principals to participate in the 

study. Then in all schools where principals agree to participate, REL Southwest is inviting up to 

10 teachers to participate. You are one of only 10 teachers selected by REL Southwest to 

participate. Since you were randomly selected, we can’t replace you with any other teacher. 

Thus, your participation is very important. Principals who agree to participate are assigned at 

random to receive one of two types of guidance about effective feedback to teachers. Teachers 

who participate in the study receive the same guidance as their principals.

REL Southwest researchers will keep your survey answers confidential. The survey will not 

be used to evaluate you or your school. Only researchers from the REL Southwest will view 

your answers, and all answers will be aggregated before being included in the public report. The 

survey will give you an opportunity to provide feedback on the evaluation process, and the

results will help NM PED refine NM TEACH, and hopefully help other states improve their 

teacher evaluation systems. 

Click this link to read your rights as a research subject and to take the survey. 
99

https://survey.rand.org/research/hosting/ced/

Login Code: [MMIC code here]

If you prefer that we use a different email address for you, please send your request to the survey 

coordinator, [MMIC staff person here].

Below are some answers to frequently asked questions. If you have any additional questions 

about the survey, please feel free to contact me at [MMIC survey coordinator email address 

here], tel. [MMIC survey coordinator # here]. 

Sincerely,

[MMIC survey coordinator name]

Survey Coordinator

Southwest REL researcher

Frequently Asked Questions

How was I selected for this survey? We are asking up to 10 teachers to participate in each 

school where a principal has agreed to participate in the study. 

What do I get for participating in the study? You will receive a $25 gift card for completing 

each of two surveys. You’ll be able to choose your gift card from more than 200 merchants. To 

see the full list of merchants, go to http://www.icardgiftcard.com/gift-cards/.

What is the purpose of the study and how will the results be used? The Southwest Regional 

Educational Lab is conducting this study to learn about the feedback principals give to teachers 

under NM TEACH, and teachers’ perceptions of that feedback. The study is funded by the U.S. 

Department of Education. For more information about the study, visit {REL Southwest website}. 

Results will be used to provide New Mexico Public Education Department feedback regarding 

how to improve NM TEACH training for principals and teachers.

Who is conducting the study? Southwest REL researchers are conducting this study, including 

the survey. The researchers are from the RAND Corporation, which is an independent, 

nonprofit, nonpartisan research organization.

Who will get the survey results? Only Southwest REL researchers will see your answers to the 

surveys. The survey results will be summarized across all survey respondents, and Southwest 

REL will present summaries of results to NM PED and to the federal Department of Education. 

We adhere to strict codes of confidentiality and will NEVER release individual responses in any 

way that could identify respondents.
100

Am I required to complete the survey? No, the survey is voluntary. However, we strongly 

encourage you to participate. The survey is an opportunity to share your views of NM TEACH 

and to describe the post-observation feedback process. 

How do I complete the survey? You complete the survey online, accessing it from a link shown 

above. This is your own individual link, so please do not share it with anyone or alter it.

Who can I contact if I have more questions or concerns? If you have general questions, you can 

contact [first and last name of survey coordinator]. [He/She] is available by email ([survey email 

account]@rand.org) or by phone (1-866-MMIC- – toll-free). If you have concerns or questions 

about your rights as a participant, contact the REL Southwest at SEDL (which is responsible for 

the protection of project participants): {email}, toll free at {XX}, or c/o IRB, {address}.
101

Wave	2	Principal	Recruitment	Script– – SPRING	2016

Subject: Invitation to take final $25 survey to improve the NM TEACH teacher evaluation 

process

title first_name_ppal last_name_ppal

School_name

full_district_name

Dear title last_name_ppal,

Thank you for participating in the Regional Education Lab Southwest (REL Southwest) study of 

New Mexico principals’ feedback to teachers! You completed our spring 2015 survey. See your 

responses here. We are now inviting you to complete the second and final principal survey.

As a reminder, all New Mexico public school principals were invited to participate in the study, 

which is supported by both the New Mexico Public Education Department and your school 

district. (To see the letter of support from NM PED and districts, visit {REL Southwest 

website}.)

Click this link to read your rights as a research subject and complete the spring 2016 

survey.

https://survey.rand.org/research/hosting/ced/

Login Code: [MMIC code here]

REL Southwest researchers will keep your survey answers confidential. The survey will not 

be used to evaluate you or your school. Only researchers from the REL Southwest will view 

your answers, and all answers will be aggregated before being included in the public report. The 

survey results will help NM PED refine NM TEACH, and hopefully help other states improve 

their teacher evaluation systems. 

If you prefer that we use a different email address for you, please send your request to the survey 

coordinator, [MMIC staff person here].

Below are some answers to frequently asked questions. If you have any additional questions 

about the survey, please feel free to contact me at [MMIC survey coordinator email address 

here], tel. [MMIC survey coordinator # here]. 

Sincerely,

[MMIC survey coordinator name]

Survey Coordinator

Southwest REL researcher
102

Frequently Asked Questions

How was I selected for this survey? We are asking all public school principals who completed 

the spring 2015 survey (for $25) to also complete the 2016 survey (for another $25). 

What do I get for completing the survey? You will receive a $25 gift card for completing the 

survey. You’ll be able to choose your gift card from more than 200 merchants. To see the full 

list of merchants, go to http://www.icardgiftcard.com/gift-cards/.

What is the purpose of the survey and how will the results be used? The Southwest Regional 

Educational Lab is conducting this study to learn about the feedback principals give to teachers 

under NM TEACH, and teachers’ perceptions of that feedback. The study is funded by the U.S. 

Department of Education. For more information about the study, visit {REL Southwest website}. 

Results will be used to provide New Mexico Public Education Department feedback regarding 

how to improve NM TEACH training for principals and teachers.

Who is conducting the survey? Southwest REL researchers are conducting this survey. The 

researchers are from the RAND Corporation, which is an independent, nonprofit, nonpartisan 

research organization.

Who will get the survey results? Only Southwest REL researchers will see individual results. The 

survey results will be summarized across all survey respondents, and Southwest REL will 

present summaries of results to NM PED and to the federal Department of Education. We adhere 

to strict codes of confidentiality and will NEVER release individual responses in any way that 

could identify respondents.

Am I required to complete the survey? No, the survey is voluntary. However, we strongly 

encourage you to participate. The survey is an opportunity to share your views of NM TEACH 

and to describe the post-observation feedback process. 

How do I complete the survey? You complete the survey online, accessing it from a link shown 

above. This is your own individual link, so please do not share it with anyone or alter it.

Who can I contact if I have more questions or concerns? If you have general questions, you can 

contact [first and last name of survey coordinator]. [He/She] is available by email ([survey email 

account]@rand.org) or by phone (1-866-MMIC- – toll-free). If you have concerns or questions 

about your rights as a participant, contact the REL Southwest at SEDL (which is responsible for 

the protection of project participants): {email}, toll free at {XX}, or c/o IRB, {address}.
103

Wave	2	Teacher	Recruitment	Script– – SPRING	2015

Subject: Invitation to take final $25 survey to improve the NMTEACH teacher evaluation process

title first_name_teacher last_name_teacher

School_name

full_district_name

Dear title last_name_teacher,

Thank you for participating in the Regional Education Lab Southwest (REL Southwest) study of 

New Mexico principals’ feedback to teachers! You completed our spring 2015 survey. See your 

responses here. We are now inviting you to complete the second and final teacher survey.

As a reminder, all New Mexico public school principals were invited to participate in the study, 

which is supported by both the New Mexico Public Education Department and your school 

district. (To see the letter of support from NM PED and districts, visit {REL Southwest 

website}.)

Click this link to read your rights as a research subject and complete the spring 2016 

survey. 

https://survey.rand.org/research/hosting/ced/

Login Code: [MMIC code here]

REL Southwest researchers will keep your survey answers confidential. The survey will not 

be used to evaluate you or your school. Only researchers from the REL Southwest will view 

your answers, and all answers will be aggregated before being included in the public report. The 

survey results will help NM PED refine NM TEACH, and hopefully help other states improve 

their teacher evaluation systems. 

If you prefer that we use a different email address for you, please send your request to the survey 

coordinator, [MMIC staff person here].

Below are some answers to frequently asked questions. If you have any additional questions 

about the survey, please feel free to contact me at [MMIC survey coordinator email address 

here], tel. [MMIC survey coordinator # here]. 

Sincerely,

[MMIC survey coordinator name]

Survey Coordinator

Southwest REL researcher

Frequently Asked Questions
104

How was I selected for this survey? We are asking all teachers who completed the spring 2015 

survey (for $25) to take the spring 2016 survey (for another $25). We invited up to 10 teachers 

to take the 2015 survey in each school where a principal has agreed to participate in the study. 

What do I get for completing the survey? You will receive a $25 gift card for completing the 

survey. You’ll be able to choose your gift card from more than 200 merchants. To see the full 

list of merchants, go to http://www.icardgiftcard.com/gift-cards/.

What is the purpose of the survey and how will the results be used? The Southwest Regional 

Educational Lab is conducting this study to learn about the feedback principals give to teachers 

under NM TEACH, and teachers’ perceptions of that feedback. The study is funded by the U.S. 

Department of Education. For more information about the study, visit {REL Southwest website}. 

Results will be used to provide New Mexico Public Education Department feedback regarding 

how to improve NM TEACH training for principals and teachers.

Who is conducting the survey? Southwest REL researchers are conducting this survey. The 

researchers are from the RAND Corporation, which is an independent, nonprofit, nonpartisan 

research organization.

Who will get the survey results? Only Southwest REL researchers will see individual results. The 

survey results will be summarized across all survey respondents, and Southwest REL will 

present summaries of results to NM PED and to the federal Department of Education. We adhere 

to strict codes of confidentiality and will NEVER release individual responses in any way that 

could identify respondents.

Am I required to complete the survey? No, the survey is voluntary. However, we strongly 

encourage you to participate. The survey is an opportunity to share your views of NM TEACH 

and to describe the post-observation feedback process. 

How do I complete the survey? You complete the survey online, accessing it from a link shown 

above. This is your own individual link, so please do not share it with anyone or alter it.

Who can I contact if I have more questions or concerns? If you have general questions, you can 

contact [first and last name of survey coordinator]. [He/She] is available by email ([survey email 

account]@rand.org) or by phone (1-866-MMIC- – toll-free). If you have concerns or questions 

about your rights as a participant, contact the REL Southwest at SEDL (which is responsible for 

the protection of project participants): {email}, toll free at {XX}, or c/o IRB, {address}.
105

Wave	1	Principal	Recruitment	Script– SPRING	2015

Subject: Reminder: Invitation to take survey to provide feedback on the NM TEACH teacher 

evaluation process (and receive $25)

title first_name_ppal last_name_ppal

School_name

full_district_name

Dear title last_name_ppal,

Recently you received an email asking you to be a part of a study under The Regional Education 

Lab Southwest (REL Southwest) on principals’ feedback to teachers. Will you please be part of 

the study? 

Participation involves three things:

1. Receive a short guide about providing effective feedback to teachers for your use 

throughout the 2015-2016 school year. You and your teachers will be assigned at random 

to one of two types of guidance.

2. Complete a 30-minute online survey now using the link below (and receive $25).

3. Complete a second 30-minute online survey in spring 2016 (and receive an additional 

$25). 

All New Mexico public school principals are invited to participate in the study.

REL Southwest researchers will keep your survey answers confidential. The survey will not 

be used to evaluate you or your school. Only researchers from the REL Southwest will view your 

answers, and all answers will be aggregated before being included in the public report. The 

survey will give you an opportunity to provide feedback on the evaluation process, and the

results will help NM PED refine NM TEACH, and hopefully help other states improve their 

teacher evaluation systems. 

Click this link to read your rights as a research subject and to take the survey. 

https://survey.rand.org/research/hosting/ced/

Login Code: [MMIC code here]

If you prefer that we use a different email address for you, please send your request to the survey

coordinator, [MMIC staff person here].

Below are some answers to frequently asked questions. If you have any additional questions 

about the survey, please feel free to contact me at [MMIC survey coordinator email address 

here], tel. [MMIC survey coordinator # here]. 

Sincerely,
106

[MMIC survey coordinator name]

Survey Coordinator

Southwest REL researcher

Frequently Asked Questions

How was I selected for this study? We are asking all New Mexico public school principals to 

complete the survey. 

What do I get for completing the surveys? You will receive a $25 gift card for completing each 

survey. You’ll be able to choose your gift card from more than 200 merchants. To see the full 

list of merchants, go to http://www.icardgiftcard.com/gift-cards/.

What is the purpose of the survey and how will the results be used? The Southwest Regional 

Educational Lab is conducting this study to learn about the feedback principals give to teachers 

under NM TEACH, and teachers’ perceptions of that feedback. The study is funded by the U.S. 

Department of Education. For more information about the study, visit {REL Southwest website}. 

Results will be used to provide New Mexico Public Education Department feedback regarding 

how to improve NM TEACH training for principals and teachers.

Who is conducting the study? Southwest REL researchers are conducting this study, including 

the survey. The researchers are from the RAND Corporation, which is an independent, 

nonprofit, nonpartisan research organization.

Who will get the study results? Only Southwest REL researchers will see your answers to 

survey questions. The survey results will be summarized across all survey respondents, and 

Southwest REL will present summaries of results to NM PED and to the federal Department of 

Education. We adhere to strict codes of confidentiality and will NEVER release individual 

responses in any way that could identify respondents.

Am I required to complete the survey? No, the survey is voluntary. However, we strongly 

encourage you to participate. The survey is an opportunity to share your views of NM TEACH 

and to describe the post-observation feedback process. 

How do I complete the survey? You complete the survey online, accessing it from a link shown 

above. This is your own individual link, so please do not share it with anyone or alter it.

Who can I contact if I have more questions or concerns? If you have general questions, you can 
107

contact [first and last name of survey coordinator]. [He/She] is available by email ([survey email 

account]@rand.org) or by phone (1-866-MMIC- – toll-free). If you have concerns or questions 

about your rights as a participant, contact the REL Southwest at SEDL (which is responsible for 

the protection of project participants): {email}, toll free at {XX}, or c/o IRB, {address}.
108

Wave	1	Teacher	Recruitment	Script– – SPRING	2015

Subject: Reminder: Invitation to take survey to provide feedback about the NM TEACH teacher 

evaluation process (and receive $25)

title first_name_teacher last_name_teacher

School_name

full_district_name

Dear title last_name_teacher,

Recently you received an email asking you to be a part of a study under The Regional Education 

Lab Southwest (REL Southwest) on the quality of the feedback you receive after formal 

observations that are a part of NM TEACH. Will you please be part of the study? 

Your participation involves three things:

1. Complete a 30-minute online survey using the link below (and receive $25).

2. Receive a short document about effective feedback sessions between you and your school 

leader for your use throughout the 2015-2016 school year. You and your principal will be 

assigned at random to one of two types of guidance.

3. Complete a second 30-minute online survey in spring 2016 (and receive an additional 

$25). 

REL Southwest has first invited all New Mexico public school principals to participate in the 

study. Then in all schools where principals agree to participate, REL Southwest is inviting up to 

10 teachers to participate. You are one of only 10 teachers selected by REL Southwest to 

participate. Since you were randomly selected, we can’t replace you with any other teacher. 

Thus, your participation is very important. Principals who agree to participate are assigned at 

random to receive one of two types of guidance about effective feedback to teachers. Teachers 

who participate in the study receive the same guidance as their principals.

REL Southwest researchers will keep your survey answers confidential. The survey will not 

be used to evaluate you or your school. Only researchers from the REL Southwest will view 

your answers, and all answers will be aggregated before being included in the public report. The 

survey will give you an opportunity to provide feedback on the evaluation process, and the

results will help NM PED refine NM TEACH, and hopefully help other states improve their 

teacher evaluation systems. 

Click this link to read your rights as a research subject and to take the survey. 

https://survey.rand.org/research/hosting/ced/
109

Login Code: [MMIC code here]

If you prefer that we use a different email address for you, please send your request to the survey

coordinator, [MMIC staff person here].

Below are some answers to frequently asked questions. If you have any additional questions 

about the survey, please feel free to contact me at [MMIC survey coordinator email address 

here], tel. [MMIC survey coordinator # here]. 

Sincerely,

[MMIC survey coordinator name]

Survey Coordinator

Southwest REL researcher

Frequently Asked Questions

How was I selected for this survey? We are asking up to 10 teachers to participate in each 

school where a principal has agreed to participate in the study. 

What do I get for participating in the study? You will receive a $25 gift card for completing 

each of two surveys. You’ll be able to choose your gift card from more than 200 merchants. To 

see the full list of merchants, go to http://www.icardgiftcard.com/gift-cards/.

What is the purpose of the study and how will the results be used? The Southwest Regional 

Educational Lab is conducting this study to learn about the feedback principals give to teachers 

under NM TEACH, and teachers’ perceptions of that feedback. The study is funded by the U.S. 

Department of Education. For more information about the study, visit {REL Southwest website}. 

Results will be used to provide New Mexico Public Education Department feedback regarding 

how to improve NM TEACH training for principals and teachers.

Who is conducting the study? Southwest REL researchers are conducting this study, including 

the survey. The researchers are from the RAND Corporation, which is an independent, 

nonprofit, nonpartisan research organization.

Who will get the survey results? Only Southwest REL researchers will see your answers to the 

surveys. The survey results will be summarized across all survey respondents, and Southwest 

REL will present summaries of results to NM PED and to the federal Department of Education. 

We adhere to strict codes of confidentiality and will NEVER release individual responses in any 

way that could identify respondents.
110

Am I required to complete the survey? No, the survey is voluntary. However, we strongly 

encourage you to participate. The survey is an opportunity to share your views of NM TEACH 

and to describe the post-observation feedback process. 

How do I complete the survey? You complete the survey online, accessing it from a link shown 

above. This is your own individual link, so please do not share it with anyone or alter it.

Who can I contact if I have more questions or concerns? If you have general questions, you can 

contact [first and last name of survey coordinator]. [He/She] is available by email ([survey email 

account]@rand.org) or by phone (1-866-MMIC- – toll-free). If you have concerns or questions 

about your rights as a participant, contact the REL Southwest at SEDL (which is responsible for 

the protection of project participants): {email}, toll free at {XX}, or c/o IRB, {address}.
111

APPENDIX C – Secondary	Data	Elements

Student, teacher and principal administrative data variables

Below we list the names of the files and variables we will be requesting from NM PED on 

students, teachers and principals as a part of the analysis. The information is based on the Student 

Teacher Accountability Reporting System (STARS) Volume 1 User Guide, 2014-15. STARS is a comprehensive 

student, staff, and course information system that provides a standard data set for each student served by New 

Mexico’s 3Y/4Y – Grade 12 public education system. STARS uses the eScholar data warehouse which integrates 

detail student, staff, and course information via a standard set of templates that provide a consistent format for 

loading data from various student information systems used in public school districts or charter schools. For each 

data requested we list the Domain name, Template Name, and variable name and a variable description. 

Staff Domain, Staff Assignment Template

Field Name Description

District Code PED defined three character district code. 

Staff ID Masked Staff ID fixed across years and locations

Assignment Code PED defined assignment or position codes.

School Year Date School Year

Location Code PED defined three character location code of school

Assignment Date Report Date

Percentage of Time Assigned The portion of the staff member’s time that is allocated 

to the assignment.

Staff Domain, Staff Development Fact Template

Field Name Description

District Code PED defined three character district code. 

Staff ID Masked Staff ID fixed across years and locations

Activity Code Professional Development Activity Code

Activity Start Date Professional Development Activity Start Date

Staff Domain, Staff Template

Field Name Description

District Code PED defined three character district code. 

Staff ID Masked Staff ID fixed across years and locations

Gender Code Gender of Staff Member

Ethnic Code Ethnicity Code of Staff Member

Teaching of Principal Years of Experience The number of years that the staff member has been 

employed by any school district in any state as a 

TEACHER or PRINCIPAL (includes current district).

Teaching of Principal Years of Experience in District The number of years that the staff member has been 

employed by the current school district as a TEACHER 

or PRINCIPAL.

Highest Degree Earned Highest degree earned by the staff member.

Staff Qualification Status Staff Qualification Status such as Certified or Substitute 

teacher
112

Hispanic Indicator Indicator of whether the staff member is of Hispanic 

ethnicity

Tribal Affiliation The primary New Mexico Tribe or Pueblo with which 

the staff member is affiliated or enrolled

Student Domain, Student Template

Field Name Description

District Code PED defined three character district code. 

Student ID State issued student identification number.

Alternate Student ID District's local student identifier.

Location Code PED defined three character location code of school

School Year Date School Year

Current Grade Level Student current grade level

Birthdate Student date of birth.

Gender Code Gender code of Student

Ethnic Code Ethnicity Code of Student

Hispanic Indicator Indicator of whether the student is of Hispanic 

ethnicity

Economic Status Student participation in free or reduced lunch program.

Economic Disadvantaged Status Code Identifies the type of Direct Cert the student is receiving, 

e.g. SNAP, TANF, Homeless.

Special Education Student is IEP Y/N

Level of Integration Amount of special education service.

Repeating Last Year If student is in grades K-12, Y/N for retained or failed to 

be promoted following the completion of the past school 

year

Gifted Participation Code Indication of whether the student is Gifted

Migrant Status Indicator of migratory status

English Proficiency Code Current or former ELL student.

Assessment Domain, Assessment Fact Template

Field Name Description

District Code PED defined three character district code. 

Student ID State issued student identification number.

Test Description Description of type of test

Assessment School Year Date School Year

Item Description Code Item description code

Test Date Date of Assessment

Location Code PED defined three character location code.

Score Score of administered test.

Scoring Model The Measurement used to determine the outcomes the 

respective assessment
113

Attendance Domain, Student Daily Attendance Template

Field Name Description

District Code PED defined three character district code. 

Student ID State issued student identification number.

Location Code PED defined three character location code.

Attendance Date Report date of student unexcused absence.

Attendance Code Provide the Attendance Code that indicates the 

unexcused absence status for the date.

Attendance Domain, Student Summary Attendance Template

Field Name Description

District Code PED defined three character district code. 

Student ID State issued student identification number.

Location Code PED defined three character location code.

School Year Date School Year

Period Start Date Attendance period start date.

Period End Date Attendance period end date.

Days Present Provide the total number of days within the Reporting 

Period that the student was marked as Present at the 

location.

Days Enrolled Provide the total number of days within the Reporting 

Period that the student was counted as Enrolled at the 

location.

Course and Grades Domain, Course Template

Field Name Description

District Code PED defined three character district code. 

Location Code PED defined three character location code.

School Year Date School Year

Course Name Course Name

Semester Indicator of reporting period

Course Code PED-defined 8 character course code offered by the 

school.

Course and Grades Domain, Course Instructor Template

Field Name Description

District Code PED defined three character district code. 

Location Code PED defined three character location code.

School Year Date School Year

Primary Instructor ID State issued student identification number.

Course Name Course Name

Semester Indicator of reporting period

Course Code PED-defined 8 character course code offered by the 
114

school.

Section Code Class section number assigned by school district to 

identify distinct classes.

Snapshot Date Field indicating the report date. See valid values.

Special Course Indicator Indicator for honors, gifted, special education, etc.

Course and Grades Domain, Student Course Enrollment Template

Field Name Description

District Code PED defined three character district code. 

Location Code PED defined three character location code.

School Year Date School Year

Enrollment Period Reporting date

Student ID State issued student identification number.

Section Code Class section number assigned by school district to 

identify district classes.

Course Code PED-defined 8 character course code offered by the 

school.

Enrollment Domain, School Enrollment Template

Field Name Description

District Code PED defined three character district code. 

Location Code PED defined three character location code.

School Year Date School Year

Student ID State issued student identification number.

Enrollment Date The actual date the mobility transaction occurred.

Enrollment Code Code identifying the type of mobility transaction.

Enroll Change Code Provide a code that indicates the student enrollment was 

a result of SCHOOL CHOICE.
115

NM TEACH Reflect Data

Below we list the variables we will be requesting from NM PED from the Teachscape 

operated REFLECT teacher evaluation data system. 

 Staff ID that is consistent across school years and data sources

 Teacher Evaluation Group (A, B, or C)

 Graduated Consideration Tag

 District ID

 School year

 School NCES ID

 Masked Evaluator ID

 Observation number

 Observation date

 NM TEACH Summative evaluation component - Observation Element name

 NM TEACH Summative evaluation component - Observation Element Score (Level of 

Performance)

 NM TEACH Summative evaluation component - Student Achievement measure name

 NM TEACH Summative evaluation component - Student achievement measure score

 NM TEACH Summative evaluation component - Multiple measure name

 NM TEACH Summative evaluation component - Multiple measure score

 NM TEACH Summative evaluation overall score

 NM TEACH Summative evaluation effectiveness rating
116

Principal and Teacher contact information

File 1 – Principal contact information

 Principal Last Name

 Principal First Name

 Principal Middle Name or initial

 Principal Maiden Name

 Principal primary email address

 Principal secondary email address

 School Name

 School year

 School NCES ID

File 2 – Teacher contact information

 Teacher Last Name

 Teacher First Name

 Teacher Middle Name or initial

 Teacher Maiden name

 Teacher primary email address

 Teacher secondary email address

 School Name

 School year

 School NCES ID
117

APPENDIX	D	– Technical	Working	Group	Suggestions

In January 2014, Southwest REL researchers obtained comments from two TWG reviewers: Dan 

Goldhaber and Geoff Borman. This Appendix summarizes in bullet form the conceptual

suggestions that Southwest REL researchers received. Line edits are omitted from this summary.

The researchers response are noted under each bullet prefaced by “RESPONSE.” 

 The power of the intervention to produce changes in practice may be compromised 

if there are limited quality PD opportunities for principals to recommend in order 

for teachers to improve their instruction. This could lead the investigators to 

incorrectly conclude that the guide was not useful for changing practice and, 

eventually, achievement.

o RESPONSE: While ED’s contractor agrees that the quality of the PD modules 

could be an important moderator of the impact of the Guide, the decentralized 

acquisition or creation of PD by districts or by schools within districts makes it 

impractical to know ahead of time what the PD will be on offer during the school 

year 2015-2016. However, ED’s contractor believes that this lack of a 

standardized menu of pre-approved/pre-screened PD sessions is representative of 

the norm and thus a good test of the efficacy of a feedback guide in other states 

besides New Mexico.

 Unless it is vital for treatment group teachers to receive the treatment guide, it 

seems inadvisable because the threat of cross-over seems quite strong.

o RESPONSE: ED’s contractor decided to distribute the guide to teachers as well as 

to principals for the following two reasons: (a) To minimize Hawthorne effects, 

ED’s contractor intends to trump principals’ survey responses regarding usage of 

the guide with teachers’ responses about having seen or ever used the feedback 

checklist. (b) ED’s contractor intends to disseminate the checklist to teachers 

because the point of the treatment guide is to structure a two-way, mutual 

conversation instead of principals talking at teachers (as is currently the common 

practice). ED’s contractor agrees this increases the risk that the treatment guide 

will be shared with control group school staff. ED’s contractor think the best idea 

to discourage contamination and reduce John Henry effects is to disseminate a 

“Guide” to all principals, blinding them to their status in treatment or control 

group, which is addressed in the next point.

 The provision of [only] the treatment guide and the reliance on self-reported survey 

data (and principal-reported classroom observation scores) could cause Hawthorne 
118

effects due to both principals and teachers expecting improvement solely because 

they have been offered a new treatment. Providing a "guide" to all participating 

principals could, potentially, minimize the risk of cross-over (because everyone gets 

something), and would certainly address the issue of a Hawthorne effect.

o RESPONSE: ED’s contractor agrees and has adopted this suggestion. 

 None of the outcome measures are well-validated outcomes, and it is not clear 

whether the authors intend to construct composite measures from the surveys or 

whether outcomes will be based on single items.

o RESPONSE: ED’s contractor agreed and has included student achievement as an 

outcome (a well-validated outcome) and developed composite indexes derived 

from survey questions.

 The estimated recruitment rates of 65% for principals and 60% of teachers seem 

relatively high.

o RESPONSE: ED’s contractor has powered the study for a range of participation 

rates, between 50 and 65 percent for principals and between 50 and 60 percent for 

teachers, and demonstrated that for the research design the drop-off is very small 

from the upper to the lower ends of these ranges (approximately 0.02 to 0.03 in 

MDES). ED’s contractor based the upper end of our anticipated participation rates 

on prior experience with a similar efforts in New York City Public Schools, where 

there was also a direct connection to regular school responsibilities (although 

somewhat weaker than in the case of the current study) and participation was well 

over 60% (McCombs, Kirby, and Mariano, 2009).

 The teacher inclusion criteria for the study could bias the results of the study.

o RESPONSE: ED’s contractor agreed, and has dropped eligibility screeners for 

either principals or teachers. The only remaining eligibility criteria is that the 

study participant work in NM public schools at the time of the survey.

 The study would be considerably better if it had value added components. 

o RESPONSE: ED’s contractor agreed that value-added would be a highly useful, 

validated outcome. Therefore, ED’s contractor added student achievement as an 

outcome measure.

 It would be beneficial to include charter schools. 

o RESPONSE: This suggestion has been enacted.
119

APPENDIX	E	– Educational	Sciences	Reform	Act
120

APPENDIX	F	– IRB	Approval

NOTIFICATION OF INITIAL APPROVAL

From: Carolyn Tschopik 

To: Kata Mihaly - PI

Cc: Heather Schwartz - Primary Contact

Lou Mariano - Study Team

Heather Schwartz - Study Team

ID: 2014-0239

Title: New Mexico teacher feedback 

Description: The above study has been approved and all requirements have been met.

Approval of the study is for the period of 4/22/2014 to 4/21/2015.

Action 

Required: No action is required of you at this time.

CURRENT STUDY REVIEW STATUS

Project 

Number: There are no items to display

Funding 

Source: Department of Education

Prime 

Recipient:

Other, Specify:

SEDL
121

RAND 

Unit(s): Education

Human 

Subjects: Yes

Type of 

Review: Expedited

Component 

Approvals:

NM students

Procedure Approved Motion Category

secondary

data

Tue Apr 22 

00:00:00 PDT 

2014

Does not involve 

human subjects

principals & 

teachers who 

work in any 

NM public 

school

Procedure Approved Motion Category

secondary

data

Tue Apr 22 

00:00:00 

PDT 2014

Expedited 

-

Approved

Expedited Category:

 7 - Individual or 

group 

characteristics or 

behavior 

(Minimal risk 

surveys, 

interviews, focus 

groups, etc.)

principals & 

teachers 

whom we 

will recruit 

for 2 focus 

groups to 

pilot our 

surveys

Procedure Approved Motion Category

focus 

group to 

pilot our 

surveys

Tue Apr 

22 

00:00:00 

PDT 2014

Expedited 

-

Approved

Expedited Category:

 7 - Individual or 

group 

characteristics or 

behavior 

(Minimal risk 

surveys, 

interviews, focus 
122

groups, etc.)

principals 

whom we 

will recruit to 

participate in 

study

Procedure Approved Motion Category

Randomized 

Controlled 

Trial 

intervention

Tue Apr 

22 

00:00:00 

PDT 2014

Expedited 

-

Approved

Expedited Category:

 7 - Individual 

or group 

characteristics 

or behavior 

(Minimal risk 

surveys, 

interviews, 

focus groups, 

etc.)

Survey

Tue Apr 

22 

00:00:00 

PDT 2014

Expedited 

-

Approved

Expedited Category:

 7 - Individual 

or group 

characteristics 

or behavior 

(Minimal risk 

surveys, 

interviews, 

focus groups, 

etc.)

teachers 

whom we 

will recruit to 

participate in 

the study

Procedure Approved Motion Category

Randomized 

Controlled 

Trial 

intervention

Tue Apr 

22 

00:00:00 

PDT 2014

Expedited 

-

Approved

Expedited Category:

 7 - Individual 

or group 

characteristics 

or behavior 

(Minimal risk 

surveys, 
123

interviews, 

focus groups, 

etc.)

Survey

Tue Apr 

22 

00:00:00 

PDT 2014

Expedited 

-

Approved

Expedited Category:

 7 - Individual 

or group 

characteristics 

or behavior 

(Minimal risk 

surveys, 

interviews, 

focus groups, 

etc.)

Contingencies: Description Date Due Date Completed

There are no items to display

Assurance 

number: FWA00003425

IRB number: IRB00000051

Administrator: Carolyn Tschopik 

The HSPC is RAND's Institutional Review Board to review research involving human subjects, 

as required by federal regulations. RAND's "Federalwide Assurance for the Protection of Human 

Subjects" (FWA00003425, effective through July 1, 2018, at 

http://intranet.rand.org/groups/hspc/fwa.pdf ) serves as our assurance of compliance with the 

regulations of 16 federal departments and agencies. According to this assurance, the Committee 

is responsible for review regardless of source of funding.
124

APPENDIX	G	–Study	Participants’	Consent	

As shown in Appendix B, ED’s contractor will first email all New Mexico principals inviting 

them to participate in the study. Those principals who agree will click on a link enclosed in the 

email invitation, which will direct principals to the following consent form. Principals must click 

on the button indicating that they have read and understand the terms of participation and agree 

to participate. Upon clicking that button, they are directed to the spring 2015 survey. Teachers 

have a parallel consent process. The consent statements shown here a duplicate of the first 

section of the surveys included in Appendix A. 

Spring	2015	School	Leader	Survey

[[Version: On-line administration. Wave 1.]] 

Understanding	Your	Participation	in	the	Study

Welcome to the spring 2015 survey for school leaders about the feedback you give to teachers’ 

regarding their instruction as a part of the NM TEACH Educator Effectiveness System. 

Study Purpose. With funding from the federal Department of Education as a part of the 

Southwest Regional Educational Lab, this study is intended to learn about the feedback 

principals give to teachers under the NM TEACH Educator Effectiveness System, and teachers’ 

perceptions of that feedback. For more information about the study, please see {REL Southwest 

website}. In {month, year} REL Southwest researchers asked all New Mexico’s public school 

principals to participate in a study about evaluation feedback to teachers. Results will be used 

to provide New Mexico Public Education Department feedback regarding how to improve the 

NM TEACH Educator Effectiveness System training for principals and teachers.

Survey Purpose. This survey is intended to gather information on the provision of feedback to 

teachers after formal observations by school leaders. REL Southwest researchers will ask study 

participants to fill out an on-line, 30-minute survey once in spring 2015 and again in spring 

2016. Survey results will not be used to evaluate you, other teachers, or school leaders. 

Procedures. Online completion of the survey should take approximately 30 minutes. By using 

your personal survey link provided in the invitation email, you will be able to complete the 

survey in multiple sessions; responses will be saved as you enter them. However, once you click 

“submit” at the very end of the survey, you will no longer be able to return.

Confidentiality. REL Southwest will keep all survey responses confidential and will only use 

them for the purpose of the study. Our reports will contain summaries, and responses will 
125

never be presented in any way that would permit readers to identify you or your school. No 

one at your school, district, or the state will have access to survey responses that include 

respondents’ names, school names, or other information that could potentially be used to 

identify individuals or schools.

Participants. At the same time that we are emailing you, REL Southwest is also inviting all New 

Mexico public school school-based principals to participate in this study. Up to 10 teachers per 

school are also being surveyed. 

Benefits. Your completion of this survey gives you the opportunity to reflect on and share your 

perspective about the teacher evaluation system. You will be providing valuable information 

that may help New Mexico Public Education Department improve the NM TEACH Educator 

Effectiveness System. 

Compensation. Within two weeks of completion of this online survey, you will receive via email 

a link to your $25 online gift card for completing the survey. We will provide another $25 

online gift card upon completion of the spring 2016 survey to each participant.

Risks. The risks associated with your participation in this study are minimal. 

Voluntary Participation. Your participation in this survey is completely voluntary. You can 

decide not to participate or to discontinue your participation at any time. A few survey 

questions, marked with a red asterisk, are required for routing to later questions.

Questions? If you have any questions about this survey or the study as a whole, please feel free 

to call us toll-free at {XXX}. You can also contact {XX}, the survey coordinator, at {email} or by 

phone at {XX}. If you have concerns or questions about your rights as a participant, contact the 

REL Southwest at SEDL (which is responsible for the protection of project participants): {email}, 

toll free at {XX}, or c/o IRB, {address}.

[Required question] Informed Consent. By clicking the box below, you are indicating that you 

have read and understood the information provided to you about your participation in this 

survey.

 I have read and understood the information and choose to participate.

Thank you very much for your help in this important study!

Spring	2016	School	Leader	Survey

[[Version: On-line administration. Wave 2.]] 
126

Understanding	Your	Participation	in	the	Study

Welcome to the spring 2016 survey for school leaders about the feedback you give to teachers’ 

regarding their instruction as a part of the NM TEACH Educator Effectiveness System. This is a 

follow-on to our fall 2015 survey to learn about the feedback you have provided to teachers 

this school year.

Study Purpose. With funding from the federal Department of Education as a part of the 

Southwest Regional Educational Lab, this study is intended to learn about the feedback 

principals give to teachers under the NM TEACH Educator Effectiveness System, and teachers’ 

perceptions of that feedback. For more information about the study, please see {REL Southwest 

website}. In {month, year} REL Southwest researchers asked all New Mexico’s public school 

principals to participate in a study about evaluation feedback to teachers. Results will be used 

to provide New Mexico Public Education Department feedback regarding how to improve the 

NM TEACH Educator Effectiveness System training for principals and teachers.

Survey Purpose. This survey is intended to gather information on the provision of feedback 

after formal observations by school leaders to teachers about their performance. Survey results 

will not be used to evaluate you, other teachers, or school leaders. 

Procedures. Online completion of the survey should take approximately 30 minutes. By using 

your personal survey link provided in the invitation email, you will be able to complete the 

survey in multiple sessions; responses will be saved as you enter them. However, once you click 

“submit” at the very end of the survey, you will no longer be able to return.

Confidentiality. REL Southwest will keep all survey responses confidential and will only use 

them for the purpose of the study. Our reports will contain summaries, and responses will 

never be presented in any way that would permit readers to identify you or your school. No 

one at your school, district, or the state will have access to survey responses that include 

respondents’ names, school names, or other information that could potentially be used to 

identify individuals or schools.

Participants. REL Southwest originally invited all New Mexico public school school-based 

principals to participate in this study. Only those principals who completed surveys in spring 

2015 are being sent this survey now in spring 2016. 

Benefits. Your completion of this survey gives you the opportunity to reflect on and share your 

perspective about the teacher evaluation system. You will be providing valuable information 

that may help New Mexico Public Education Department improve the NM TEACH Educator 

Effectiveness System.
127

Compensation. Within two weeks of completion of the online survey, you will receive via email 

a link to your $25 online gift card for completing the survey.

Risks. The risks associated with your participation in this study are minimal. 

Voluntary Participation. Your participation in this survey is completely voluntary. You can 

decide not to participate or to discontinue your participation at any time. A few survey 

questions, marked with a red asterisk, are required for routing to later questions.

Questions? If you have any questions about this survey or the study as a whole, please feel free 

to call us toll-free at {XXX}. You can also contact {XX}, the survey coordinator, at {email} or by 

phone at {XX}. If you have concerns or questions about your rights as a participant, contact the 

REL Southwest at SEDL (which is responsible for the protection of project participants): {email}, 

toll free at {XX}, or c/o IRB, {address}.

[Required question] Informed Consent. By clicking the box below, you are indicating that you 

have read and understood the information provided to you about your participation in this 

survey.

 I have read and understood the information and choose to participate. 

Thank you very much for your help in this important study!
128

Spring	2015	Teacher	Survey

[[Version: Teachers. On-line administration. Wave 1.]]

Understanding	Your	Participation	in	the	Study

Welcome to the spring 2015 survey for school leaders about the feedback you give to teachers’ 

regarding their instruction as a part of the NM TEACH Educator Effectiveness System. 

Study Purpose. With funding from the federal Department of Education as a part of the 

Southwest Regional Educational Lab, this study is intended to learn about the feedback 

principals give to teachers under the NM TEACH Educator Effectiveness System, and teachers’ 

perceptions of that feedback. For more information about the study, please see {REL Southwest 

website}. In {month, year} REL Southwest researchers asked all New Mexico’s public school 

principals to participate in a study about evaluation feedback to teachers. Results will be used 

to provide New Mexico Public Education Department feedback regarding how to improve the 

NM TEACH Educator Effectiveness System training for principals and teachers.

Survey Purpose. This survey is intended to gather information on the provision of feedback to 

teachers after formal observations by school leaders. REL Southwest researchers will link your 

survey responses to basic employment information, such as race, ethnicity and years of 

experience that REL Southwest researchers will obtain from NM PED. Survey results will not be 

used to evaluate you, other teachers, or school leaders. REL Southwest researchers will ask 

study participants to fill out an on-line, 30-minute survey once in spring 2015 and again in 

spring 2016.

Procedures. Online completion of the survey should take approximately 30 minutes. By using 

your personal survey link provided in the invitation email, you will be able to complete the 

survey in multiple sessions; responses will be saved as you enter them. However, once you click 

“submit” at the very end of the survey, you will no longer be able to return.

Confidentiality. REL Southwest will keep all survey responses confidential and will only use 

them for the purpose of the study. Our reports will contain summaries, and responses will 

never be presented in any way that would permit readers to identify you or your school. No 

one at your school, district, or the state will have access to survey responses that include 

respondents’ names, school names, or other information that could potentially be used to 

identify individuals or schools.

Participants In each of the participating New Mexico public schools, REL Southwest has invited 

the principal to participate in this study Up to 10 teachers in your school were selected at 

random to be surveyed. 
129

Benefits. Your completion of this survey gives you the opportunity to reflect on and share your 

perspective about the teacher evaluation system. You will be providing valuable information 

that may help New Mexico Public Education Department improve the NM TEACH Educator 

Effectiveness System. 

Compensation. Within two weeks of completion of this online survey, you will receive via email 

a link to your $25 online gift card for completing the survey. We will provide another $25 

online gift card upon completion of the spring 2016 survey to each participant.

Risks. The risks associated with your participation in this study are minimal. 

Voluntary Participation. Your participation in this survey is completely voluntary. You can 

decide not to participate or to discontinue your participation at any time. A few survey 

questions, marked with a red asterisk, are required for routing to later questions.

Questions? If you have any questions about this survey or the study as a whole, please feel free 

to call us toll-free at {XXX}. You can also contact {XX}, the survey coordinator, at {email} or by 

phone at {XX}. If you have concerns or questions about your rights as a participant, contact the 

REL Southwest at SEDL (which is responsible for the protection of project participants): {email}, 

toll free at {XX}, or c/o IRB, {address}.

[Required question] Informed Consent. By clicking the box below, you are indicating that you 

have read and understood the information provided to you about your participation in this 

survey.

 I have read and understood the information and choose to participate.. 

Thank you very much for your help in this important study!

Spring	2016	Teacher	Survey

[[Version: Teachers. On-line administration. Wave 2.]]

Understanding	Your	Participation	in	the	Study

Welcome to the spring 2016 survey for teachers about feedback you receive about your 

instruction as a part of the NM TEACH Educator Effectiveness System. This is a follow-on to our 

spring 2015 survey to learn about the feedback you received from your school leaders last year.

Study Purpose. With funding from the federal Department of Education as a part of the 
130

Southwest Regional Educational Lab, this study is intended to learn about the feedback 

principals give to teachers under the NM TEACH Educator Effectiveness System, and teachers’ 

perceptions of that feedback For more information about the study, please see {REL Southwest 

website}. In {month, year} REL Southwest researchers asked all New Mexico’s public school 

principals to participate in a study about evaluation feedback to teachers. Results will be used 

to provide New Mexico Public Education Department feedback regarding how to improve the 

NM TEACH Educator Effectiveness System training for principals and teachers.

Survey Purpose. This survey is intended to gather information on the provision of feedback 

after formal observations by school leaders to teachers about their performance. REL 

Southwest researchers will link your survey responses to basic employment information, such 

as race, ethnicity and years of experience that REL Southwest researchers will obtain from NM 

PED. Survey results will not be used to evaluate you, other teachers, or school leaders. 

Procedures. Online completion of the survey should take approximately 30 minutes. By using 

your personal survey link provided in the invitation email, you will be able to complete the 

survey in multiple sessions; responses will be saved as you enter them. However, once you click 

“submit” at the very end of the survey, you will no longer be able to return.

Confidentiality. REL Southwest will keep all survey responses confidential and will only use 

them for the purpose of the study. Our reports will contain summaries, and responses will 

never be presented in any way that would permit readers to identify you or your school. No 

one at your school, district, or the state will have access to survey responses that include 

respondents’ names, school names, or other information that could potentially be used to 

identify individuals or schools.

Participants. In each of the participating New Mexico public schools, REL Southwest has 

invited the principal to participate in this study. Up to 10 teachers in your school were selected at 

random to be surveyed. Only those teachers who completed surveys in spring 2015 are being 

sent this survey now in spring 2016.

Benefits. Your completion of this survey gives you the opportunity to reflect on and share your 

perspective about the teacher evaluation system. You will be providing valuable information 

that may help New Mexico Public Education Department improve the NM TEACH Educator 

Effectiveness System. 

Compensation. Within two weeks of completion of the online survey, you will receive via email 

a link to your $25 online gift card for completing the survey.

Risks. The risks associated with your participation in this study are minimal. 
131

Voluntary Participation. Your participation in this survey is completely voluntary. You can 

decide not to participate or to discontinue your participation at any time. A few survey 

questions, marked with a red asterisk, are required for routing to later questions.

Questions? If you have any questions about this survey or the study as a whole, please feel free 

to call us toll-free at {XXX}. You can also contact {XX}, the survey coordinator, at {email} or by 

phone at {XX}. If you have concerns or questions about your rights as a participant, contact the 

REL Southwest at SEDL (which is responsible for the protection of project participants): {email}, 

toll free at {XX}, or c/o IRB, {address}.

[Required question] Informed Consent. By clicking the box below, you are indicating that you 

have read and understood the information provided to you about your participation in this 

survey.

 I have read and understood the information and choose to participate. 

Thank you very much for your help in this important study!
132

APPENDIX	H	- Confidentiality	Form	and	Affidavits

CONFIDENTIALITY AGREEMENT

Impacts of a Detailed Checklist on Feedback to Teachers

(RAND under Contract No. ED-IES-12-C-0012)

Safeguards for Individuals Against Invasion of Privacy: In accordance with the Privacy Act of 

1974 (5 United States Code 552a), the Education Sciences Reform Act of 2002 (Public Law 107-

279), the Federal Statistical Confidentiality Order of 1997, the E-Government Act of 2002 

(Public Law 107-347), and the Computer Security Act of 1987, American Institutes for 

Research (RAND) and all its subcontractors are required to comply with the applicable 

provisions of the legislation, regulations, and guidelines and to undertake all necessary

safeguards for individuals against invasions of privacy.

To provide this assurance and these safeguards in performance of work on this project, all staff, 

consultants, and agents of RAND, and its subcontractors who have any access to study data, shall 

be bound by the following assurance.

Assurance of Confidentiality

1. In accordance with all applicable legislation, regulations, and guidelines, RAND assures all 

respondents that their responses may be used only for statistical purposes and may not be 

disclosed, or used, in identifiable form for any other purpose except as required by law 

[Education Sciences Reform Act of 2002 (ESRA 2002), 20 U.S. Code, § 9573].

2. The following safeguards will be implemented to assure that confidentiality is protected as 

allowable by law (20 U.S.C. § 9573) by all employees, consultants, agents, and 

representatives of RAND and all subcontractors and that physical security of the records is 

provided:

a. All staff with access to data will take an oath of nondisclosure and sign an affidavit to 

that effect.

b. At each site where these items are processed or maintained, all confidential records that 

will permit identification of individuals shall be kept in a safe, locked room when not in 

use or personally attended by project staff.

c. When confidential records are not locked, admittance to the room or area in which they

reside shall be restricted to staff sworn to confidentiality on this project.

d. All electronic data shall be maintained in secure and protected data files, and personally 

identifying information shall be maintained on separate files from statistical data 

collected under this contract.

e. All data files on network or multi-user systems shall be under strict control of a database 

manager with access restricted to project staff sworn to confidentiality, and then only on 

a need-to-know basis.
133

f. All data files on single-user computers shall be password protected and all such machines 

will be locked and maintained in a locked room when not attended by project staff sworn 

to confidentiality.

g. External electronically stored data files (e.g., tapes on diskettes) shall be maintained in a 

locked storage device in a locked room when not attended by project staff sworn to 

confidentiality.

h. Any data released to the general public shall be appropriately masked such that linkages 

to individually identifying information are protected to avoid individual identification in 

disclosed data.

i. Data or copies of data may not leave the authorized site for any reason.

3. Staff, consultants, agents, or RAND and all its subcontractors will take all necessary steps to 

ensure that the letter and intent of all applicable legislation, regulations, and guidelines are 

enforced at all times through appropriate qualifications standards for all personnel working 

on this project and through adequate training and periodic follow-up procedures.

By my signature affixed below, I hereby swear and affirm that I have carefully read this 

statement and fully understand the statement as well as legislative and regulatory assurances that 

pertain to the confidential nature of all records to be handled in regard to this project, and will 

adhere to all safeguards that have been developed to provide such confidentiality. As an 

employee, consultant, agent, or representative of RAND or one of its subcontractors, consultants, 

agents, or representatives, I understand that I am prohibited by law from disclosing any such 

confidential information to anyone other than staff, consultant, agents, or representatives of 

RAND, its subcontractors, or agents, and Institutes of Education Science. I understand that any 

willful and knowing individual disclosure or allowance of disclosure in violation of the 

applicable legislation, regulations, and guidelines is punishable by law and would subject the 

violator to possible fine or imprisonment.

(Signature) (Date)
134

AFFIDAVIT OF NONDISCLOSURE

Impacts of a Detailed Checklist on Feedback to Teachers

(RAND under Contract No. ED-IES-12-C-0012)

[insert name]

[insert position]

Date of Assignment to Impacts of a Detailed Checklist on Feedback to Teachers: November 

2013

RAND 

1776 Main Street

Santa Monica, CA 90401-3208

I, [insert name], do solemnly swear (or affirm) that when given access to any Impacts of a 

Detailed Checklist on Feedback to Teachers databases or files containing individually

identifiable information, I will not:

i. use or reveal any individually identifiable information furnished, acquired, retrieved or 

assembled by me or others, under the provisions of Section 183 of the Education 

Sciences Reform Act of 2002 (PL 107-279) and Title V, subtitle A of the E-Government

Act of 2002 (PL 107-347) for any purpose other than statistical purposes specified in the 

NCES survey, project or contract;

ii. make any disclosure or publication whereby a sample unit or survey respondent could be 

identified or the data furnished by or related to any particular person under this section 

could be identified; or

iii. permit anyone other than the individuals authorized by the Commissioner of the National 

Center for Education Statistics to examine the individual reports.

(Signature)

(The penalty for unlawful disclosure is a fine of not more than $250,000 [under 18 U.S.C. 3571] 

or imprisonment for not more than 5 years [under 18 U.S.C. 3559], or both. The word "swear" 

should be stricken out wherever it appears when a person elects to affirm the affidavit rather than 

to swear to it.)

State of _____________________________

County of _______________________________

Subscribed and sworn/affirmed before me, ______________________, a Notary Public in and 
135

for

________________County, State of ________________________, on this date, 

______________________.

___________________________________________

Notary Public

My commission expires: _____________________________.