APPENDIX D

RECRUITING PROTOCOL WITH DISTRICT ADMINISTRATOR
DISTRICT ADMINISTRATOR RECRUITING PROTOCOL

INTRODUCTION AND REACHING THE PROPER PERSON

[Use the district director of early childhood programs as the first point of contact.]

IF YOU REACH THE GATEKEEPER:

Hello, my name is ___________________ from Mathematica Policy Research. I am calling on 

behalf of the U.S. Department of Education, which is conducting a study of Tiered Quality 

Rating and Improvement Systems (TQRIS) under the Race to the Top—Early Learning 

Challenge program. The study will involve all states that were awarded Round 1 RTT-ELC 

grants and selected districts in some of the RTT-ELC states. May I speak with 

__________________ about this study? 

[Before you are transferred, get the gatekeeper’s name. If contact is not available, ask to leave 

voicemail on direct line.]

IF YOU REACH THE CONTACT DIRECTLY:

Hello, my name is ___________________ from Mathematica Policy Research. I am calling on 

behalf of the U.S. Department of Education. You were sent a letter from the U.S. Departments of 

Education and Health and Human Services, which I hope you received, that requested your 

participation in the study of Tiered Quality Rating and Improvement Systems in Race to the 

Top—Early Learning Challenge (or RTT-ELC) states. Mathematica is conducting this study for 

the U.S. Department of Education. The study will involve all states that were awarded grants in 

Round 1 of the Race to the Top—Early Learning Challenge competition and selected districts in 

some of those states. I’m calling to discuss the study plans and what your district’s participation 

will entail, as well as to address your questions about the process.

As you may know, a key aspect of the RTT-ELC program is the development and 

implementation of Tiered Quality Rating and Improvement Systems (TQRIS). This study will 

focus on the TQRIS developed and implemented under RTT-ELC, to inform TQRIS 

development and refinement across the nation.

This discussion will take about 30 minutes. If now is not a good time or you would like to 

include others in the call, we can set up another time, preferably within the next two weeks, to 

continue this discussion. 

May I proceed, or would you like to schedule another time to continue our discussion?

[IF WANT TO SCHEDULE ANOTHER TIME] When would be the best time to talk in the next 

two weeks? [MAKE A FIRM APPOINTMENT FOR THE FOLLOW-UP CALL.]

[IF WILLING TO PROCEED WITH CALL] Great! I’ll first provide a brief overview of the 

study and address questions you may have. I then have some questions about center-based early 

care and education (ECE) programs in your district and the availability of kindergarten entry 

assessment data. In preparation for this call, we reviewed your state’s RTT-ELC grant 
application and talked to your state’s grant coordinator. During this call, I would like to confirm 

the information we collected and collect some additional details. 

If, during our conversation, some topics come up that you are uncertain about and you think of 

other people I should talk to, please let me know so that I can arrange to get in touch with them 

about those topics. Also, if you think of documents that contain information that could address

the questions I am asking, please let me know so the study team can review those materials.

BACKGROUND, PURPOSE, AND STUDY OVERVIEW

[Provide enough details about the study to proceed with the call. Do not get bogged down in 

design issues.]

The first phase of the study involved interviewing state administrators in fall 2014 to gather 

information on the structure of each state’s TQRIS and the components that factor into quality 

ratings. 

Now, we are turning to the second phase of the study, in which we will examine the relationship 

between TQRIS ratings and children’s kindergarten readiness outcomes. We plan to collect 

TQRIS and kindergarten assessment data as they become available in fall 2015. This is the part 

of the study for which we are seeking your district’s assistance and participation.

Do you have any questions so far about the study? [If so, answer to best of your ability. If you 

are not sure of the answer, please say “That is a good question. I will ask the study team and get 

back to you as soon as I can.”]

A. QUESTIONS ABOUT DISTRICT-SPONSORED PRESCHOOL PROGRAMS IN 

THE DISTRICT

[If this is a follow-up call, thank the contact for taking the time to talk and reiterate that we are 

focusing on the district’s preschool programs.]

I would like to begin by asking you some questions about district-sponsored preschool programs 

in your district. By “district-sponsored preschool programs,” I mean center-based ECE programs 

serving children ages 3 to 5 that operate with district oversight. Such programs may include 

preschool programs located in public schools, or other preschool programs affiliated with the 

district. My questions are about programs that are currently operating as well as those you expect 

to be operating in fall 2015, when we will be conducting our data collection.

If you’re not sure about an answer, please let me know, and I’ll send the question in a follow-up 

email. 

1. What is the total number of preschool-age children in your district?

a. What is total number of low-income preschool-age children in your district?

2. How many district-sponsored preschool programs are currently in your district?

a. How many children do these programs serve?

b. How many low-income children do these programs serve?
c. How many district-sponsored preschool programs do you expect in your district in 

fall 2015?

d. Do your current district-sponsored preschool programs serve children younger than 

age 4, in addition to 4-year-olds? 

i. If yes, how many of the children served by state-funded preschool programs are 

younger than age 4?

ii. If no, do you expect these programs to serve children younger than age 4 in fall 

2015?

3. Do all elementary schools in your district offer district-sponsored preschool 

programs?

a. [If no] Would you be able to send me a list of elementary schools in your district that 

offer preschool programs?

4. Roughly what percentage of children enrolled in district-sponsored preschool 

programs in your district go on to attend public kindergarten in your district? 

5. Are any district-sponsored preschool programs located in community-based centers?

a. [If yes] How many district-sponsored preschool programs in your district are 

located in community-based centers? 

6. By fall 2015, how many district-sponsored preschool programs located in 

community-based centers do you expect there to be in your district?

B. QUESTIONS ABOUT KINDERGARTEN ASSESSMENT

Now I have some questions about your district’s kindergarten entry assessment (KEA). 

1. According to your state’s RTT-ELC application, you are currently using [INSERT 

NAME OF INSTRUMENT INDICATED BY STATE] as your KEA. Is that correct?

a. [IF NOT USING STATE-IDENTIFIED ASSESSMENT] What instrument are you 

using? 

i. Which of the following domains does this assessment measure? [Retain in list 

below only those domains identified in the application; delete others that are 

not listed.] 

 Language and literacy

 Cognition and general knowledge (including early mathematics and early 

scientific development)

 Approaches toward learning

 Physical well-being and motor development (including adaptive skills)

 Social and emotional development
ii. Are there any other domains that are measured by the assessment? [Retain in list 

below only those domains that are NOT listed in the application; delete others.] 

For example:

 Language and literacy

 Cognition and general knowledge (including early mathematics and early 

scientific development)

 Approaches toward learning

 Physical well-being and motor development (including adaptive skills)

 Social and emotional development

 Other (specify)

b. Are there separate scores for each domain of school readiness covered by the 

assessment?

c. Was the assessment piloted? If yes, when did the pilot take place?

d. How long have you been using this assessment?

e. Have any notable changes or revisions been made to the assessment since it was first 

implemented? If yes, describe the changes or revisions made, the motivation for 

them, and when they were made.

f. Are there any plans to change this assessment? Would the change be implemented by

fall 2015?

i. [If yes] Can you tell me the name of the new assessment you will be using? Will 

that assessment be state-developed, commercially available, or other (such as a 

modified or adapted version of a commercially available assessment)?

ii. [If yes] Can you tell me which of the following domains the new assessment 

will measure? [Encircle all domains identified below.]

 Language and literacy

 Cognition and general knowledge (including early mathematics and early 

scientific development)

 Approaches toward learning

 Physical well-being and motor development (including adaptive skills)

 Social and emotional development

 Other (specify)

2. Is this assessment mandatory for all kindergarteners in the district?

a. [IF MANDATORY] When did it become mandatory?

b. [IF NOT MANDATORY] Are there plans to make it mandatory for all 

kindergarteners? When will these plans be implemented? 

c. [IF NOT MANDATORY] How are children selected to take the test? 
i. Are children randomly selected to take the test? 

ii. Do all schools administer the assessment?

iii. Do you select a certain number of kindergarteners from each school? If so, how 

many?

iv. How many kindergarteners take the assessment in your district? 

 Does the number vary substantially from year to year?

 How do you determine the number of children who are tested?

 For context, about how many incoming kindergarteners are there in the 

district each year? 

d. Under what circumstances would a child be exempted from taking the test?

3. When is the assessment typically administered? [Ask whether fall and/or spring of 

each year and which months.]

 Within the first week of school

 Within the first month of school

 Beyond the first month, specify: ______________________

 Varies by district or school (specify range): _____________________

4. What happens if a child misses the assessment administration?

5. Under what circumstances would a child be exempted from taking the assessment?

I now have some questions about how the kindergarten assessment data are stored and whether it 

is possible to link the kindergarten data to data on children’s preschool. We are seeking this 

information because we need to link student-level kindergarten assessment data to the TQRIS 

data of the preschool program the student attended.

6. Are KEA data currently stored in a district database?

a. [If yes] In which database are the data stored?

b. [If no] Will these data be stored in a district database by fall 2015?

i. [If yes] In which database will the data be stored?

7. Does the district database that includes the kindergarten assessment data contain 

student ID numbers?

a. How/when are students assigned a student ID? 

b. Does the district keep the same ID number for a student from year to year?

i. [If no] Under what circumstances, if any, would a child’s district ID change?

8. Are children assigned student ID numbers during preschool only if they attend a 

district-sponsored preschool program? 
9. Would children assigned a district student ID when they enroll in a districtsponsored preschool program have the same district ID when they go on to 

kindergarten?

10. Are student IDs used by your district identical to those used by the state?

a. [If no] Does the district database also contain a state student ID?

11. If a kindergartener attended a district-sponsored preschool program in the district, 

would we be able to identify which program they attended using the district KEA 

database or another district database? [If another database is identified, note which 

database.]

a. For example, do any district databases contain data on kindergarteners’ previous 

school history (meaning whether the child attended a center-based ECE program and 

where)?

i. Does the district database contain data on all previous schools the child attended 

or just the last school attended?

ii. How is a child’s previous school(s) identified in the database? Is there a 

program ID, program name, or school name in the database? 

b. Would it be possible to obtain preschool enrollment rosters from school year 2014–

2015 for all district-sponsored preschool programs in the district?

c. If it is currently not possible to identify the preschool programs that kindergarteners 

attended or to obtain preschool enrollment rosters, will it be possible to do so by fall 

2015?

12. I now have a question that may be difficult to address off the top of your head, so 

we understand if this is something we’ll need to follow up on after the call. Roughly 

what percentage of kindergarten students were previously enrolled in a districtsponsored preschool program in your district? [This is likely something they won’t 

know off the top of their head, so tell them you will send this question in a follow-up 

email and that we’d like to receive the answer within two weeks.] 

13. Does the KEA district database or any other district database contain the following 

student-level data? If not, will these data be available by fall 2015?

a. Gender

b. Socioeconomic status (for example, free or reduced-price lunch status)

c. Race/ethnicity

d. English-language-learner status

e. Special education status

f. Maternal education 

g. Preschool and kindergarten attendance 

14. What is the process for obtaining data from your district? 
a. Do we need to submit a formal data request? To whom?

b. How many years of KEA data using the current assessment are available?

c. In what month do KEA results typically become available each year?

d. What is the turnaround time (after submitting a request) for receiving these data?

15. Who should be our main contact person for collecting student-level data (including 

kindergarten assessment data)? [GET NAME, TITLE, TELEPHONE NUMBER, 

AND EMAIL.]

C. WRAP-UP

Thank you for this extremely helpful information. 

[IF THERE ARE OUTSTANDING QUESTIONS] As I indicated, I will follow up with an email 

listing the questions that are still outstanding. It would be great to obtain your responses to those 

questions within the next two weeks. Would that work?

In the next three weeks, we will contact the staff involved with the district’s KEA data system to 

discuss next steps for us to obtain these data. I will also send you a draft memorandum of 

understanding (MOU) for the district to review and sign. It will summarize what participating in 

the study entails. 

Are you the best person to contact regarding the MOU and any other data collection 

requirements? If not, please let me know who I should work with. 

Are there any other district requirements or necessary steps that we should be aware of to move 

forward with the study?

[IF DISTRICT SAYS A RESEARCH APPLICATION IS REQUIRED] We have a standardized 

research application that we plan to use with all districts participating in this study. We are 

prepared to submit this to your district. To whom should I send the application?

[IF SOMEONE ELSE, GET THAT PERSON’S CONTACT INFORMATION AND ASK FOR 

AN EMAIL INTRODUCTION.]

[AVOID THIS IF AT ALL POSSIBLE, BUT IF DISTRICT INSISTS THAT THEIR OWN 

APPLICATION BE SUBMITTED] Can you please direct me to the forms we will need to 

submit?

Before we end the call, do you have any questions for me? 

Thank you so much for your time.
AFTER THE CALL

MAKE SURE YOU: 

1. SEND AN EMAIL (TEMPLATE BELOW) TO THANK THE CONTACT AND 

CONFIRM ANY NEXT STEPS THAT YOU AGREED TO DURING THE CALL.

2. ENTER INFORMATION FROM YOUR CALL IN SHAREPOINT AND UPDATE 

PROTOCOL DOCUMENTATION TOOL.

TEMPLATE FOR THANK YOU EMAIL (FOLLOWING INTRODUCTORY PHONE CALL)

Subject: ED Study of RTT-ELC TQRIS

Attachment: District Memorandum of Understanding (Appendix C)

Good [morning/afternoon] [Dr./Mr./Mrs.] _________________,

I want to thank you again for speaking with me on [Date] for the U.S. Department of Education 

study of Tiered Quality Rating and Improvement Systems (TQRIS) under the Race to the Top—

Early Learning Challenge program. The information you shared will inform TQRIS development 

and refinement across the nation. 

I am following up on the next steps that we discussed at the end of our call. [Adjust remaining 

email text as appropriate to cover the relevant issues.]

First, please respond to the following questions that we did not get through during our call. As 

we discussed, it would be helpful if you could send your written responses by _____ (date).

1. [Question 1]

2. [Question 2]

3. [And so on]

In addition, I have attached a draft memorandum of understanding (MOU) for your district to 

review and sign. This document summarizes what participating in the study entails. Once the 

appropriate district personnel have had a chance to review the document, please return a signed 

PDF copy to me at _______________ (email address).

Finally, I have attached a copy of the [formal] data request which we plan to submit to 

_____________ (person), according to your instructions, on _________ (date).

Please do not hesitate to contact me if you have any questions that I might be able to address in 

the meantime.

Sincerely,

(name) [Include your phone number and email]