**APPENDIX D**  **RECRUITING PROTOCOL WITH DISTRICT ADMINI**  **STRATOR DISTRICT ADMINISTRAT**  **OR RECRUITING PROTOC OL** 

**INTRODUCTION AND REACHING THE PROPER PERSON** [ Use the district director of early childhood programs as the first point of contact. ] IF YOU REACH THE GATEKEEPER: Hello, my

name is ___________________ from Mathematica Policy Research. I am calling on behalf of the U.S. Department of Education, which is conducting a study o f Tiered Quality Rating and Improvement Systems (TQRIS) under the Race to the Top

—Earl y Learnin g Challenge program. The study will involve all states that were awarded Round 1 RTT - ELC grants and selected districts in some of the RTT - ELC states. May Ispeak with __________________ about this study? [Before you are transferred, get the gatekeeper’s n ame. If contact is not available, ask to leave voicemail on direct line. ] IF YOU REACH THE CONTACT DIRECTLY: Hello, my

name is ___________________ from Mathematica Policy Research. I am calling on behalf of the U.S. Department of Education. You were sent a letter from the U.S. Department s of Education and Health and Human Services , which I hope you received, that requested your participation in the study o f Tiered Quality

Rating and Improvement Systems in Race to the Top

—Earl y Learning Challenge (or RTT - ELC ) states. Mathematica is conducting this study f or the U.S. Department of Education. The study will involve all states that were awarded grants in Round 1 of the Race to the Top

—Early L earning Challenge competition and selected districts in some of those s tates. I’m calling to discuss the study p lans and what your district’s participation will entail, as well as to address your questions about the process. As you may

know, a key

a spect of the RTT - ELC program is the development and implementation of Tiered Quality Rating and Improvement Systems (TQRI S). This study will focus on the TQRIS developed and implemented under RTT - ELC, to inform TQRI S development and refinement across the nation. This discussion will take about 30 minutes. If now is not a good time or you would like to include others in the call, we can set up another time, preferabl y within the next two weeks, to continue this discussion. May Iproceed, or would you like to schedule another time to continue our discussion? [IF WANT TO SCHEDULE ANOTHER TIME] When would be the best time to talk in the next two weeks? [MAKE A FIRM APPOINTMENT FOR THE FOLLOW - UP CALL.] [IF WILLING TO PROCEED WITH CALL] Great! I’ll first provide a brief overview of the study and address questions you may h ave. I then have some questions about center-based earl y care and education (ECE) programs in your district and the availability of kindergarten entry assessment data. In preparation for this call, we reviewed your state’s RTT - ELC grantapplication and talked to your state ’s grant coordinator. During this call, I would like to confirm

the information we collected and collect some additional details. If, during our conversation, some topics come up that you are uncertain about and you think of other people I should talk to, please let me know so that I can arrange to get in touch with them about those topics. Also, if you think of documents that contain information that could address the questions Iam asking, please let me know so the study team can review those materials. **BACKGROUND, PURPOSE, AND STUDY OVERVIEW** [Provide enough details about the study to proceed with the call. Do not get bogged down in design issues .] The first phase of the study involved interviewing state administrators in fall 2014 to gather information on the structure of each state’s TQRIS and the components that factor into quality ratings. Now, we are turning to the second phase of the study, in which we will examine the relationship between TQRI S ratings and children’s kindergarten readiness outcomes. We plan to collect TQRI S and kindergarten assessment data as they b ecome available in fall 2015. This is the part of the study

for which we are se eking your district’s assistance and participation. Do you have any questions so far about the study? [If so, answer to best of your ability
. If you are not sure of the answer, please say “That is a good question. I will ask the study team and get back to you as soon as I can.” ] **A. QUESTIONS**  **ABOUT DISTRICT**  **- SPONSORED**  **PRESCHOOL PROGRAMS**  **IN THE DISTRICT**  [If this is a follow
- up call, thank the contact for taking the time to talk and reiterate that we are focusing on the district’s preschool programs .] I would l ike to begin by

a sking you some questions about district - sponsored preschool programs in your district. By

“ district - sponsored preschool programs ,” I mean center -based ECE programs serving children ages 3 to 5 that operate with district oversight . Such pro grams may include preschool programs located in public schools, or other preschool programs affiliated with the district. My

questions are about programs that are currentl y operating as well as those you expect to be operating in fall 2015, when we will be conducting our data collection. If you’re not sure about an answer, please let me know , and I’ll send the question in a follow -up email. 1. What is the total number of preschool - age children in your district? a. What is total number of low - income preschool - age children in your district? 2. How many district - sponsored preschool programs are currentl y in your district? a. How many children do these programs serve? b. How many low - income children do these programs serve?c. How many district - sponsored preschool programs do you expect in your district in fall 2015? d. Do your current district - sponsored preschool programs serve children yo unger than age 4, in addition to 4 - ye ar- olds? i. If yes, how many

of the children served by s tate - funded preschool programs are yo unger than age 4? ii. If no, do you expect these programs to serve children younger than age 4 in fall 2015? 3. Do all elementary

s chools in your district offer district - sponsored preschool programs? a. [If no] Would you be able to send me a list of elementary s chools in your distri ct that offer preschool programs? 4. Roughl y what percentage of children enrolled in district - sponsored preschool programs in your district go on to attend public kindergarten in your district? 5. Are any district - sponsored preschool programs located in community-based centers? a. [If ye s] How many district - sponsored preschool programs in your district are located in community-based centers ? 6. By f all 2015, how many district - sponsored preschool programs located in community-based centers do you expect there to be in your district? **B. QUESTIONS ABOUT KINDERGARTEN ASSESSMENT**  Now I have some questions about your district’s kindergarten entry assessment (KEA). 1. According to your state’s RTT - ELC application, you are currently u sing [INSERT NAME OF INSTRUMENT INDICATED BY STATE] as your KEA. Is that correct? a. [IF NOT USING STATE - IDENT IF IED ASSESSMENT] What instrument are you using? i. Which of the following domains does this assessment measure? [Retain in list below only those domains identified in the application; delete others that are not listed. ]  Language and literacy  Cognition and general knowledge (including earl y mathematics and earl y scientific development)  Approaches toward learning  Physical well -being and motor development (including adaptive skills)  Social and emotional developmentii. Are there any other domains that are measured b y the assessment? [Retain in list below only those domains that are **NOT** listed in the application; delete others. ] For example:  Language and literacy  Cognition and general knowledge (including earl y mathematics and earl y scientific development)  Approaches toward learning  Physical well -being and motor development (including adaptive skills)  Social and emotional development  Other (specify) b. Are there separate scores for each domain of school readiness covered by the assessment? c. Was the assessment piloted? I f ye s, when did the pilot take place? d. How long have you been using this assessment? e. Have any notable changes or revisions been made to the assessment since it was first implemented? If ye s, describe the changes or revisions made, the motivation for them, and when they

were made. f. Are there any

plans to change this assessment? Would the change be implemented by fall 2015? i. [If yes] Can you tell me the name of the new assessment you will be using? Will that assessment be state - developed, commercially available, or other (such as a modified or adapted version of a commercially available assessment)? ii. [If ye s] Can you tell me which of the following domains the new assessment will measure? [Encircle all domains identified below. ]  Language and literacy  Cognition and general knowledge (including earl y mathematics and earl y scientific development)  Approaches toward learning  Physical well -being and motor development (including adaptive skills)  Social and emotional development
 Other (specify) 2. Is this assessment mandatory for all kindergarteners in the district? a. [IF MANDATORY] When did it become mandatory? b. [IF NOT MANDATORY] Are there plans to make it mandatory f or all kindergarteners? When will these plans be implemented? c. [IF NOT MANDATORY] How are children selected to take the test?i. Are children randoml y selected to take the test? ii. Do all schools administer the assessment? iii. Do you select a certain number o f kindergarteners from each school? If so, how many? iv. How many kindergarteners take the assessment in your district?  Does the number vary

substantially

from ye ar to year?  How do you determine the number of children who are tested?  For context, about how m any incoming kindergarteners are there in the district each ye ar? d. Under what circumstances would a child be exempted from taking the test? 3. When is the assessment ty picall y administered? [Ask whether fall and/or spring of each ye ar and which months.]  Within the first week of school  Within the first month of school  Be yond the first month, specify : ______________________  Varies b y district or school (specify range): _____________________ 4. What happens if a child misses the assessment administration? 5. Under what circumstances would a child be exempted from taking the assessment? I now have some questions about how the kindergarten assessment data are stored and whether it is possible to link the kindergarten data to data on children’s preschool. We are seeking this information because we need to link student
- level kindergarten assessment data to the TQRIS data of the preschool program the student attended. 6. Are KEA data currentl y s tored in a district database? a. [If yes] In which database are the data stored? b. [If no] Will these data be stored in a district database b y fall 2015? i. [If yes] In which database will the data be stored? 7. Does the district database that includes the kindergarten assessment data contain student ID numbers? a. How/when are students assigned a student ID? b. Does the district keep the same ID number for a student from year to year? i. [If no] Under what circumstances, if any, would a child’s district ID change? 8. Are children assigned student IDnumbers during preschool only i f they attend a district - sponsored preschool program ?9. Would children assigned a district student ID when they enroll in a district sponsored preschool program have the same district ID when they g o on to kindergarten? 10. Are student IDs used b y yo ur district identical to those used by the state? a. [If no] Does the district database also contain a state student ID? 11. If a kindergartener attended a district - sponsored preschool program in the district, would we be able to identify which program they

attended using the district KEA database or another district database? [If another database is identified, note which database. ] a. For example, do any district databases contain data on kindergarteners’ previous school history

(meaning whether the child attended a center -based ECE program and where)? i. Does the district database contain data on all previous schools the child attended or just the last school attended? ii. How is a child’s previous school(s) identified in the database? Isthere a program ID, program name, o r school name in the database? b. Would it be possible to obtain preschool enrollment rosters from school ye ar 2014

– 2015 for all district - sponsored preschool programs in the district? c. If it is currentl y not possible to identify the preschool programs that kindergarteners attended or to obtain preschool enrollment rosters, will it be possible to do so by f all 2015? 12. I now have a question that may b e difficult to address off the top of your head, so we understand if this is something we’ll need to follow up on after the call. Roughl y what percentage of kindergarten students were previousl y enrolled in a district sponsored preschool program in your district? [ This is likely something they won’t know off the top o

f their head, so tell them you will send this question in a follow
- up email and that we’d like to receive the answer within two weeks. ] 13. Does the KEA district database or any other district database contain the following student - level data? If not, will th ese data be available by fall 2015? a. Gender b. Socioeconomic status (for example, free or reduced-price lunch status) c. Race/ethnicity d. English - language - learner status e. Special education status f. Maternal education g. Preschool and kindergarten attendance 14. What is the process for obtaining data from your district?a. Do we need to submit a formal data request? To whom?

b. How many years of KEA data using the current assessment are available? c. In what month do KEA results ty picall y become available each year? d. What is the turnaround time (after submitting a request) for receiving these data? 15. Who should be our main contact person for collecting student - level data (including kindergarten assessment data)? [GET NAME, TITLE, TELEPHONE NUMBER, AND EMAIL.] **C. WRAP**  **- UP**  Thank you fo r this extremely helpful information. [IF THERE ARE OUTSTANDING QUESTIONS] As I indicated, I will follow up with an email listing the questions that are still outstanding. Itw ould be great to obtain your responses to those questions within the next two w eeks. Would that work? In the next three weeks, we will contact the staff involved with the district’s KEA data system to discuss next steps for us to obtain these data. I will also send you a draft memorandum of understanding (MOU) for the district to review and sign. It will summarize what participating in the study entails. Are yo u the best person to contact regarding the MOU and any other data collection requirements? If not, please let me know who I should work with. Are there any other district requirements or necessary s teps that we should be aware of to move forward with the study? [IF DI STRICT SAYS A RESEARCH APPLICATION IS REQUIRED] We have a standardized research application that we plan to use with all districts participating in this study . We are prepared to submit this to your district. To whom should Isend the application? [IF SOMEONE EL SE, GET THAT PERSON’S CONTACT INFORMATION AND ASK FOR AN EMAIL INTRODUCTION.] [AVOID THI S IF AT ALL POSSIBLE, BUT IF DI STRICT INS ISTS THAT THEIR OWN APPLICAT ION BE SUBMITTED] Can you please direct me to the forms we will need to submit? Before we end the call, do you have any questions for me? Thank you so much for your time.**AFTER THE CALL**

MAKE SURE YOU: 1. SEND AN EMAIL (TEMPLATE BELOW) TO THANK THE CONTACT AND CONFIRM ANY NEXT STEPS THAT YOU AGREED TO DURING THE CALL. 2. ENTER INFORMATION FROM YOUR CALL IN SHAREPOINT AND UPDATE PROTOCOLDOCUMENTATION TOOL. TEMPLATE FOR THANK YOU EMAIL (FOLLOWING INTRODUCTORY PHONE CALL) **Subject:** ED Study

of RTT - ELC TQRI S **Atta chment:**  District Memorandum of Understanding (Appendix C) Good [morning/afternoon] [Dr./Mr./Mrs.] _________________, I want to thank you again for speaking with me on [Date] for the U.S. Department of Education study o f Tiered Quality

Rating and Improvement Systems (TQRI S) under the Race to the Top

— Earl y Learning Challenge program. The information you shared will inform TQRI S development and refinement across the nation. I am following up on the next steps that we discussed at the end of our call. [ Adjust remaining email text as appropriate to cover the relevant issues. ] First, please respond to the following questions that we did not get through during our call. As we discussed, it would be helpful if you could send your written responses by _____ (date).

1. [ Question 1 ] 2. [ Question 2 ] 3. [ And so on ] In addition, I have attached a draft memorandum of understanding (MOU) for your district to review and sign. This document summarizes what participating in the study entails. Once the appropriate district personnel hav e had a chance to review the document, please return a signed PDF cop y to me at _______________ (email address). Finally, I have attached a copy o f the [formal] data request which we plan to submit to _____________ (person), according to your instructions, on _________ (date). Please do not hesitate to contact me if you have any questions that I might be able to address in the meantime. Sincerel y, (name) [Include your phone number and email]