Ameri can Institutes for Research Task Order 18: Study on Sustaining the Positive Effects of Preschool

—I

- 1 1695_03/15 

## Appendix **I .**  **Evaluator Protocol**Ameri can Institutes for Research Task Order 18: Study on Sustaining the Positive Effects of Preschool

—I

- 2 

# Study on **Sustaining the Positive Effects of Preschool**  **Evaluator Protocol** Interviewer_____________________________________ Interviewee ID #_________________ Site __________________________________________ Date/Time_____________________ **Introduction** My name is _____

_, and I work for American Institutes for Research (AI R). We’ve been contracted b y the U.S. Department of Education to conduct case studies on sites that are implementing particular programs to support early elementary students. As y ou may

know, the purpo se of the stud y is to document characteristics of programs and policies in earl y elementary school that support student growth and development, such as PK

– 3 alignment, differentiated instruction, and other programs designed to sustain the advantages of pre school. We are interested in learning more about the theoretical or practical background of these programs, how they are implemented, how they are sustained and funded, and their outcomes. I want to reassure you that y our responses to this interview will be used only for research purposes . Though your site ’s name will appear named in the final report, we will not provide yo ur name in any

reporting or to any one outside the study team, except as required by law. Because we are producing case studies of onl y a small number of named sites, it may be possible for a reader to identify

y ou based on the data that we report, even though your name will not be used. If there is information that y ou do not want shared directl y in an y reporting, please let me know. I wi ll now give y ou a consent form that provides more details about the study and your rights as a participant. Do y ou have an y questions about the study ? I plan to audio record this session, solely for our note

- taking purposes. Only research staff will use th e audio recording, and we’ll destro y it when the project is done. Is that okay with y ou? **Role and Perspective** 1. I’d like to start b y asking you to tell me a bit about yourself. I understand that you are the evaluator of [PROGRAM]. Briefl y, can you tell me what your role has been in relation to [PROGRAM]? a. How did y ou come to be involved with [PROGRAM]? b. What related experiences did y ou have before you took on this role? 2. Can y ou tell me more about the organization that y ou work for and about the kind of work they t ypically engage in?Ameri can Institutes for Research Task Order 18: Study on Sustaining the Positive Effects of Preschool

—I

- 3

3. Why did [ORGANIZATION] become interested in partnering with [or evaluating] [PROGRAM] ? a. Can y ou describe the sequence of [PROGRAM’s] initiation and the [ORGANIZATION’s] involvement? **Evaluation Questions** 4. When did y our work evaluating [PROGRAM/STRATEGY] begin? 5. Can y ou tell me about the basic study design of the evaluation? 6. What ty

pes of info rmation have y ou collected over the course of the evaluation work? a. Why did y ou select this information to collect? 7. How does the evaluation measure students’ progress? 8. Has the evaluation documented gains in student outcomes in kindergarten through third grade? If interviewee reports effect: a. For which outcomes have y ou seen gains? b. How are these gains measured or documented? c. Have these gains been consistent over time? d. Can y ou share or point us to the data or report illustrating these patterns? If no effect : a. What do y ou think are the barriers to seeing more student gains? 9. Has the evaluation documented changes in teacher practices? Teacher turnover? If yes: a. Have you seen an y patterns? b. Can y ou share any

data with us? If no: a. What do y ou think are the barriers to seeing changes in teacher practice? 10. What else have you learned from the evaluation so far? 11. How have you shared the results of your evaluation with district and school staff? 12. In what way s and for how long will y ou continue evaluating the progr am? 13. Do y ou expect y our work to help the district improve the program? I f so, how?Ameri can Institutes for Research Task Order 18: Study on Sustaining the Positive Effects of Preschool

—I

- 4

**Success, Challenges, and Closing** 14. What challenges have yo u encountered working with a district on this ty pe of initiative? How have you overcome those challenges? 15. Overall, what is working well with [PROGRAM]? How do y ou know? 16. What aspect of [PROGRAM] needs strengthening? Why ? 17. What challenges have yo u noticed in the district(s) and schools in implementing [PROGRAM], and what steps may be needed to address them? 18. [We already

have (XX DOCUMENT)] OR [We do not have any written documents on (PROGRAM)]. Are there [other] written resources that document the program and its evaluation? a. Could we get a cop y of those? 19. Those are all the questions Ihave. Are there any questions that Ishou ld ask about [PROGRAM] that Ididn’t? I s there an ything else y ou’d like to share with me today in regard to [PROGRAM]?