**Appendix B**  **: Administrator Interviews Protocol A:**  **(Fall 2015; Spring 2016; Spring 2017)** These questions are from the School

- wide Evaluation Tool (SET) 1 and Individual Students Systems

Evaluation Tool (ISSET) 2 and stay the same at each wave. Date: In terv iewer_I D: District_I D: School_I D: STAFF _I D: **Introduction (Introduction**  **will be repeated each wave )**  Good morning/good afternoon. My name is [name] and Iwork with the Multi
- Tiered Sy stems of Support for Behavior (MTSS
- B) study team. We are conducting i nterviews with school administrators to learn their thoughts about behavior support practices at their school. We anticipate the interview will take about [45 minutes (BAU schools)/ 60 minutes (program schools)]. Before we begin, Iwould like to ask y ou to review this consent form and let me know if you agree to be interviewed and if I

can audio
- record the interview. We record the interviews to ensure that we do not miss any relevant information. All the information we collect is confidential, and y ou, y o ur school, or any one working at or attending your school will not be identified by name. You can ask me to stop recording at any time. ( Review consent form.) Do y ou have any

questions before we begin? **Let’s talk about your discipline system** 1) Do y ou collec t and summarize office discipline referral information? Yes No I f no, skip to #4. 2) What sy stem do y ou use for collecting and summarizing office discipline referrals? (E2) a) What data do y ou collect? __________________ b) Who collects and enters the data?

____________________ 3) What do y ou do with the office disc ipline referral information? (E3 ) a) Who looks at the data? ____________________ b) How often do y ou share it with other staff? ____________________ What ty pe of problems do you expect teachers to refer t o the office rather than handling in the classroom/ specific setting? (D2) 4) What is the procedure for handling extreme emergencies in the building (i.e. stranger with a gun)? (D4)

1 Sugai et al., 2005 2 Anderson et al., 2014.**Let’s talk about your school rules or motto** 5) Do y ou have school rules or a m otto? Yes No I f no, skip to # 10. 6) How many

are there? ______________ 7) What are the rules/motto? 8) What are they

called? 9) Do y ou acknowledge students for doing well socially ? Yes No I f no, skip to # 12. 10) What are the social acknowledgements/ activities/ routines called (student of month, positive referral, letter home, stickers, high 5's)? **Do you have a team that addresses school -**  **wide discipline? If no, skip to # 19** 11) Has the team taught/reviewed the school
- wide program with staff this y ear? Y es No 12) Is y our school
- wide team representative of your school staff? Yes No 13) Are y ou on the team? Yes No 14) How often does the team meet?

__________ 15) Do y ou attend team meetings consistently ? Yes No 16) Who is y our team leader/facilitator? ______

_____________ 17) Does the team provide updates to faculty on activities & data summaries? Yes No If y es, how often? ______________________ 18) Do y ou have an out
- of
- school liaison in the state or district to support y ou on positive behavior support sy stems developme nt? Yes No If y es, who? ___________________ 19) What are y our top 3 school improvement goals? 20) Does the school budget contain an allocated amount of money for building and maintaining scho ol
- wide behavioral support? Yes No **Let’s talk abou**  **t your team for targeted and individualized supports**  **.** 2 1 ) Do y ou have a team of two or more people at your school that receives requests for assistance, develops support plans and mon itors intervention results? Yes No if no, skip to question 2 Ho w often does this team meet? _______________How often are y ou able to attend those meetings? (A1) _____________

(0= 0

- 24%, 1= 25%

- 49% of meetings, 2= 50% or more ) 2 2 ) I s there someone identified as the coordinator of function
- based support for all the students in y our school? Yes No If y es, is that person on y our school staff? Yes No I f y es, who is it? ______________________ 23 ) I s there a person(s) on, or available to, the school behavior support team who is trained to conduct functional behavioral assessments and lead a team in the use of functional behavioral assessment information to deve lop behavior support plans? Yes No 24 ) I s the number of students receiving targeted or intensive interventions and their overall pro gress reported to faculty ? Yes No If y es, how often? _____________ **Before ending the administrator interview check the list of permanent products on page 4 and collect and/or review any that he/she has available.** **Protocol B :**  **Administrator of Program Schools Supplem ent (Fall 2015)**  I would now like to ask y ou a few questions about the readiness activities that took place last spring and early summer to establish the infrastructure to implement MTSS
- B in y our school. 1. As part of the readiness activities last spring, yo ur school established a School L eadership Team (SL T). Did y ou find the CSBS trainer helpful in the process of creating the SL T? a. If **Yes,** then ask: How did he/she help you or the school staff to establish the team? [Probe: For example, did y ou see a webinar about the MTSS
- B study in general and about the role of the SL T? Did y ou discuss the roles and functions of the team members with the CSBS trainer?] b. If **No,** then ask: What kind of additional support would have been helpful in establishing the SL T? 2. Did y o ur school modify the office discipline referral process and forms last spring? a. If **Yes,** then ask: Did the CSBS trainer provide support for modifying the process? For example, did the CSBS trainer: i) provide you with an example ODR form, and ii) explain why the forms needed to be changed? i. If **Yes,** then ask: Was the support helpful? Can y ou give examples? Can y ou think of any additional supports that could have been helpful? ii. If **No,** then ask: What kind of support would have been helpful for your school staff? b. If **No,** then ask: Could y ou explain briefly why the office discipline referral process was not changed? For example, did the CSBS trainer agree that the process was close enough to the process they

usually recommend? 3. Your school was provided with the SWI S behavior monitoring sy stem to document and monitor student behavior. How did y ou find the process of getting the sy stem set up for y our staff? 4. Did the CSBS trainer support y ou and/or y our staff getting started with SWI S? a. If **Yes,** then ask: Was the support helpful? Can y ou give examples? [Probe: For example, did he/she get y ou a copy

of the licensing agreement? Answer questions about the sy stem? Help yo u identify

appropriate staff for SWI S training? Make sure that staff could log into the sy stem?] Are there any

additional supports that could have been helpful? b. If **No,** then ask: What kind of support would have been helpful for your school staff? 5. How many

people from y our school participated in the trainings for the Schoolwide Information Sy stem (SWI S)?6. These school staff were trained by CSBS in the use of SWI S. In y our opinion, how useful was the training? How do y ou know it was useful? [Probe: Was the training long enough and covering enough content? Was the training hands on with real examples using the software?] 7. Iny our opinion, do the school staff feel ready to use the SWI S sy stem at this stage? a. If **Yes,** then ask: How do y ou know? Can y ou give examples? [Probe: For example, have yo u or y our staff started to use the sy stem? I s someone assigned to enter ODRs ? How about someone assigned to regularly print out reports? Are y ou or y our staff using or planning to use them during SL T or other meetings?] Can y ou think of any other supports that could have been helpful? b. If **No,** then ask: What kind of support would have been helpful for y our school staff? 8. Has y our school contacted CSBS for additional support regarding SWI S? a. If **Yes,** then ask: On what topics? Why was extra support needed? Was the extra support helpful? Can y ou give examples? [Probe: For example, did th e trainer provide responses in a timely manner? Answer questions about the sy stem? Was the trainer knowledgeable about the sy stem? (For example, did the trainer explain how to change passwords? How to locate additional resources? How to export and import d ata to the sy stem?)] Can y ou think of any other supports that could have been helpful? b. If **No,** then ask: I s there a kind of support that would have been helpful to your school?”? 9. As a school implementing the MTSS
- B program in this study , y our school was as signed an MTSS
B coach. Do y ou know who the assigned MTSS
- B coach is for y our school? a. If **Yes,** then ask: What is the name of the MTSS
- B coach? b. If **No,** say : OK

— we will make sure the coach comes to your office next time he/she is around. 10. What is the role of th e MTSS
- B coach in y our school? [Probe: What do y ou think the coach is supposed to do?] 11. How did the CSBS trainer explain the role and function of the MTSS
- B coach? Was the CSBS trainer helpful in explaining the role and function of the MTSS
- B coach? a. If **Ye s,**  then ask: Was the support helpful? Can y ou give examples? [Probe: Did y ou receive information about the role of the MTSS
- B coach? Do y ou know how often the coach will be in y our school? Were y ou given information about areas in which the coach will prov ide support to y our teams?] b. If **No,** then ask: What kind of support would have been helpful for your school staff? 12. Were there any

other ty pes of support that y ou or y our school staff received from the CSBS trainer before the summer trainings took place?a. If **Yes,** then ask: What ty pe of support? Was the support helpful? Can y ou give examples? [Probe: For example, technical assistance via email? Phone calls?] Can y ou think of any other supports that could have been helpful? b. If **No,** then ask: What kind of additio nal support would have been helpful for your school staff? 13. You, y our school’s SL T and the MTSS
- B coach were trained during the summer by CSBS. As part of this, y ou and y our school’s SL T and coach were expected to start creating your school plan for implem enting MTSS
- B. Can y ou tell me about the process of creating and finalizing the plan? Did the CSBS trainer help y ou set up a process, after the summer trainings to finalize the school’s implementation plan and to prepare the school for the kick
- off meeti ng? a. If **Yes,** then ask: Was the support helpful? Can y ou give examples? [Probe: For example, technical assistance via email? Phone calls?] Can y ou think of any other supports that could have been helpful? b. If **No,** then ask: What kind of additional support wou ld have been helpful for your school staff? 14. Iny our opinion, are you and y our staff ready to implement MTSS
- B Tier I ? a. If no, then ask: Why

do y ou not feel ready ? What challenges do y ou anticipate facing? What additional support would have been helpful for yo ur school staff. These were all the questions Ihad. I s there any thing else yo u would like to share with me about the support yo u have received for implementing the MTSS
- B program? Thank y ou so much for y our time. **Protocol C :**  **Administrator of Program School Supplement (Spring 2016)**  I would like to ask y ou a few questions about the support activities that have taken place over the last few months to implement MTSS
- B in y our school. Iwould especially like to learn about the activities the CSBS trainer engaged in **over the last two months** and their usefulness for MTSS
- B implementation in your school. Iwould like to ask a set of questions about the kind of activities the trainer could be engaged in. Are yo u ready ? 1. Most schools participating in the study were encouraged to change their rewards and consequences sy stem regarding student behavior and discipline. Was that true for you? Did y ou change the sy stem in y our school? a. If **Yes,** then ask: Can you briefly describe how the sy stem changed? Did the CSBS trainer help y ou to modify

the sy stem? i. If **Yes,** then ask: What kind of supports did he/she provide? Can y ou give examples? [Probe: Did the trainer discuss the benefits or reasoning behind having positive reinforcement sy stems in place and changing the focu s from negative consequencesto positive reinforcements? Did the trainer provide examples of sy stems other schools have established? Technical assistance via email or phone?] Can you think of any other supports that could have been helpful? ii. If **No,** then ask : What kind of additional support would have been helpful for your school staff? b. If **No,** then ask: What were the reasons for not modifying y our current rewards/consequences sy stem? [Probe: Did the current sy stem already align with the MTSS
- B approach? Did y ou think that changing the sy stem was unnecessary ?] 2. Has the CSBS trainer conducted webinars about the classroom components **over the**  **last two**  **months** ? a. If **Yes,** then ask: Who participated in the webinars? Do you think y our staff and teachers found the informat ion in the webinars useful? Do y ou think they are using the specific strategies and tools? How do you know? Can y ou think of any other supports that could have been helpful? b. If **No,** then ask: Do y ou know why the webinars did not occur? 3. Has the CSBS provider conducted any on - site trainings in y our school **over the last two months** ? a. If **Yes,** then ask: Who participated in the trainings? What was the content of the trainings? Do y ou think y our staff and teachers found the trainings useful? How do you know? Do y ou think they are using the specific strategies and tools covered in the trainings? Can you think of any other supports that could have been helpful? b. If **No,** then ask: Do y ou know why the trainings did not take place? 4. Has the CSBS trainer specifically addressed MTSS
- B implementation in classrooms during the site visits **over the last two months** ? a. If **Yes,** then ask: Can you provide some examples? For example, did the trainer conduct a training on the Positive Behavior Game? Or did the trainer conduct classroom walkthroughs? b. If **No,** move on to the next question. 5. Did the CSBS trainer help with analy zing student data, and did he/she use the data to help modify the school’s implementation plan? a. If **Yes,** then ask: Who did the CSBS trainer work with to help with the analy sis of student data from SWI S? Do y ou think y our staff and teachers found the support from the CSBS trainer useful? How do you know? Do y ou think they are applying the skills they learned in analy zing student data and reports from SWI S? Can y ou think of any o ther supports that could have been helpful?b. If **No,** then ask: Do y ou know why the CSBS trainer has not been helping staff to analy ze student data? 6. Did the CSBS trainer address any specific problems that the school experienced with implementation (for exampl e, a specific classroom struggling with behavior; a specific area in the school, such as the cafeteria)? a. If **Yes,** then ask: Can you provide some examples? b. If **No,** move on to the next question. 7. Was the CSBS trainer helpful in structuring and supporting facul ty meetings where MTSS
- B implementation was presented? a. If **Yes,** then ask: Can you provide some examples? (For example, did he/she help with student data, preparing slides, and so on?) b. If **No,** move on to the next question 8. As a school implementing the MTSS
- B program in this study , y our school was assigned an MTSS
B coach. In y our opinion, is the MTSS
- B coach successfully integrating him/herself into the school? For example, do y ou believe teachers find the coach knowledgeable and a valuable source of support ? 9. Has the MTSS
- B coach supported the SLT and school staff in implementing the MTSS
- B program? a. If **Yes** , then ask: Has the coach been helpful? Can you give examples of the activities the MTSS
- B coach has engaged in? [Probe: For example, has the MTSS
- B coach attended or participated in the SL T meetings? Provided one
- on - one coaching or small group trainings to the school staff? Helped with analy zing student behavior data?] Can y ou think of any other supports that could have been helpful? b. If **No** , then ask: What k ind of support would have been helpful for your school staff? These were all the questions I had. I s there any

thing else y ou would like to share with me about the support yo u have received for implementing the MTSS
- B program? Thank y ou so much for y our tim e. **Protocol D :**  **Administrator of Program School Supplement (Spring 2017)** I would like to ask y ou a few questions about the support activities that have taken place over the last few months to implement MTSS
- B in y our school. Iwould especially like to learn about the activities the CSBS trainer engaged in **over the last two months** and their usefulness for MTSS
- B implementation in your school. Iwould like to ask a set of questions about the kind of activities the trainer could be engaged in. Are yo u ready ? 1. Has the CSBS provider conducted any trainings in y our school **over the last two months** ?a. If **Yes,** then ask: Who participated in the trainings? What was the content of the trainings?

Do y ou think y our staff and teachers found the trainings useful? How do yo u know? Do y ou think they are using the specific strategies and tools covered in the trainings? Can you think of any other supports or training that could have been helpful? b. If **No,** then ask: Do y ou know why the trainings did not take place? 2. Has the CSBS tr ainer specifically addressed MTSS
- B implementation in classrooms during the site visits **over the last two months** ? a. If **Yes,** then ask: Can you provide some examples? For example, did the trainer conduct retraining or a booster session on the Positive Behavior Game? Or did the trainer conduct classroom walkthroughs? b. If **No,** move on to the next question. 3. Did the CSBS trainer help with analy zing student data, and did he/she use the data to help modify the school’s implementation plan? a. If **Yes,** then ask: Did the tra iner help both with SWI S and the SWI S [CI CO] component? Who did the CSBS trainer work with to help with the analy sis of student data from SWI S and/or the SWI S [CI CO] component? Do y ou think y our staff and teachers found the support from the CSBS trainer us eful? How do y ou know? Do y ou think they are applying the skills they learned in analy

zing student data and reports from SWI S? Can y ou think of any other supports that could have been helpful? b. If **No,** then ask: Do y ou know why the CSBS trainer has not been helping the staff to analy ze student data? 4. Did the CSBS trainer address specific problems that the school experienced with the implementation (e.g., a specific classroom struggling with behavior; a certain area in the school, such as the cafeteria)? a. If **Yes ,**  then ask: Can you provide some examples? b. If **No,** move on to the next question. 5. Has the CSBS trainer specifically addressed MTSS
- B implementation in classrooms during the site visits? a. If **Yes,** then ask: Can you provide some examples? For example, did the tr ainer conduct retraining or a booster session on the Positive Behavior Game? Or did the trainer conduct classroom walkthroughs? b. If **No,** move on to the next question. 6. Was the CSBS trainer helpful during the faculty meetings where the MTSS
- B implementation wa s presented?a. If **Yes,** then ask: Can you provide some examples? (For example, did the trainer help with

student data, preparing slides, and so on?) b. If **No,** move on to the next question. 7. Has the CSBS provider worked with the Targeted Team regarding implementation of [CI CO]? a. If **Yes,** then ask: What kind of supports did she/he provide? Can y ou give examples? [Probe: For example, has the provider helped the team to identify students who are not responsive to Tier Iand who would benefit from (CI CO)? Helped the te am to create behavior expectations for students? Modeled giving feedback to students?] b. If **No,** then ask: What kind of additional support would have been helpful for your school staff? 8. As a school implementing the MTSS
- B program in this study , y our school wa s assigned an MTSS
- B coach. In y our opinion, is the MTSS
- B coach successfully integrating him/herself into the school? For example, do y ou believe teachers find the coach knowledgeable and a valuable source of support? 9. Has the MTSS
- B coach been supportin g the SLT, the Targeted Team, and the school staff in implementing the MTSS
- B program? a. If **Yes,** then ask: Has the coach been helpful? Can you give examples of the activities the MTSS
- B coach has engaged in? [Probe: For example, has the MTSS
- B coach attende d or participated in the SL T meetings? Provided one
- on - one coaching or small group trainings to the school staff? Helped with analy zing student behavior data? Helped the Targeted Team with day
- to - day

implementation of (CI CO)?] Can y ou think of any other su pports that could have been helpful? b. If **No,** then ask: What kind of support would have been helpful for your school staff? 10. Iny our opinion, how is the MTSS
- B program implementation going in your school? [Probe: Are all elements implemented consistently ? (SL T meets regularly , SWI S and behavior data are used for decision making), Do most teachers use the Positive Behavior Game in their classrooms? Are staff being supported by

the MTSS
- B coach?) I s it different as compared to last y ear? 11. What barriers or obsta cles do y ou believe have hindered implementation of the MTSS
- B program in y our school? Was the CSBS trainer and/or the MTSS
- B coach helpful in addressing these obstacles? Have there been specific features about your school that have helped in the implemen tation of the MTSS
- B? 12. What are y our plans regarding MTSS
- B implementation next y ear? These were all the questions I had. I s there any

thing else y ou would like to tell me about the support you have received for implementing the MTSS
- B program? Thank y ou s o much for y our time.