Scoring Guide DCA Item: 2 points 1 point 0 points Data Source There is a District Implementation Team (DIT) to support implementation of Effective Innovations (EI) A team is developed and is Representative of the district (e.g., K-12) Of functional size A team is developed and representative of the district -HOWEVER- The size of the team is not functional (e.g., too or too small) to effectively accomplish work There is not a team -OR- Team composition is not representative of the district List of team members, roles, and job titles DIT includes someone with executive leadership authority DIT includes someone with executive leadership authority to approve and support team decisions (e.g., adequate funding, resource allocation, Information Technology - IT support, and positions) -AND- Attendance at meetings is regular -AND- When scheduling conflicts occur, the leader makes sure (s)he is provided with relevant information (decisions and potential barriers that need to be addressed by other district leaders) within 1-2 days after the meeting DIT includes someone who has executive leadership authority to approve and support team decisions -AND- Attendance at meetings is regular -HOWEVER- When scheduling conflicts occur there is not a mechanism for the leader to be provided with relevant information within 1-2 days after the meeting There is no one with executive leadership authority represented on the DIT -OR- The executive leader’s attendance at meetings is infrequent Executive leader job description List of team members, roles, and job titles Linking communication protocol DIT includes an identified Coordinator assumes a lead role in preparing for and DIT includes a designated coordinator DIT does not include a designated coordinator Coordinator job description
updates shortly following the meeting -AND- Team documents and completes assignments outlined on an action plan within designated timelines -OR- Assignments are inconsistently completed within the designated timelines District outlines a formal procedure for selecting EIs through the use of guidance documents Guidance documents and formal procedures are in place -AND- Procedure to select an EI includes an analysis of the following variables: Need for the EI; Fit and alignment with other EIs/initiatives/programs; Resources needed to fully implement; Evidence to demonstrate effectiveness; Maturity of the EI; Capacity within the district to successfully implement the EI (e.g. Hexagon Tool) -AND- Procedure is consistently used A formal procedure is in place -BUT- The procedure to select an EI includes an analysis of only some (at least half) of the following variables: Need; Fit; Resources; Evidence; Maturity of the EI; Capacity to implement -OR- The procedure is not consistently used No formal procedure is in place -OR- The procedure to select an EI includes only one or two of the following variables: Need; Fit; Resources; Evidence; Maturity of the EI; Capacity to implement Guidance documents Documentation showing how the procedure has been used within the past 2 years District documents how current EIs link together Documentation displays new and existing EIs the district supports -AND- Documentation displays the new and existing EIs the district is supporting -BUT- is no documentation of how new and existing EIs are compatible -OR- Document displaying how all EIs are linked or compatible
Documentation includes statements regarding how all EIs are compatible and add value to one another to achieve improved implementation and student outcomes It is unclear how the initiatives/practices are compatible and add value to one another Documentation was once created but has not been updated in the past 2-3 years, making it obsolete Funds are available to support the implementation of the EI There is evidence of commitment to sustain funding for on-going implementation and scale-up of the selected EI is evidence of commitment to funding for the EI for a minimum of one year or less There is no commitment to funding the EI Professional Learning budget allocations Grant budget allocations DIT has an implementation plan for the EI The plan is updated as needed using: Fidelity data Student outcome data Capacity data (e.g., Organization, Competency, Leadership data) Scale up data (e.g., each school’s stage of implementation) -AND- The plan’s are S.M.A.R.T. and include strategies/activities to achieve the goals -AND- The plan has been approved by executive leadership The plan is developed and focuses most heavily on: Fidelity data Student outcome data -AND- The plan has been approved by executive leadership -BUT- The plan is lacking in strategies to address: Capacity data Scale up data -OR- The plan includes only goals to implement the EI, not S.M.A.R.T. goals and strategies/activities -OR- There is not a plan -OR- District has no goal(s) to implement the EI -OR- The plan focuses primarily on a training plan for the EI but to encompass a minimum of two of the following: Fidelity data Student outcome data Capacity data up data -OR- The plan has not been approved by executive leadership District implementation plan Record of approval (meeting minutes or other written communication, signature)
-AND- The plan for implementing the EI is integrated into the district’s continuous improvement planning process The plan has not yet been fully integrated into the continuous improvement planning process but the intent is to do so DIT actively monitors implementation of the plan DIT monitors implementation of the plan a minimum of three times per year -AND- Monitoring includes documentation of: Completion status of activities Reasons activities were not completed (e.g. insufficient funding, training) Team decisions (e.g., provide required resources to complete activities, next steps with communication of barriers) DIT monitors the plan three times per year -HOWEVER- Monitoring only includes documentation of: Completion status of activities Reasons activities were not completed (e.g. insufficient funding, training) DIT monitors the plan less than three times per year Documentation of monitoring District utilizes a communication plan The plan is written and accessible to all staff -AND- The plan includes all of the following components: List of stakeholder groups identified in the district’s organizational (e.g., outside agencies, families) The plan is in the process of being written and accessible to all staff -AND- Currently, communication is informally happening and/or is dependent on one main person -OR- There is not a plan for communication -OR- Stakeholders are reporting communication to be ineffective Communication plan Stakeholder report summaries indicating communication has been effective
Type of information to share and receive from identified stakeholders Who is responsible for communication with each group Frequency and methods of communication Plan to evaluate communication method and data at least annually -AND- Stakeholders report the communication has been effective The plan focuses primarily on following components: List of stakeholder groups identified in the district’s organizational chart (e.g., outside agencies, families) Who is responsible for communication with each group Frequency and methods of communication District uses a process for addressing internal barriers A formal process is in place (e.g., specific documents and steps) -AND- The process is consistently used to remove internal barriers (e.g., policy and guidance documents revised to support new ways of work, resources are The process is informal -OR- The process is used inconsistently across all situations that would warrant use is not a process -OR- The process is not used for addressing internal barriers preventing successful implementation of the EI Guidance document outlining process Documentation showing how the process has been used in the past six months (e.g., examples of identifying a barrier, defining a solution, and
implementing the solution with effect) District uses a process to report policy relevant information to outside entities A formal process is in to report policy relevant information (e.g., state/federal laws, mandated use of funds, bargaining agreements) to regional units, state department of education, etc. -AND- The process is consistently used for reporting to outside entities The process is informal -OR- The process is used inconsistently across all situations that warrant use There is not a process -OR- The process is not used for reporting policy-relevant information to the regional unit or state department Guidance document outlining process Evidence of use DIT supports the use of a fidelity measure for implementation of the EI DIT supports schools to use a research validated fidelity measure as recommended that is highly correlated with (i.e., predictive of) intended outcomes for the EI DIT supports schools to use a fidelity measure for the EI as recommended, but the measure is currently in development (i.e., not yet correlated with outcomes or research validated) -OR- District has developed practice profiles to operationalize the EI for use in developing a fidelity measure DIT does not support schools to use any fidelity measures for the EI -OR- DIT does not support schools to use the fidelity measure as recommended (e.g., frequency, audience) Fidelity measure or practice profile Data (e.g., local or published) demonstrating that fidelity predicts intended outcomes DIT has access to data for the EI All of the following data are accessible for the DIT to analyze: Fidelity data The DIT only has access to at least two of the following types of data, but not all types: No data are accessible -OR- Sample data reports
Capacity data for the EI are used to enhance leadership, organizational or competency supports Scale-up data are used to create differentiated plans for schools based on their current stage of implementation District provides a status report on the EI to the school board The report includes at five of the following seven types of information: Number of schools across the district working to implement the EI Each school’s stage of implementation Internal capacity to develop structures to support the EI (leadership, organization, competency) Fidelity of implementation for the EI Impact of the EI on student outcomes Stakeholder information (e.g., survey data from staff and parents) about implementation of the EI The report includes less than five of the different of information outlined in the criteria A status report has never been provided to the school board -OR- Report focuses only on action, not on data Copy of most recent school board status report
Upcoming work to scale-up the EI and continue improving -AND- At minimum twice a year Building Implementation Teams (BITs) are developed and functioning to support implementation of EI Every school in the district has a BIT -AND- BITs overlap as much as possible (e.g., one or more members) with the school improvement team -AND- DIT supports BITs (e.g., provides training, coaching, etc.) Some, but not all, schools in the district have a BIT -OR- BITs do not strategically overlap with the school improvement team -OR- BITs do not have the necessary supports from DIT None of the schools in the district have a BIT List of BIT members List of school improvement team members Linking communication protocol BIT implementation plans are linked to district improvement plan 80% or more of schools with BITs have implementation plans linked to the district priorities within the district improvement plan At least half of the BITs have implementation plans that are linked to the district priorities within the district improvement plan BITs do not have implementation that are linked to the district priorities within the district improvement plan School level plan BITs have a process for using data for decision making BITs use a specific model -AND- All data listed below are used in the following ways: Fidelity data are analyzed to improve implementation supports such as selection, training BITS use a specific model -AND- The BIT primarily uses a combination of two of the three sources of data: Fidelity data Student outcome data relying mostly on BIT does not use a specific problem-solving model -OR- BIT chooses to primarily use annual summative assessment data (e.g., state test) to analyze student outcomes Evidence of the problem-solving process Analysis of action plans and updated improvement plans based on
and coaching to ensure the EI is implemented as intended Student outcome data (screening, progress monitoring, and summative assessment/state test) are used to determine the impact of the EI Capacity data are used to develop structures to support the EI (leadership, organization, competency) screening data but not consistently using other measures like progress monitoring data and summative assessment data Capacity data are used to develop structures to support the EI (leadership, organization, competency) analysis of the data District uses a process for selecting staff (internal and/or external) who will implement and support the EI Job descriptions with the function of positions required to support the EI -AND- Job interview protocol includes documentation and assessment of core skills needed to implement the EI -AND- Interview protocol includes specific procedures for assessing candidate capacity to perform key skills (e.g., work task, role play) and use feedback provided during the interview to improve Job descriptions exist and include general descriptions that may align with competencies needed to implement the EI -OR- Interview and selection protocols exist but do not include documentation and assessment of core skills or demonstrated ability to perform skills in simulated activity during the interview -OR- Interview protocol is refined and revised less than annually Job descriptions exist but do not align with competencies needed to implement the EI -OR- Generic job interview protocol (e.g. similar protocol for any position) exists in the district Job descriptions Interview protocol (including procedures used during the selection process)
-AND- All staff have opportunities to receive training as outlined in their professional learning plans All staff do not have opportunities to execute a professional learning plan. Plans are limited to either new staff or staff who are relatively new in their positions (e.g., non-tenured teachers) accurately implement/support the EI Sample of staff professional learning plans DIT uses training effectiveness data Training evaluation data (e.g., pre-post of knowledge/skills, observations) and training performance assessment data (e.g., schedule, content, process) are analyzed to determine effectiveness of training (initial and -AND- Training effectiveness data are utilized to inform needs in selection/recruitment, coaching, and other implementation supports Training evaluation data are primarily analyzed to determine the effectiveness of training (initial and on-going) -OR- Training effectiveness data are only utilized to inform improvements to the training content and delivery Data are not analyzed to determine effectiveness of training Training outcome data Evidence that data are used for improvements DIT uses a coaching service delivery plan Coaching service delivery plan for the EI includes a combination of: Direct observation Prompting Modeling Feedback Assistance in adaptation of EI to local context The plan only includes consultation without direct observation -OR- Coaching service delivery plan developed but is not current (over a year old) No coaching service delivery plan exists -OR- The coaching service delivery plan is not being implemented Sample of coaching service delivery plans
Consultation without direct observation -AND- Adherence to the coaching service delivery plan is regularly reviewed DIT uses coaching effectiveness data Coaching effectiveness is assessed at least every 6 months, using multiple sources of data including: Fidelity measures observations Staff satisfaction surveys (coaching recipients, coach, other stakeholders) Coaching service delivery adherence data -AND- Coaching effectiveness data are utilized to inform improvements in coaching, selection/recruitment, training, and other implementation supports Coaching effectiveness is assessed annually and multiple sources of data are used to improve coaching -OR- Coaching effectiveness data are only utilized to inform coaching improvements Coaching effectiveness is not assessed using multiple sources of information Coaching effectiveness data such as staff satisfaction surveys Evidence the data are used to inform improvements Staff performance feedback is Performance feedback (e.g., fidelity) process is in place to provide consistent feedback to all staff who are implementing The process for performance feedback related to implementing the EI is either in development or partially in (e.g., process is in place No process is in for providing performance feedback to staff implementing or supporting the EI Evidence of performance feedback process