# Evaluati **on of the Comprehensive**

**Technical Assistance Centers OMB Clearance Request for Data Collection Instruments**  **Part B: Supporting Statement for Paperwork Reduction Act Submission** 



## September **18 , 2014** 

**Prepared for :**  U.S. Department of Education Contract No. ED
- IES
- 13
- C
-0059 **Prepared by:** IMPAQ International1 

### INTRODUCTION This document has been prepared to support the clearance of data collection instruments for the National Evaluation of the Comprehensive Technical Assistance Centers. The Institute of Education Sciences (IES) within the U.S Department of Education (ED) is conducting this evaluatio n. In the introduction to the supportin g statement, we provide a description of the Comprehensive Technical Assistance Centers program, the evaluation questions and study design. The remaining sections of this document respond to specific instructions of the Office of Management and Budget (OMB ) for the preparation of a supporting statement
. 

### The Comprehensive Technical Assistance Centers Title II of the Educational Technical Assistance Act of 2002 ( F.T AA, Section 203) authorized t he Comprehensive Center (CC) Program, a discretionary grant program establishing technical assistance centers. The CCs were last awarded in 2012, to provide technical assistance to “ help State Education Agencies (SEAs) build their capacity to implement State
- level initiatives and to support district

- a nd school

- level in itiatives that improve educational outcomes for all students, close achievement gaps, and improve the quality of instruction” ( 77 FR 33564 ). In 2012, the Department of Education awarded new five

-year grants to 15 Regional Centers and 7 Content Centers u nder the Comprehensive Centers program . The R egional Centers each serve one to seven U.S. states, territories, and possessions. They provide technical assistance (TA) that builds the capacity of SEAs to implement, support, scale up, and sustain initiatives that help districts and schools improve student outcomes. The Regional Centers focus their work on seven Federal priority areas: 1. Implementing college

- and career

- ready standards and aligned, high

- quality assessments for all students; 2. Identifying, recruit ing, developing, and retaining highly effective teachers and leaders; 3. Turning around the lowest

- performing schools; 4. Ensuring the school readiness and success of preschool

- age children and their successful transition to kindergarten; 5. Building rigorous instr uctional pathways that support the successful transition of all students from secondary education to college without the need for remediation, and careers; 6. Identifying and scaling up innovative approaches to teaching and learning that significantly improve student outcomes; and 7. Using data

- based decision

- making to improve instructional practices, policies, and student outcomes. The Content Centers provide the Regional Centers and SEAs with in

- depth content knowledge and expertise by providing information, pu blications, tools, and specialized technical assistance. The 7 Content Centers are: 1. Center on Standards and Assessments Implementation 2. Center on Great Teachers and Leaders2

3. Center on School Turnaround 4. Center on Enhancing Early Learning Outcomes 5. Center on Co llege and Career Readiness and Success 6. Center on Building State Capacity and Productivity 7. Center on Innovations in Learning 

### The National Evaluation of the Comprehensive Technical Assistance Centers The National Evaluation is charged with examining and documenting how the individual CCs intend to build SEA capacity (theories of action) and what types of activities they actually conduct to build capacity. It will also explore and document the outcomes of the capacity building efforts. It is designed to build on the previous evaluation of the CCs, which documented the type, extent, and quality of services provided. 

### Evaluation Questions The evaluation will address broad questions in three areas:

 **Progra m Design: How did the C** **C s design their work?**  1. How did CCs define capacity building? Did their definitions change over time? If so, how? 2. What theories of action did CCs use to guide their general capacity

- building work? Did the theories change over time? If so, how? 3. How did CCs assess the needs of their constituencies? In addressing th e s e evaluation question s, we seek to identify how the CCs designed their work as TA providers, including the underlying theories of action driving the work. The evaluation will seek to surface the theories of action and definitions of capacity building employed by CCs, as well as describe CCs’ plans for assessing the needs of their constituencies and developing TA work plans to address those needs.  **Program Implementation: How did the C** **C s operate?**  4. What strategies did CCs employ to achieve their outcomes? 5. To what extent did CCs implement technical assistance to their constituencies as planned? 6. To what extent and how did CCs collaborate with each other? In addressing th e s e evalua tion question s, we seek to identify the various strategies CCs used to build capacity, describe the characteristics of those strategies, and document the extent to which CCs implemented the TA as planned. We will also seek to identify common challenges and barriers CCs faced in building capacity, and ways they met those challenges.  **Program Outcomes: What was the result of the C C** **s’ work?** 7. To what extent did CCs achieve their goals and objectives? In addressing this evaluation question, we seek to identify the extent to which CCs achieved their goals and objectives, particularly as they relate to building their constituents’ capacity. We will also explore the extent to which outcomes aligned with and supported the CCs’ theories of action, and to identify factors that may have contributed to CCs’ success (or failure) in achieving expected outcomes.3

As a way to focus the evaluation to gather data in depth rather than breadth, the evaluation will limit data collection on the implementation and outcomes questions to two of the seven federal priority areas: identifying, recruiting, developing, and retaining highly effective teachers and leaders, and ensuring the school readiness and success of preschool

- age children and their successful transition to kindergarten . All of the Regional Centers have implemented projects and strategies that address the effective teachers and leaders priority, so it clearly represents an issue of nationwide focus. It is an area where most SEAs have significant needs for technical as sistance and capacity building, as many are newly moving into a role in which they are choosing and implementing educator evaluation systems or supporting districts and schools as they hire and evaluate their professional staff. This priority is also tied to school reform efforts and large federal funding streams such as the Race to the Top initiative, the School Improvement Grants, and the Teacher Incentive Fund. Early learning is a high

- profile issue, recently gaining renewed attention from SEAs, the Dep artment of Education, and President Obama. States vary in the attention and funding they have historically given to early childhood education, from universal full

- day prekindergarten in some states to no state
- supported pre

- K at all in others. Research evi dence is clear, however, that high

- quality early educat ion leads to better student outcomes, thus leading states to increase their focus on this issue. Given the overarch ing nature of the Great Teachers and Leaders priority, and the recent policy focus on early learning efforts, we believe that focusing on these two priorities will allow us to learn about how CCs develop ed SEA capacity in these two areas and what difference it made. 

### Data Sources Data collection for this study will consist of surveys of TA recipients, interviews of SEA staff, surveys and interviews of CC staff, and observations of TA events. **TA recipient surveys**

 Purpose: To gather information about TA received in selected priority areas from the C Cs, understand the actions resulting from participati ng in TA, and exam ine outcomes related to that TA  Sample: All SEA and possibly district staff who received TA from the CCs in the selected priority areas, along with Regional Center staff receiving TA from Content Centers  Timing: Once yearly, beginning in January 2015 (following OMB approval) **CC staff and SEA interviews**

 Purpose: To gather information about the Centers’ capacity building efforts, th eories of action, program implementation, and outcomes ;  Sample: A purposeful sample of Center Directors, Managers, Evaluators, and TA staff; Recipients of CC TA, including SEA and possibly LEA staff, and other C Cs ;  Timing: In coordination with site visits
.4

**CC staff surveys**

 Purpose: To gather information from multiple CC TA staff about t he nature, successes and challenges of their work building SEA capacity or Regional Center capacity, and to help identify high

- leverage projects for more detailed study;  Sample: All Center staff providing TA in the selected priority areas ;  Timing: Once ye arly, beginning in January 2015 ( immediately following OMB approval)
. **TA event observations** 

 Purpose: To obtain detailed data about the strategies that CCs used to support capacity building and achieve planned outcomes ;  Sample: Observable services or events that are planned for the selected priority areas in profiled projects, or projects that are potential profiled projects;  Timing: On an ongoing basis beginning in Jan uary 2015, as relevant observable activities are identified
. 

### D **ESCRIPTION OF STATIS TICAL**  **METHODS (PART B) 1.**  **Sampling Design** T he study team has purposively select ed two of the seven Federal priority areas of the 22 Comprehensive Centers (CCs), and proposed to include the universe of CC staff and TA recipients who are heavily involved with CC activities in those priority areas . See Exhibit 1 for a more detailed description of sampling by instrument. The two selected Federal priority areas are 1. Identifying, recruiting, developing, and retaining highly effective teachers and leaders, and 2. Ensuring the school readiness and success of preschool

- age children and their successful transition to kindergarten. The selection of two priority areas in no way implies that the Department has a preference for these areas over others, or that the CCs or states should shift the focus of their efforts to these areas. Rather, this narrowing of focus allows us to target our resources in such a way that we are able to learn more about specific capacity

- building activities and outcomes. The target population, which includes the 22 CCs and their state education agency constituents, is small. Furthermore, each CC is unique, as it serves a different set of constituents with different capacity building needs. Sampling within this small population might not involve some of the work of some CCs. However, focusing on only two of the seven Federal priorities means that our data collection efforts will target projects related to these priorities, thus reducing burden on respondents. Therefore, in the interest of data qua lity with this small population, statistical methods to identify a probability sample from a larger respondent universe will not be used.5

**Project Profiles** There is one additional sample selection component within this study
. We will profile a small number of projects in depth to illustrate how capacity building can be implemented. Profiles of up to three successful projects will be developed within each of the selected Federal priority areas. We will employ a purposeful select ion process to id entify projects for these profiles as follows:

 In the surveys in 2015 and in the first data collection s ite visits (Q1 2015), site visitors will collect information from CC staff about the projects within the selected priority areas that might be appropria te for project profiles . These may be multi

-year, high

leverage projects that serve multiple state department s of education and/or that address critical c apacity needs. One project per key priority area will be identified for consideration at each Regional Center and at the relevant Content Centers.  Site visitors will conduct observations of events delivered as part of these projects during the first site visit (Q1 2015) if feasible given cost and availability of staff
.  In mid

- 2015 the evaluators will ident ify the six most promising projects based on: o State department of education feedback, via the surveys and interviews, on the effectiveness or outcomes of the projects, and o Evidence from CC staff interviews as to the projects’ progress in meeting their benchmarks, goals, and objectives
. Evaluators will gather details on these projects through documents and continued interviews and observations through early 201 7 (see Exhibit 1 ). Stakeholders outside of the CCs and state educati on agencies, such as partner organizations and school districts, may be considered for additional interviews if needed to develop a thorough understanding of the implementation and outcomes of a given project
. 

### 2. **Procedures for Information**  **Collection** The d ata collection procedures are discussed below. Exhibit 1 includes each instrument, its purpose, its administration time frame, its estimated sample, and the research questions addressed.6 **Exhibit 1**  **. Instruments, Purpose, Timing,**  **Sample, and Research Quest ions**  **Instrument Purpose**  **Administration timing**  **Estimated sample Major research questions**  **addressed** 1. Design
focused interview s with CC staff Understand

design/theory of action/capacity building model Feb

- Mar 2015 All CCs; Leadership staff; e stimated total of 88 respondents ( average of 4 per CC) 1. How did CCs define capacity building? Did their definitions change over time? If so, how? 2. What theories of action did CCs use to guide their general capacity

building work? Did the theories change over time? If so, how? 3. How did CCs assess the needs of their constituencies? 2. Implementation

focused interview s with CC staff Understand program

implementation, including key strategies, partnerships, successes and challenges, in the two priority areas Feb

- Mar 2015 and Jan
- Feb 2016 CCs with work in the two

selected priority areas; staff who work on projects in those areas; e stimated total of 11 4 ( 6 in each Regional Center and 6 in each of 4 Content Centers ) 4. What strategies did CC s employ to achieve their outcomes? 5. To what extent did CCs implement technical assistance to their constituencies as planned? 6. To what extent and how did CCs collaborate with each other? 7. To what extent did CCs achieve their goals and objectives? 3. Outcomes
focused interviews with CC staff* Understand technical

assistance outcomes, including longer term outcomes, and follow up on projects identified for profiles Jan
- Feb 2017 Note: this protocol will be developed after the first site visit and is not included in this package for review CCs with work in the two

selected priority areas; staff who work on projects in those areas; e stimated total of 88 respondents (4 per CC) 7. To what extent did CCs achieve their goals and objectives?

4. Interviews with SEAs/TA recipients Understand CCs'

implementation strategies and outcomes, including how CCs and SEAs work together in the two selected priority areas Feb

- Mar 2015, Jan
- Feb 2016 State education and

Regional Center staff identified by CCs as key contacts or participants in key CC projects in the selected priority areas; e stimated total of 88 (average of 4 per CC) 4. What strategies did CC s employ to achieve their outcomes? 5. To what extent did CCs implement technical assistance to their constituencies as planned? 7. To what extent did CCs achieve their goals and objectives? 5. CC staff online survey Understand CC staff

individual roles and perceptions of projects in the two priority areas, including how projects build capacity Jan 2015 , Nov 2015, Nov 2016 All CC staff and partners

who provide TA in the selected priority areas; e stimated total of 2 64 (average of 1 2 per CC) 4. What strategies did CC s employ to achieve their outcomes? 5. To what extent did CCs implement technical assistance to their constituencies as planned? 7. To what extent did CCs achieve their goals and objectives?7

6. TA recipient online survey Investigate how

technical assistance recipients, including state education staff and other participants as relevant, perceive and use the services they receive, and how these services have helped them build capacity Jan 2015, Nov 2015, Nov 2016 TA recipients in the

selected priority areas ; estimated total of 600 respondents (about 2 0 per CC) 7. To what extent did CCs achieve their goals and objectives?

7. TA event observations Provide observational

data about the strategies that CCs used to support capacity building and achieve planned outcomes. Will inform proje ct profiles
. On an ongoing basis beginning in Jan 2015 , as relevant observable activities are identified Observable events of TA

activities in the two selected priority areas ; e stimated total of 6
- 10 observations per year 4. What strategies did CC s employ to achieve their outcomes? 5. To what extent did CCs implement technical assistance to their constituencies as planned? 7. To what extent did CCs achieve their goals and objectives? 8
. Outcome
focused interviews with SEAs/TA recipients * Understand CCs'

implementation strategies and outcomes, including how CCs and SEAs work together in the two selected priority areas Jan
- Feb 2017 Note: this protocol will be developed after the first site visit and is not included in this package for review State education and

Regional Center staff identified by CCs as key contacts or participants in key CC projects in the selected priority areas; e stimated t otal of 88 (average of 4 per CC) 4. What strategies did CC s employ to achieve their outcomes? 5. To what extent did CCs implement technical assistance to their constituencies as planned? 7. To what extent did CCs achieve their goals and objectives?

*Phase 2 OMB submission **C C**  **staff interviews (instruments 1**  **- 3)**  CC staff will be interviewed as part of yearly site visits to each CC, taking place in early 2015, 2016, and 2017. There are three separate interview protocols because the evaluation team will be focusing on different evaluation questions, and therefore asking different interview questions during site visits as the work of the CCs progresses. Instrument 1 is a protocol for interviewing CC leaders in a group setting during the site visits about h ow they design their work with constituents (see Appendix A). It will be administered once in early 2015 during the first site visit. Instrument 2 (see Appendix B) focuses on the implementation of the technical assistance work, and will be administered in a group setting in the 2015 and 2016 site visits. Respondents for Instrument 2 will include staff members who are working on projects in the two selected federal priority areas. Instrument 3 will be administered in the 2017 site visits. It will be addresse d in a later OMB submission. For all of the CC staff interviews, the evaluation team will work with a coordinator from each CC who will help schedule the site visits. The evaluation team will provide guidelines for the selection of participants in intervi ews, and will follow recommendations from CCs regarding staff to be included in each interview. Each s ite visit team will consist of an evaluator from IMPAQ (the site liaison) and a consultant from a subcontractor
. The evaluators have expertise8

in the stud y design and evaluation methods, while the consultants have expertise in technical assistance to state and local education agencies. Having the two visitors work as a team during the site visits will enhance the accuracy of the data gathered
. The use of ex perienced interviewers, coupled with careful preparation, will guide the specific wording of each question and probe on the protocol, to ensure the interviews are customized appropriately to address the unique situations of each interviewee. The same pair of visitors will attend all site visits of the CCs to which they are assigned , and both site visitors will attend all interviews if possible. However, site visitors may conduct interviews separately if this is necessary to collect information from all resp ondents. Because of this, and to ensure accurate notes, each interview will be audio recorded if the interviewees permit . The IMPAQ evaluator (site liaison) for each site will have ultimate responsibility for completing and submitting the specified set of standard deliverables on the checklist for each site visit. Throughout the process of data collection and reporting, the contractors will make all efforts to protect the privacy of respondents participating in the site visits. The study team will not identify by name any of the interviewees, nor will the study team attribute quotes by name , a lthough the study team will identify the names of states and CCs in final reporting
. The study leaders will train the site visit team s so that all team members sha re a consistent understanding of the study, the research questions, the interview questions and probes, and the data collection needs. Prior to each wave of site visits all site v isitors will convene in Washington , DC or Oakland, CA for a half

- day trainin g session. The session will address the research questions, site visit logistics and activities, site pre
- and post

-visit communication, data collection procedures on site (including a review of the interview protocols), and data handling. T he site visit t ask leaders will develop a site visitor guide with a checklist that will outline all tasks the site visitors need to perform before, during, and after each visit. During the months of site visits t here will be regular meetings of the site visitors and stud y leaders to discuss issues and concerns
. **Technical assistance recipient interviews (instrument 4 and**  **8 )**  Prior to each visit, s ite visitors will ask Center staff to identify the key SEA representatives they have worked with on each major project within the two selected priority areas (up to two projects per priority area )
. State education agency representatives may include Chief State School Officers and their deputies, division leaders, and middle managers and technical staff within divisions, a s relevant to the projects. Evaluators will contact the respondents and conduct these interviews in person, if feasible in conjunction with site visits to the centers, or by tele phone. These interviews (see Appendix C) will include questions about planning, needs identification, and ongoing communication between the Centers and the state education agencies, implementation of projects in the selected priority areas, and outcomes of these projects. A second TA recipient interview protocol , to be administered in the 2017 site visits, will be developed after the first site visit. The instrument will be based on the first instrument but will be more focused on outcomes and informed by data gathered in the first site visit. It will be addressed in a later OMB subm ission.9

**CC staff online**  **survey (instrument 5)** The CC staff survey (see Appendix D) will be administered once per year via online survey software prior to the site visits, starting in January 2015
. The survey includes questions about technical assistance implementation in projects under the two selected federal priorities. Each CC’s list of projects will be unique to that CC’s work plan, but the questions asked about the projects will be the same from su rvey to survey. The survey addresses the kinds of technical assistance involved in each project, the role of the respondent, challenges and supporting factors, staff members and collaborators on each project, and perceptions of capacity building. It also a sks for suggestions for projects suitable for project profiles. Once data collection procedures have been approved by OMB, IMPAQ staff will contact each of the CCs to obtain the list of respondents for each site. The evaluation team will provide guidelines for the selection of survey respondents (people who are involved in technical assistance with projects in the two selected federal priority areas), and will follow recommendations from CCs regarding staff to be included in each survey
. Participants will receive a survey link via email
. The study team will not identify by name any of the survey respondents, nor will the study team attribute quotes by name. **Technical assistance (TA)**  **recipient online**  **survey (instrument 6)** The TA recipient survey (see Appendi x E) will be administered once per year via online survey software, starting in January 2015
. It will be addressed to state education agency staff, as well as to school district staff and other relevant stakeholders who are direct recipients of TA from CCs
. The survey includes questions about technical assistance received from CCs during past 12 months under the two selected F ederal priorities. Similar to the CC staff survey, the list of TA activities provided by e ach CC will be unique , but the questions as ked about the TA will be the same from survey to survey. The survey addresses specific TA received, actions taken as a result of the received TA, and statements about individual and organizational capacity built under the two selected federal priorities. I t also asks for the most helpful aspects of TA, challenges encountered , and specific plans to use TA in upcoming months
. Once data collection procedures have been approved by OMB, IMPAQ staff will ask each of the CCs for the list of specific TA activities delivered in past 12 months and for the contact information for their TA recipients (including SEA staff, LEA staff and other as applicable) . The evaluati on team will provide guidelines as to what constitutes a TA activity (such as a workshop, webinar, con sultation, product, and toolkit)
. TA recipients will receive survey links via email
. The study team will not identify by name any of the survey respondents, nor will the study team attribute quotes by name. **Technical assistance event observations (instrument 7)** 10

Observations of TA events will provide detailed data about the strategies that C Cs use to support capacity building and achieve planned outcomes. These data will inform research questions about differences in strategies between C Cs, characteristics of strategies, and the extent of collaboration with other C Cs. Follow

- up data on the capacity building outcomes of the event may be obtained through the TA recipient surveys and interviews, particularly for events that are part of the p rojects to be profiled. In preparation for each observation, observers will gather information from the C C staff leading the event or extant documents in order to identify:

 Goals of the event/needs to be addressed  How these goals relate to the CC’s theory of action, workplan, and relevant strategy or project plan  Intended audience  Expected outcomes of the event  As relevant, other background on the planning or preparation for the event A structured protocol for the observations, including questions to be a ddressed prior to the event, is included as Appendix F
. During the meetings/site visits with C Cs, or in follow

- up calls as needed, the evaluators will ask C C staff members to identify observable services or events of profiled projects (or potentially profiled projects) that are planned for the two selected Federal priority areas in the coming year. These events may include webinars, stakeholder meetings (virtual or in

- person), and in

person training events, workshop s, or presentations. When possible, observation s will be conducted in conjunction with site visits. Observations of webinars or other virtual events will be conducted as they are scheduled. O bservations will focus on a convenience sample of high

leverage a ctivities that may illustrate the capacity building process. For example, rather than observing training events where individuals impart knowledge to other individuals, we will attempt to observe more dynamic events, such as roundtables or cross - state even ts and planning sessions, where we might be able to observe how the C Cs work with their constituents and how the constituents respond. 

### 3. **Methods to Maximize Response** The evaluation team has designed instruments so that they are easy to understand and pla ce as little burden on participants as possible, which will encourage response. **CC staff interviews and surveys** IMPAQ has invested time and effort in site development at each CC
. In winter 2013, site liaisons and consultants spoke with each CC director an d with CC evaluators in order to explain the goals of the evaluation and to preview upcoming data collection activities. In spring/summer 2014, site liaisons and consultants visited each CC to introduce themselves and to have informal conversations about t he work of the CCs. Because CCs know they are not being “judged” by this evaluation, but rather are participating in a descriptive study of the work of the system of CCs, CC representatives and evaluation team members have established a cooperative relatio nship.11

This spirit of cooperation and the eagerness of CCs to have their work understood by a larger audience are likely to result in high response rates for interviews and surveys
. Site liaisons will have the opportunity to plan the list of CC staff surve y participants with CC directors or their designees. Once the participant list is selected, the evaluation team will notify intended respondent s via email several days prior to the survey release, in order to let them know to expect an upcoming online survey link. Approximately one week after the survey links have gone out, IMPAQ staff will begin to follow up with any nonrespondents. This will initially involve email reminders to respondents, followed by telephone calls
. I nterviews will take place in a group setting during site visits, unless scheduling difficulties necessitate individual interviews. S ite liais ons will identify the lists of participants in advance with CC directors. Site liaisons and consultants will then interview identified participant s in person during site visits. In the event that some identified CC staff members are participating in site visits via videoconferencing or teleconferencing, site visitors will conduct the interviews using these technologies. If some CC staff members are not able to participate in site visits at all, but should be included in interviews, site visitors will (with the help of CC directors) plan follow

- up call s or videoconference s with them to conduct the interviews. Because of the site development work prev iously accomplished, and given that under th eir cooperative agreement the CCs are required to provide data for evaluation purposes, the evaluation team anticipates a 100 percent response rate for the interviews and a 90 percent response rate for the CC sta ff surveys
. **TA recipient surveys and interviews** Based on our experience, we expect a n 80 percent response rate for the TA recipient surveys
. W e will take the following steps to maximize response rate . Evaluators will rely on CC staff to identify TA recipie nts with whom they have worked or who have participated in CC

- organized events such as workshops and webinars and to provide us with their contact information. Once we send surveys to recipients, s everal rounds of email reminders will be sent to any nonrespondents
. If these are not successful, we will proceed with phone or mail follow

- up to boost the overall response rates. Regarding the TA recipient interviews, evaluators will ask CCs for the contact information of key TA recipients in projects in the two selected priority areas. Evaluators w ill contact the TA recipients using the following procedures : 1) All identified SEA recipients will receive an initial contact via email, requesting their participation in a one

- hour interview, and providing an overview of the scope and purpose of the evaluation
. The general purpose of the interview and the topics to be covered will also be described. 2) If there is no response within a week, evaluators will follow up by telephone within one week of sending each email. 3) If the first telephone call does not reach the respondent or does not accomplish scheduling the interview, evaluators will continue to follow up through a combination of email and tele phone. If in

person interviews cannot be arranged during a sit e visit, telephone interviews will be arrange d to accommodate the respondent s’ schedule s within the next two months. We expect that this flexibility will result in a response rate of 80 %. If a respondent is unreachable or completely12

unava ila ble for an inte rview, evaluators will ask the relevant CC for names of alternate TA recipients who might serve as replacements or representatives. 

### 4. **Review and Testing of Instruments**  Study team members have pre -tested drafts of the interview protocol questions with up t o nine respondents from CCs. Responses to the questions were collected by site liaisons via interview notes and used to make revisions to the instruments in terms of wording, flow, length, and order of questions. The interview protocol notes were also used to inform the construction of the survey items. 

### 5. **Individuals and Organizations Involved** The following individuals and organizations are involved in data collection, analysis, and consulting on statistical aspects of the study design. **Responsibility Organization**  **Contact Name Telephone**  **Number** Co
- Principal Investigator and Project Director IMPAQ International Cheri Fancsali 443
- 283
- 6220
Co
- Principal Investigator IMPAQ International Phyllis Weinstock 510
-465
- 7884 x221 Associate Education Research Scientist Institute of Education Sciences, ED Amy Johnson 202
- 208
- 7849 Research Scientist Institute of Education Sciences, ED Joy Lesnick 202
- 219
- 2013 Technical Working Group Member Consultant Margaret Goetz 609
- 737
- 2464 Technical Working Group Member Consultant Thomas Adams 530
-848
-9728 Technical Working Group Member Consultant Constancia Warren 212
- 367
-4595 Analytic Support IMPAQ International Kelley Akiya 510
-465
-7884 Task Lead, Site Liaison IMPAQ International Andrea Beesley 443
- 832
- 2313 Task Lead, Site Liaison IMPAQ International Anne Chamberlain 443
- 718
-4343 Analytic Support IMPAQ International Maria DiFuccia 443
- 367
-0088 Task Lead, Site Liaison IMPAQ International Michaela Gulemetova 443
- 367
-0088 Analytic Support IMPAQ International Ashley Hunt 202
- 696
- 1012 Site Liaison IMPAQ International Kay Magill 510
-465
-7884 Site Liaison IMPAQ International Nada Rayyes 510
-465
-7884 Analytic Support IMPAQ International Eliana Saltares 202
- 696
- 103313

**Responsibility Organization**  **Contact Name Telephone**  **Number** Site Liaison IMPAQ International Raquel Sanchez 626
- 365
-4610 Site Liaison IMPAQ International Linda Toms Barker 808
- 934
-9297 Technical Advisor IMPAQ International Xiaodong Zhang 443
- 718
-4344 Technical Assistance Expert, Site Visitor Consultant Michelle Feist 804
- 252
-5714 Technical Assistance Expert, Site Visitor Consultant Deborah Jonas 503
- 381
-4164 Technical Assistance Expert, Site Visitor Consultant Leslie Rennie

- Hill 410
- 206
-0394 Technical Assistance Expert, Site Visitor Consultant Paul Smith 541
- 543
-9179 Technical Assistance Expert, Site Visitor Consultant Michelle Swanson 804
- 252
-5714