Evaluation of the Comprehensive 

Technical Assistance Centers 

OMB Clearance Request for Data Collection Instruments

Part B: Supporting Statement for Paperwork Reduction Act Submission

September 18, 2014

Prepared for:

U.S. Department of Education

Contract No. ED-IES-13-C-0059

Prepared by:

IMPAQ International
1

INTRODUCTION 

This document has been prepared to support the clearance of data collection instruments for 

the National Evaluation of the Comprehensive Technical Assistance Centers. The Institute of 

Education Sciences (IES) within the U.S Department of Education (ED) is conducting this 

evaluation. In the introduction to the supporting statement, we provide a description of the 

Comprehensive Technical Assistance Centers program, the evaluation questions and study 

design. The remaining sections of this document respond to specific instructions of the Office of 

Management and Budget (OMB) for the preparation of a supporting statement.

The Comprehensive Technical Assistance Centers

Title II of the Educational Technical Assistance Act of 2002 (F.T AA, Section 203) authorized the 

Comprehensive Center (CC) Program, a discretionary grant program establishing technical 

assistance centers. The CCs were last awarded in 2012, to provide technical assistance to “help 

State Education Agencies (SEAs) build their capacity to implement State-level initiatives and to 

support district- and school-level initiatives that improve educational outcomes for all students, 

close achievement gaps, and improve the quality of instruction” (77 FR 33564). 

In 2012, the Department of Education awarded new five-year grants to 15 Regional Centers and 

7 Content Centers under the Comprehensive Centers program. The Regional Centers each serve 

one to seven U.S. states, territories, and possessions. They provide technical assistance (TA) 

that builds the capacity of SEAs to implement, support, scale up, and sustain initiatives that 

help districts and schools improve student outcomes. The Regional Centers focus their work on 

seven Federal priority areas:

1. Implementing college- and career-ready standards and aligned, high-quality 

assessments for all students;

2. Identifying, recruiting, developing, and retaining highly effective teachers and leaders;

3. Turning around the lowest-performing schools;

4. Ensuring the school readiness and success of preschool-age children and their successful 

transition to kindergarten;

5. Building rigorous instructional pathways that support the successful transition of all 

students from secondary education to college without the need for remediation, and 

careers;

6. Identifying and scaling up innovative approaches to teaching and learning that 

significantly improve student outcomes; and

7. Using data-based decision-making to improve instructional practices, policies, and 

student outcomes.

The Content Centers provide the Regional Centers and SEAs with in-depth content knowledge

and expertise by providing information, publications, tools, and specialized technical assistance. 

The 7 Content Centers are:

1. Center on Standards and Assessments Implementation

2. Center on Great Teachers and Leaders
2

3. Center on School Turnaround

4. Center on Enhancing Early Learning Outcomes

5. Center on College and Career Readiness and Success

6. Center on Building State Capacity and Productivity

7. Center on Innovations in Learning

The National Evaluation of the Comprehensive Technical Assistance Centers

The National Evaluation is charged with examining and documenting how the individual CCs 

intend to build SEA capacity (theories of action) and what types of activities they actually 

conduct to build capacity. It will also explore and document the outcomes of the capacity 

building efforts. It is designed to build on the previous evaluation of the CCs, which 

documented the type, extent, and quality of services provided. 

Evaluation Questions

The evaluation will address broad questions in three areas: 

 Program Design: How did the CCs design their work?

1. How did CCs define capacity building? Did their definitions change over time? If 

so, how?

2. What theories of action did CCs use to guide their general capacity-building 

work? Did the theories change over time? If so, how?

3. How did CCs assess the needs of their constituencies?

In addressing these evaluation questions, we seek to identify how the CCs designed 

their work as TA providers, including the underlying theories of action driving the work. 

The evaluation will seek to surface the theories of action and definitions of capacity 

building employed by CCs, as well as describe CCs’ plans for assessing the needs of their 

constituencies and developing TA work plans to address those needs. 

 Program Implementation: How did the CCs operate?

4. What strategies did CCs employ to achieve their outcomes?

5. To what extent did CCs implement technical assistance to their constituencies as 

planned?

6. To what extent and how did CCs collaborate with each other?

In addressing these evaluation questions, we seek to identify the various strategies CCs 

used to build capacity, describe the characteristics of those strategies, and document 

the extent to which CCs implemented the TA as planned. We will also seek to identify 

common challenges and barriers CCs faced in building capacity, and ways they met 

those challenges. 

 Program Outcomes: What was the result of the CCs’ work? 

7. To what extent did CCs achieve their goals and objectives?

In addressing this evaluation question, we seek to identify the extent to which CCs 

achieved their goals and objectives, particularly as they relate to building their 

constituents’ capacity. We will also explore the extent to which outcomes aligned with 

and supported the CCs’ theories of action, and to identify factors that may have 

contributed to CCs’ success (or failure) in achieving expected outcomes. 
3

As a way to focus the evaluation to gather data in depth rather than breadth, the evaluation 

will limit data collection on the implementation and outcomes questions to two of the seven 

federal priority areas: identifying, recruiting, developing, and retaining highly effective teachers 

and leaders, and ensuring the school readiness and success of preschool-age children and their 

successful transition to kindergarten. All of the Regional Centers have implemented projects 

and strategies that address the effective teachers and leaders priority, so it clearly represents 

an issue of nationwide focus. It is an area where most SEAs have significant needs for technical 

assistance and capacity building, as many are newly moving into a role in which they are 

choosing and implementing educator evaluation systems or supporting districts and schools as 

they hire and evaluate their professional staff. This priority is also tied to school reform efforts 

and large federal funding streams such as the Race to the Top initiative, the School 

Improvement Grants, and the Teacher Incentive Fund. 

Early learning is a high-profile issue, recently gaining renewed attention from SEAs, the 

Department of Education, and President Obama. States vary in the attention and funding they 

have historically given to early childhood education, from universal full-day prekindergarten in 

some states to no state-supported pre-K at all in others. Research evidence is clear, however,

that high-quality early education leads to better student outcomes, thus leading states to 

increase their focus on this issue. Given the overarching nature of the Great Teachers and 

Leaders priority, and the recent policy focus on early learning efforts, we believe that focusing 

on these two priorities will allow us to learn about how CCs developed SEA capacity in these 

two areas and what difference it made. 

Data Sources

Data collection for this study will consist of surveys of TA recipients, interviews of SEA staff, 

surveys and interviews of CC staff, and observations of TA events.

TA recipient surveys

 Purpose: To gather information about TA received in selected priority areas from the 

CCs, understand the actions resulting from participating in TA, and examine outcomes 

related to that TA

 Sample: All SEA and possibly district staff who received TA from the CCs in the selected 

priority areas, along with Regional Center staff receiving TA from Content Centers

 Timing: Once yearly, beginning in January 2015 (following OMB approval)

CC staff and SEA interviews

 Purpose: To gather information about the Centers’ capacity building efforts, theories of 

action, program implementation, and outcomes;

 Sample: A purposeful sample of Center Directors, Managers, Evaluators, and TA staff; 

Recipients of CC TA, including SEA and possibly LEA staff, and other CCs; 

 Timing: In coordination with site visits.
4

CC staff surveys

 Purpose: To gather information from multiple CC TA staff about the nature, successes 

and challenges of their work building SEA capacity or Regional Center capacity, and to 

help identify high-leverage projects for more detailed study;

 Sample: All Center staff providing TA in the selected priority areas;

 Timing: Once yearly, beginning in January 2015 (immediately following OMB approval).

TA event observations

 Purpose: To obtain detailed data about the strategies that CCs used to support capacity 

building and achieve planned outcomes;

 Sample: Observable services or events that are planned for the selected priority areas in 

profiled projects, or projects that are potential profiled projects;

 Timing: On an ongoing basis beginning in January 2015, as relevant observable activities 

are identified.

DESCRIPTION OF STATISTICAL METHODS (PART B)

1. Sampling Design

The study team has purposively selected two of the seven Federal priority areas of the 22 

Comprehensive Centers (CCs), and proposed to include the universe of CC staff and TA

recipients who are heavily involved with CC activities in those priority areas. See Exhibit 1 for a 

more detailed description of sampling by instrument.

The two selected Federal priority areas are

1. Identifying, recruiting, developing, and retaining highly effective teachers and leaders, 

and 

2. Ensuring the school readiness and success of preschool-age children and their successful 

transition to kindergarten. 

The selection of two priority areas in no way implies that the Department has a preference for 

these areas over others, or that the CCs or states should shift the focus of their efforts to these 

areas. Rather, this narrowing of focus allows us to target our resources in such a way that we 

are able to learn more about specific capacity-building activities and outcomes.

The target population, which includes the 22 CCs and their state education agency constituents, 

is small. Furthermore, each CC is unique, as it serves a different set of constituents with 

different capacity building needs. Sampling within this small population might not involve some 

of the work of some CCs. However, focusing on only two of the seven Federal priorities means 

that our data collection efforts will target projects related to these priorities, thus reducing 

burden on respondents. Therefore, in the interest of data quality with this small population,

statistical methods to identify a probability sample from a larger respondent universe will not 

be used.
5

Project Profiles

There is one additional sample selection component within this study. We will profile a small 

number of projects in depth to illustrate how capacity building can be implemented. Profiles of 

up to three successful projects will be developed within each of the selected Federal priority 

areas. We will employ a purposeful selection process to identify projects for these profiles as

follows:

 In the surveys in 2015 and in the first data collection site visits (Q1 2015), site visitors 

will collect information from CC staff about the projects within the selected priority 

areas that might be appropriate for project profiles. These may be multi-year, highleverage projects that serve multiple state departments of education and/or that 

address critical capacity needs. One project per key priority area will be identified for 

consideration at each Regional Center and at the relevant Content Centers.

 Site visitors will conduct observations of events delivered as part of these projects 

during the first site visit (Q1 2015) if feasible given cost and availability of staff.

 In mid-2015 the evaluators will identify the six most promising projects based on:

o State department of education feedback, via the surveys and interviews, on the 

effectiveness or outcomes of the projects, and

o Evidence from CC staff interviews as to the projects’ progress in meeting their 

benchmarks, goals, and objectives.

Evaluators will gather details on these projects through documents and continued interviews 

and observations through early 2017 (see Exhibit 1). Stakeholders outside of the CCs and state 

education agencies, such as partner organizations and school districts, may be considered for 

additional interviews if needed to develop a thorough understanding of the implementation 

and outcomes of a given project.

2. Procedures for Information Collection

The data collection procedures are discussed below. Exhibit 1 includes each instrument, its 

purpose, its administration timeframe, its estimated sample, and the research questions 

addressed.
6

Exhibit 1. Instruments, Purpose, Timing, Sample, and Research Questions 

Instrument Purpose Administration 

timing

Estimated sample Major research questions 

addressed

1. Designfocused 

interviews with

CC staff 

Understand 

design/theory of 

action/capacity 

building model 

Feb-Mar 2015 All CCs; Leadership staff; 

estimated total of 88 

respondents (average of 4 

per CC)

1. How did CCs define capacity 

building? Did their definitions 

change over time? If so, how?

2. What theories of action did CCs 

use to guide their general capacitybuilding work? Did the theories 

change over time? If so, how?

3. How did CCs assess the needs of 

their constituencies?

2. 

Implementationfocused 

interviews with

CC staff 

Understand program 

implementation, 

including key 

strategies, 

partnerships, 

successes and 

challenges, in the two

priority areas

Feb-Mar 2015 

and Jan-Feb 

2016 

CCs with work in the two 

selected priority areas; 

staff who work on projects 

in those areas; estimated 

total of 114 (6 in each 

Regional Center and 6 in 

each of 4 Content Centers)

4. What strategies did CCs employ to 

achieve their outcomes?

5. To what extent did CCs implement 

technical assistance to their 

constituencies as planned?

6. To what extent and how did CCs 

collaborate with each other?

7. To what extent did CCs achieve 

their goals and objectives?

3. Outcomesfocused 

interviews with 

CC staff* 

Understand technical 

assistance outcomes, 

including longer term 

outcomes, and follow 

up on projects 

identified for profiles 

Jan-Feb 2017 

Note: this 

protocol will be 

developed after 

the first site visit 

and is not 

included in this 

package for 

review

CCs with work in the two 

selected priority areas; 

staff who work on projects 

in those areas; estimated 

total of 88 respondents (4 

per CC) 

7. To what extent did CCs achieve 

their goals and objectives?

4. Interviews 

with SEAs/TA

recipients

Understand CCs' 

implementation 

strategies and 

outcomes, including 

how CCs and SEAs 

work together in the 

two selected priority 

areas

Feb-Mar 2015, 

Jan-Feb 2016 

State education and 

Regional Center staff

identified by CCs as key 

contacts or participants in 

key CC projects in the 

selected priority areas; 

estimated total of 88 

(average of 4 per CC) 

4. What strategies did CCs employ to 

achieve their outcomes?

5. To what extent did CCs implement 

technical assistance to their 

constituencies as planned?

7. To what extent did CCs achieve 

their goals and objectives?

5. CC staff online 

survey

Understand CC staff 

individual roles and 

perceptions of 

projects in the two

priority areas, 

including how 

projects build 

capacity 

Jan 2015, Nov

2015, Nov 2016

All CC staff and partners 

who provide TA in the 

selected priority areas; 

estimated total of 264

(average of 12 per CC)

4. What strategies did CCs employ to 

achieve their outcomes?

5. To what extent did CCs implement 

technical assistance to their 

constituencies as planned?

7. To what extent did CCs achieve 

their goals and objectives?
7

6. TA recipient 

online survey

Investigate how 

technical assistance

recipients, including 

state education staff 

and other participants 

as relevant, perceive 

and use the services 

they receive, and how 

these services have 

helped them build 

capacity

Jan 2015, Nov

2015, Nov 2016

TA recipients in the 

selected priority areas;

estimated total of 600

respondents (about 20 per 

CC)

7. To what extent did CCs achieve 

their goals and objectives?

7. TA event 

observations

Provide observational 

data about the 

strategies that CCs 

used to support 

capacity building and 

achieve planned 

outcomes. Will inform 

project profiles.

On an ongoing 

basis beginning 

in Jan 2015, as 

relevant 

observable 

activities are 

identified

Observable events of TA 

activities in the two 

selected priority areas; 

estimated total of 6-10 

observations per year

4. What strategies did CCs employ to 

achieve their outcomes?

5. To what extent did CCs implement 

technical assistance to their 

constituencies as planned?

7. To what extent did CCs achieve 

their goals and objectives?

8. Outcomefocused 

interviews with 

SEAs/TA

recipients*

Understand CCs' 

implementation 

strategies and 

outcomes, including 

how CCs and SEAs 

work together in the 

two selected priority 

areas

Jan-Feb 2017

Note: this

protocol will be 

developed after 

the first site visit 

and is not 

included in this 

package for 

review

State education and 

Regional Center staff 

identified by CCs as key 

contacts or participants in 

key CC projects in the 

selected priority areas; 

estimated total of 88 

(average of 4 per CC) 

4. What strategies did CCs employ to 

achieve their outcomes?

5. To what extent did CCs implement 

technical assistance to their 

constituencies as planned?

7. To what extent did CCs achieve 

their goals and objectives?

*Phase 2 OMB submission

CC staff interviews (instruments 1-3)

CC staff will be interviewed as part of yearly site visits to each CC, taking place in early 2015, 

2016, and 2017. There are three separate interview protocols because the evaluation team will 

be focusing on different evaluation questions, and therefore asking different interview 

questions during site visits as the work of the CCs progresses. Instrument 1 is a protocol for 

interviewing CC leaders in a group setting during the site visits about how they design their 

work with constituents (see Appendix A). It will be administered once in early 2015 during the 

first site visit. Instrument 2 (see Appendix B) focuses on the implementation of the technical 

assistance work, and will be administered in a group setting in the 2015 and 2016 site visits. 

Respondents for Instrument 2 will include staff members who are working on projects in the 

two selected federal priority areas. Instrument 3 will be administered in the 2017 site visits. It 

will be addressed in a later OMB submission. 

For all of the CC staff interviews, the evaluation team will work with a coordinator from each CC 

who will help schedule the site visits. The evaluation team will provide guidelines for the 

selection of participants in interviews, and will follow recommendations from CCs regarding 

staff to be included in each interview. Each site visit team will consist of an evaluator from 

IMPAQ (the site liaison) and a consultant from a subcontractor. The evaluators have expertise 
8

in the study design and evaluation methods, while the consultants have expertise in technical 

assistance to state and local education agencies. Having the two visitors work as a team during 

the site visits will enhance the accuracy of the data gathered. The use of experienced 

interviewers, coupled with careful preparation, will guide the specific wording of each question 

and probe on the protocol, to ensure the interviews are customized appropriately to address 

the unique situations of each interviewee. The same pair of visitors will attend all site visits of 

the CCs to which they are assigned, and both site visitors will attend all interviews if possible. 

However, site visitors may conduct interviews separately if this is necessary to collect 

information from all respondents. Because of this, and to ensure accurate notes, each interview 

will be audio recorded if the interviewees permit. The IMPAQ evaluator (site liaison) for each 

site will have ultimate responsibility for completing and submitting the specified set of standard 

deliverables on the checklist for each site visit. 

Throughout the process of data collection and reporting, the contractors will make all efforts to 

protect the privacy of respondents participating in the site visits. The study team will not 

identify by name any of the interviewees, nor will the study team attribute quotes by name,

although the study team will identify the names of states and CCs in final reporting.

The study leaders will train the site visit teams so that all team members share a consistent 

understanding of the study, the research questions, the interview questions and probes, and 

the data collection needs. Prior to each wave of site visits all site visitors will convene in 

Washington, DC or Oakland, CA for a half-day training session. The session will address the 

research questions, site visit logistics and activities, site pre- and post-visit communication, data 

collection procedures on site (including a review of the interview protocols), and data handling.

The site visit task leaders will develop a site visitor guide with a checklist that will outline all 

tasks the site visitors need to perform before, during, and after each visit. During the months of 

site visits there will be regular meetings of the site visitors and study leaders to discuss issues 

and concerns. 

Technical assistance recipient interviews (instrument 4 and 8)

Prior to each visit, site visitors will ask Center staff to identify the key SEA representatives they 

have worked with on each major project within the two selected priority areas (up to two 

projects per priority area). State education agency representatives may include Chief State 

School Officers and their deputies, division leaders, and middle managers and technical staff 

within divisions, as relevant to the projects. Evaluators will contact the respondents and 

conduct these interviews in person, if feasible in conjunction with site visits to the centers, or 

by telephone. These interviews (see Appendix C) will include questions about planning, needs 

identification, and ongoing communication between the Centers and the state education 

agencies, implementation of projects in the selected priority areas, and outcomes of these 

projects. A second TA recipient interview protocol, to be administered in the 2017 site visits,

will be developed after the first site visit. The instrument will be based on the first instrument 

but will be more focused on outcomes and informed by data gathered in the first site visit. It 

will be addressed in a later OMB submission. 
9

CC staff online survey (instrument 5)

The CC staff survey (see Appendix D) will be administered once per year via online survey 

software prior to the site visits, starting in January 2015. The survey includes questions about 

technical assistance implementation in projects under the two selected federal priorities. Each 

CC’s list of projects will be unique to that CC’s work plan, but the questions asked about the 

projects will be the same from survey to survey. The survey addresses the kinds of technical 

assistance involved in each project, the role of the respondent, challenges and supporting 

factors, staff members and collaborators on each project, and perceptions of capacity building. 

It also asks for suggestions for projects suitable for project profiles.

Once data collection procedures have been approved by OMB, IMPAQ staff will contact each of 

the CCs to obtain the list of respondents for each site. The evaluation team will provide 

guidelines for the selection of survey respondents (people who are involved in technical 

assistance with projects in the two selected federal priority areas), and will follow 

recommendations from CCs regarding staff to be included in each survey. Participants will 

receive a survey link via email. 

The study team will not identify by name any of the survey respondents, nor will the study 

team attribute quotes by name.

Technical assistance (TA) recipient online survey (instrument 6)

The TA recipient survey (see Appendix E) will be administered once per year via online survey 

software, starting in January 2015. It will be addressed to state education agency staff, as well 

as to school district staff and other relevant stakeholders who are direct recipients of TA from 

CCs. The survey includes questions about technical assistance received from CCs during past 12 

months under the two selected Federal priorities. Similar to the CC staff survey, the list of TA 

activities provided by each CC will be unique, but the questions asked about the TA will be the 

same from survey to survey. The survey addresses specific TA received, actions taken as a result 

of the received TA, and statements about individual and organizational capacity built under the 

two selected federal priorities. It also asks for the most helpful aspects of TA, challenges

encountered, and specific plans to use TA in upcoming months.

Once data collection procedures have been approved by OMB, IMPAQ staff will ask each of the 

CCs for the list of specific TA activities delivered in past 12 months and for the contact 

information for their TA recipients (including SEA staff, LEA staff and other as applicable). The 

evaluation team will provide guidelines as to what constitutes a TA activity (such as a workshop, 

webinar, consultation, product, and toolkit). TA recipients will receive survey links via email.

The study team will not identify by name any of the survey respondents, nor will the study 

team attribute quotes by name.

Technical assistance event observations (instrument 7)
10

Observations of TA events will provide detailed data about the strategies that CCs use to 

support capacity building and achieve planned outcomes. These data will inform research 

questions about differences in strategies between CCs, characteristics of strategies, and the 

extent of collaboration with other CCs. Follow-up data on the capacity building outcomes of the 

event may be obtained through the TA recipient surveys and interviews, particularly for events 

that are part of the projects to be profiled.

In preparation for each observation, observers will gather information from the CC staff leading 

the event or extant documents in order to identify:

 Goals of the event/needs to be addressed

 How these goals relate to the CC’s theory of action, workplan, and relevant strategy or 

project plan 

 Intended audience

 Expected outcomes of the event

 As relevant, other background on the planning or preparation for the event 

A structured protocol for the observations, including questions to be addressed prior to the 

event, is included as Appendix F.

During the meetings/site visits with CCs, or in follow-up calls as needed, the evaluators will ask 

CC staff members to identify observable services or events of profiled projects (or potentially 

profiled projects) that are planned for the two selected Federal priority areas in the coming 

year. These events may include webinars, stakeholder meetings (virtual or in-person), and inperson training events, workshops, or presentations. When possible, observations will be 

conducted in conjunction with site visits. Observations of webinars or other virtual events will 

be conducted as they are scheduled. Observations will focus on a convenience sample of highleverage activities that may illustrate the capacity building process. For example, rather than 

observing training events where individuals impart knowledge to other individuals, we will 

attempt to observe more dynamic events, such as roundtables or cross-state events and 

planning sessions, where we might be able to observe how the CCs work with their constituents

and how the constituents respond. 

3. Methods to Maximize Response

The evaluation team has designed instruments so that they are easy to understand and place as 

little burden on participants as possible, which will encourage response. 

CC staff interviews and surveys

IMPAQ has invested time and effort in site development at each CC. In winter 2013, site liaisons 

and consultants spoke with each CC director and with CC evaluators in order to explain the 

goals of the evaluation and to preview upcoming data collection activities. In spring/summer 

2014, site liaisons and consultants visited each CC to introduce themselves and to have informal 

conversations about the work of the CCs. Because CCs know they are not being “judged” by this 

evaluation, but rather are participating in a descriptive study of the work of the system of CCs, 

CC representatives and evaluation team members have established a cooperative relationship. 
11

This spirit of cooperation and the eagerness of CCs to have their work understood by a larger 

audience are likely to result in high response rates for interviews and surveys.

Site liaisons will have the opportunity to plan the list of CC staff survey participants with CC 

directors or their designees. Once the participant list is selected, the evaluation team will notify 

intended respondents via email several days prior to the survey release, in order to let them 

know to expect an upcoming online survey link. Approximately one week after the survey links 

have gone out, IMPAQ staff will begin to follow up with any nonrespondents. This will initially 

involve email reminders to respondents, followed by telephone calls.

Interviews will take place in a group setting during site visits, unless scheduling difficulties 

necessitate individual interviews. Site liaisons will identify the lists of participants in advance 

with CC directors. Site liaisons and consultants will then interview identified participants in 

person during site visits. In the event that some identified CC staff members are participating in 

site visits via videoconferencing or teleconferencing, site visitors will conduct the interviews 

using these technologies. If some CC staff members are not able to participate in site visits at 

all, but should be included in interviews, site visitors will (with the help of CC directors) plan

follow-up calls or videoconferences with them to conduct the interviews.

Because of the site development work previously accomplished, and given that under their

cooperative agreement the CCs are required to provide data for evaluation purposes, the 

evaluation team anticipates a 100 percent response rate for the interviews and a 90 percent 

response rate for the CC staff surveys.

TA recipient surveys and interviews

Based on our experience, we expect an 80 percent response rate for the TA recipient surveys. 

We will take the following steps to maximize response rate. Evaluators will rely on CC staff to 

identify TA recipients with whom they have worked or who have participated in CC-organized 

events such as workshops and webinars and to provide us with their contact information. Once 

we send surveys to recipients, several rounds of email reminders will be sent to any

nonrespondents. If these are not successful, we will proceed with phone or mail follow-up to 

boost the overall response rates.

Regarding the TA recipient interviews, evaluators will ask CCs for the contact information of key 

TA recipients in projects in the two selected priority areas. Evaluators will contact the TA 

recipients using the following procedures: 1) All identified SEA recipients will receive an initial 

contact via email, requesting their participation in a one-hour interview, and providing an 

overview of the scope and purpose of the evaluation. The general purpose of the interview and 

the topics to be covered will also be described. 2) If there is no response within a week, 

evaluators will follow up by telephone within one week of sending each email. 3) If the first 

telephone call does not reach the respondent or does not accomplish scheduling the interview, 

evaluators will continue to follow up through a combination of email and telephone. If inperson interviews cannot be arranged during a site visit, telephone interviews will be arranged 

to accommodate the respondents’ schedules within the next two months. We expect that this 

flexibility will result in a response rate of 80%. If a respondent is unreachable or completely 
12

unavailable for an interview, evaluators will ask the relevant CC for names of alternate TA 

recipients who might serve as replacements or representatives.

4. Review and Testing of Instruments

Study team members have pre-tested drafts of the interview protocol questions with up to nine 

respondents from CCs. Responses to the questions were collected by site liaisons via interview 

notes and used to make revisions to the instruments in terms of wording, flow, length, and 

order of questions. The interview protocol notes were also used to inform the construction of 

the survey items.

5. Individuals and Organizations Involved 

The following individuals and organizations are involved in data collection, analysis, and consulting on 

statistical aspects of the study design.

Responsibility Organization Contact Name Telephone 

Number

Co-Principal Investigator 

and Project Director

IMPAQ International Cheri Fancsali 443-283-6220

Co-Principal Investigator IMPAQ International Phyllis Weinstock 510-465-7884 

x221

Associate Education 

Research Scientist

Institute of Education 

Sciences, ED

Amy Johnson 202-208-7849

Research Scientist Institute of Education 

Sciences, ED

Joy Lesnick 202-219-2013

Technical Working Group 

Member

Consultant Margaret Goetz 609-737-2464

Technical Working Group 

Member

Consultant Thomas Adams 530-848-9728

Technical Working Group 

Member

Consultant Constancia Warren 212-367-4595

Analytic Support IMPAQ International Kelley Akiya 510-465-7884

Task Lead, Site Liaison IMPAQ International Andrea Beesley 443-832-2313

Task Lead, Site Liaison IMPAQ International Anne Chamberlain 443-718-4343

Analytic Support IMPAQ International Maria DiFuccia 443-367-0088

Task Lead, Site Liaison IMPAQ International Michaela Gulemetova 443-367-0088

Analytic Support IMPAQ International Ashley Hunt 202-696-1012

Site Liaison IMPAQ International Kay Magill 510-465-7884

Site Liaison IMPAQ International Nada Rayyes 510-465-7884

Analytic Support IMPAQ International Eliana Saltares 202-696-1033
13

Responsibility Organization Contact Name Telephone 

Number

Site Liaison IMPAQ International Raquel Sanchez 626-365-4610

Site Liaison IMPAQ International Linda Toms Barker 808-934-9297

Technical Advisor IMPAQ International Xiaodong Zhang 443-718-4344

Technical Assistance 

Expert, Site Visitor Consultant Michelle Feist 804-252-5714

Technical Assistance

Expert, Site Visitor Consultant Deborah Jonas 503-381-4164

Technical Assistance 

Expert, Site Visitor Consultant Leslie Rennie-Hill 410-206-0394

Technical Assistance 

Expert, Site Visitor Consultant Paul Smith 541-543-9179

Technical Assistance 

Expert, Site Visitor Consultant Michelle Swanson 804-252-5714