1

Evaluation of the Comprehensive

Technical Assistance Centers

OMB Clearance Request for Data Collection Instruments

Part A: Supporting Statement for Paperwork Reduction Act Submission

September 18, 2014

Prepared for:

U.S. Department of Education

Contract No. ED-IES-13-C-0059

Prepared by:

IMPAQ International
2

INTRODUCTION 

This document has been prepared to support the clearance of data collection instruments for 

the National Evaluation of the Comprehensive Technical Assistance Centers. The Institute of 

Education Sciences (IES) within the U.S Department of Education (ED) is conducting this 

evaluation. In the introduction to the supporting statement, we provide a description of the 

Comprehensive Technical Assistance Centers program, the evaluation questions and study 

design. The remaining sections of this document respond to specific instructions of the Office of 

Management and Budget (OMB) for the preparation of a supporting statement.

The Comprehensive Technical Assistance Centers

Title II of the Educational Technical Assistance Act of 2002 (F.T AA, Section 203) authorized the 

Comprehensive Center (CC) Program, a discretionary grant program establishing technical 

assistance centers. The CCs were last awarded in 2012, to provide technical assistance to “help 

State Education Agencies (SEAs) build their capacity to implement State-level initiatives and to 

support district- and school-level initiatives that improve educational outcomes for all students, 

close achievement gaps, and improve the quality of instruction” (77 FR 33564). 

In 2012, the Department of Education awarded new five-year grants to 15 Regional Centers and 

7 Content Centers under the CC program. The Regional Centers each serve one to seven U.S. 

states, territories, and possessions. They provide technical assistance (TA) that builds the 

capacity of SEAs to implement, support, scale up, and sustain initiatives that help districts and 

schools improve student outcomes. The CCs focus their work on seven Federal priority areas:

1. Implementing college- and career-ready standards and aligned, high-quality 

assessments for all students;

2. Identifying, recruiting, developing, and retaining highly effective teachers and leaders;

3. Turning around the lowest-performing schools;

4. Ensuring the school readiness and success of preschool-age children and their successful 

transition to kindergarten;

5. Building rigorous instructional pathways that support the successful transition of all 

students from secondary education to college without the need for remediation, and 

careers;

6. Identifying and scaling up innovative approaches to teaching and learning that 

significantly improve student outcomes; and

7. Using data-based decision-making to improve instructional practices, policies, and 

student outcomes.

The Content Centers provide the Regional Centers and SEAs with in-depth content knowledge 

and expertise by providing information, publications, tools, and specialized technical assistance. 

The 7 Content Centers are:

1. Center on Standards and Assessments Implementation

2. Center on Great Teachers and Leaders

3. Center on School Turnaround

4. Center on Enhancing Early Learning Outcomes
3

5. Center on College and Career Readiness and Success

6. Center on Building State Capacity and Productivity

7. Center on Innovations in Learning

The National Evaluation of the Comprehensive Technical Assistance Centers

The National Evaluation is charged with examining and documenting how the individual CCs

intend to build SEA capacity (referred to as theories of action) and what types of activities they 

actually conduct to build capacity. It is designed to build on the previous evaluation of the CCs, 

which documented the type, extent, and quality of services provided. 

Evaluation Questions

The evaluation will address broad questions in three areas: 

 Program Design: How did the CCs design their work?

1. How did CCs define capacity building? Did their definitions change over time? If 

so, how?

2. What theories of action did CCs use to guide their general capacity-building 

work? Did the theories change over time? If so, how?

3. How did CCs assess the needs of their constituencies?

In addressing these evaluation questions, we seek to identify how the CCs designed 

their work as TA providers, including the underlying theories of action driving the work. 

The evaluation will seek to surface the theories of action and definitions of capacity 

building employed by CCs, as well as describe CCs’ plans for assessing the needs of their 

constituencies and developing TA work plans to address those needs. 

 Program Implementation: How did the CCs operate?

4. What strategies did CCs employ to achieve their outcomes?

5. To what extent did CCs implement technical assistance to their constituencies as 

planned?

6. To what extent and how did CCs collaborate with each other?

In addressing these evaluation questions, we seek to identify the various strategies CCs 

used to build capacity, describe the characteristics of those strategies, and document 

the extent to which CCs implemented the TA as planned. We will also seek to identify 

common challenges and barriers CCs faced in building capacity, and ways they met 

those challenges. 

 Program Outcomes: What was the result of the CCs’ work? 

7. To what extent did CCs achieve their goals and objectives?

In addressing this evaluation question, we seek to identify the extent to which CCs achieved 

their goals and objectives, particularly as they relate to building their constituents’ capacity. 

We will also explore the extent to which outcomes aligned with and supported the CCs’ 

theories of action, and to identify factors that may have contributed to CCs’ success (or 

failure) in achieving expected outcomes.
4

Focus on Two Federal Priority Areas

As a way to focus the evaluation to gather data in depth rather than breadth, the evaluation 

will limit data collection on the implementation and outcomes questions to two of the seven 

federal priority areas: 

1. Identifying, recruiting, developing, and retaining highly effective teachers and leaders, 

and 

2. Ensuring the school readiness and success of preschool-age children and their successful 

transition to kindergarten. 

These two priority areas were purposefully selected. First, effective teachers and leaders is a 

topic area in which all of the Regional Centers have ongoing projects. In addition, this is a topic 

area where most SEAs have significant TA and capacity building needs, as many are choosing 

and implementing educator evaluation systems or supporting districts and schools as they hire 

and evaluate their professional staff. This priority is also tied to school reform efforts and large 

federal funding streams such as the Race to the Top initiative, the School Improvement Grants, 

and the Teacher Incentive Fund. 

The second priority area, early learning, is another high-profile topic which has recently gained 

increased attention. In response to federal initiatives and research findings on the benefits of 

high-quality early education, many states have increased their funding for state-supported early 

childhood education programs over last few years. This evaluation is well poised to examine the 

role that the CCs play in supporting state efforts in this priority area.

Given the overarching nature of the effective teachers and leaders priority, and the recent 

policy focus on early learning efforts, we believe that focusing on these two priorities will allow 

us to learn about how CCs develop SEA capacity (and in the case of Content Centers, both SEA 

and Regional Center capacity) in these two areas and what difference the CCs’ efforts have

made. Further, we believe that SEAs’ capacity building needs and the CCs’ approach to 

providing TA in these two priorities may differ across Centers in meaningful ways. These 

differences are likely to produce different types of capacity building outcomes (i.e., the needs 

and approach to building capacity to develop great teachers and leaders may be different than 

the needs and approach to building capacity related to early learning initiatives). Thus, by 

selecting these two priorities, we may learn more about the variety of needs and the ways the 

CCs address those needs, as well as their outcomes. 

The selection of two priority areas in no way implies that the Department has a preference for 

these areas over others, or that the centers or SEAs should shift the focus of their efforts to 

these areas. Rather, this narrowing of focus allows us to target our resources in such a way that 

we are able to learn more about specific capacity building activities and outcomes.
5

Data Sources

Data collection for this study will consist of surveys of TA recipients, interviews of CC staff and

TA recipients, surveys of CC staff, and observations of TA events.

TA recipient surveys

 Purpose: To gather information about TA received in selected priority areas from the CCs, 

understand the actions resulting from participating in TA, and examine outcomes related to 

that TA;

 Sample: All SEA and possibly school district staff who received TA from the CCs in the 

selected priority areas, along with Regional Center staff receiving TA from Content Centers;

 Timing: Yearly, beginning in January 2015 (following OMB approval).

CC staff and TA recipient interviews

• Purpose: To gather information about the Centers’ capacity building efforts, theories of 

action, program implementation, and outcomes;

• Sample: A purposeful sample of Center Directors, Managers, Evaluators, and TA staff; 

Recipients of CC TA, including SEA and possibly school district staff, and other CCs;

• Timing: In coordination with site visits.

CC staff surveys

 Purpose: To gather information from multiple CC TA staff about the nature, successes and 

challenges of their work building SEA capacity or Regional Center capacity, and to help 

identify high-leverage projects for more detailed study;

 Sample: All Center staff providing TA in the selected priority areas; 

 Timing: Yearly, beginning in January 2015 (immediately following OMB approval).

TA event observations

 Purpose: To obtain detailed data about the strategies that CCs use to support capacity 

building and achieve planned outcomes;

 Sample: Observable services or events that are planned for the selected priority areas, in 

profiled projects, or projects that are potential profiled projects;

 Timing: On an ongoing basis beginning in January 2015, as relevant observable activities are 

identified.

Evaluation reports

The evaluation will produce four reports. The first report will be a formative report focusing on 

how the CCs designed their work as technical assistance providers. The report will describe the

CCs’ underlying theories of action and definitions of “capacity building,” and explain how the 

Centers assessed their constituencies’ needs and developed work plans to address those needs. 

This report will be available in November 2015.
6

Two interim summative reports (projected date: March 2017) will be produced to address the 

program implementation and outcomes questions, one report for each of the two selected 

priority areas. These reports will demonstrate how and to what extent the Comprehensive 

Technical Assistance Center program has built state capacity in these priority areas. The reports 

will include descriptions of the strategies used to build SEA capacity, common challenges faced 

and ways the CCs sought to address them, the extent to which CCs achieved their goals and 

objectives, factors that may have contributed to success (or failure) in achieving expected 

outcomes, and the extent to which CCs’ outcomes aligned with and supported their theories of 

action. The interim summative reports will also include six profiles of multi-year projects, which 

will be selected and analyzed to illustrate how the CCs have successfully worked to build 

capacity related to project design and planning, implementation, and outcomes. 

A final report will be produced in September 2018. The final report will summarize findings 

documented in the interim reports and update findings based on new data. The report shall 

provide a full summary of lessons learned on the CCs’ efforts to build capacity.
7

PART A. JUSTIFICATION

1. Circumstances Necessitating the Information Collection 

Title II of the Educational Technical Assistance Act of 2002 (Section 204) requires that the 

National Center for Education Evaluation and Regional Assistance (NCEE), a component of the 

Department’s Institute of Education Sciences (IES), provide for ongoing independent evaluation 

of the Comprehensive Centers. The statute establishes the following specific goals for the 

evaluation: to analyze the services provided by the Centers; to determine the extent to which 

each of the Centers meets the objectives of its’ respective plan; and to determine whether the 

services offered by each Center meet the educational needs of SEAs, local educational agencies 

(LEAs), and schools in the region.

In October 2013, IES contracted with IMPAQ International to evaluate the Comprehensive 

Technical Assistance Centers. 

2. Purpose and Use of Information

The evaluation will provide the Department of Education and the CCs with a formative report

about how the CCs designed and carried out their work, and summative reports on program 

implementation and outcomes. In doing so, the evaluation seeks to inform CC staff, the 

Department of Education, and the larger field about the capacity building strategies and 

outcomes of the CCs’ work. The evaluation will inform ED’s staff as they support the CC and the 

CCs themselves as they implement their work. 

3. Use of Information Technology

Electronic technology will be used whenever possible to reduce the time burden on 

respondents. TA recipient and CC staff surveys will be administered online using an automated 

survey administration and data collection system. Online surveys are a proven, cost-effective 

data collection methodology that take less time to complete than paper or telephone surveys. 

In addition to enabling respondents to complete the survey at a time of their choosing, this 

method will allow the project team to monitor the survey response rate in real time and send 

customized, timely reminder emails.

4. Identification of Duplication of Information Collection Efforts

This study will yield unique data to evaluate the CCs. The evaluation will make use of data and 

documents already collected, produced, or maintained by the CCs and their local evaluators. 

Whenever possible, information will be collected and reviewed prior to interviews in order to 

avoid unnecessary interview questions. This study involves questions in surveys and interviews 

that are distinct from those asked by local evaluators. 

5. Impacts on Small Businesses or Other Small Entities

One CC is operated by a small business entity, and several CCs have small business partners. 

The evaluation team will minimize burden on these entities by scheduling interviews at the 

convenience of staff and coordinating data collection schedules with local evaluators to avoid 

multiple collections.
8

6. Consequences of Not Collecting the Data 

Failure to collect the data proposed through this study would breach the legislative mandate in 

Title II of the Educational Technical Assistance Act of 2002 (Section 204) that requires NCEE to

provide for ongoing independent evaluation of the Comprehensive Centers. It would also 

prevent ED from gaining an in-depth understanding of what capacity building strategies CCs are 

using and the outcomes of the CCs’ work. Understanding the strategies that the CCs implement 

and whether the CCs achieved their expected outcomes will enable federal policy makers and 

program managers to monitor the program and provide useful guidance to CCs. 

7. Special Data Collection Circumstances

None of the special circumstances listed apply to this data collection.

8. Federal Register Notice

9. Payments/Gifts to Respondents

There are no payments or gifts associated with this study.

10. Assurance of Privacy

TA recipients and CC staff who are surveyed and interviewed will be assured that their 

responses will be kept private to the extent permitted by law and by the design of the 

evaluation. Survey data will be stored on the evaluation contractor’s server that is protected by 

a firewall that monitors and evaluates all attempted connections from the Internet. Personal 

information (name, telephone number, and e-mail address) on each survey response will be 

maintained in a separate data file apart from the survey data so that individuals outside of the 

evaluation team cannot link particular responses to individual respondents. Once the contract 

is completed, all personal information on each survey respondent will be destroyed. The entire 

survey database will be encrypted so that any data stored will be further protected. Finally, 

access to any data with identifying information will be limited only to evaluation team members

directly working on the survey. Survey findings will be presented at a level of aggregation such 

that it will not be possible to link specific responses to individual respondents.

Everything that is discussed during interviews will be used only for the purposes of this study. 

We will not identify individual respondents by name, and will not share interview information

directly with individuals outside of the research team. Findings from the interviews will be 

reported in summary form. Respondents’ roles and the CCs they work with may be identified in 

the report, however, which may lead to individuals’ (such as CC directors or SEA chiefs) being 

identifiable. We will make respondents aware of these conditions.

Interview notes or recordings will not be shared with ED staff or anyone else outside the study

team. Paper copies of interview notes will be secured in a locked file cabinet. Electronic copies 

of notes will be stored in a SQL Server database located in the contractor’s access-controlled 

server room.
9

11. Questions of a Sensitive Nature

There are no data of a sensitive, personal, or private nature being collected in the surveys or 

interviews.

12. Estimate of Annualized Burden Hours and Costs

Exhibit 1 shows the estimated burden hours and estimated monetary burden for respondents 

for each instrument to be administered in this study. Exhibit 2 displays the estimates of 

annualized costs by respondent type. Because of the site development work previously 

accomplished, and given that under their cooperative agreement the CCs are required to 

provide data for evaluation purposes, the evaluation team anticipates a 100 percent response 

rate for the CC staff interviews and a 90 percent response rate for the CC staff surveys. Based 

on our experience, we expect an 80 percent response rate for the TA recipient surveys and an 

80 percent response rate for the SEA staff interviews.

The CC staff survey will take approximately 20 minutes to complete, including the time for 

reading our introductory letter and directions. The TA recipient survey will take about 15 

minutes to complete. Interviews with Center and SEA staff and will last about 60 minutes. The 

total burden hours are estimated at 1052. To estimate the time required for each type of data 

collection, we consulted with fewer than 10 IMPAQ staff members or CC staff members.

CC staff respondents are private sector employees in nonprofit or for-profit companies, or in 

universities. TA recipient respondents are employees of a state department of education or a 

local education agency such as a school district.
10

Exhibit 1: Estimated Burden Hours

Note: The outcomes-focused interview protocols will be developed after the first site visit and 

are not included in this package. The burden table will be updated when the outcomes-focused 

protocols are sent for review. 

Exhibit 2: Estimate of Annualized Costs by Respondent Type (per year for 3 years)

13. Estimates of Annualized Respondent Capital and Maintenance Costs

There are no direct costs to respondents other than that of their time of participation. There 

will be no start-up or ongoing financial costs incurred by respondents. There are no record 

keepers.

14. Estimates of Annualized Cost to the Government

The estimated cost for this study, including development of a detailed study design and data 

collection instruments, management of a Technical Working Group, preparation of a 

justification package, data collection, data analysis, and report preparation, is $7,861,244 for 

the five years, or an average of $1,572,244 per year. Estimated cost includes staff and 

consultant time, travel, and operational expenses.

15. Changes in Hour Burden

This request is for a new information collection. There are no changes in hour burden to report.

Respondent Type Total Cost

Annualized 

Cost

CC Staff $22,681 $7,560 

TA Recipient/SEA Staff $21,384 $7,128 

Total $44,065 $14,688 

Data Collection Activity

Total 

Sample

Size

Estimated

Response 

Rate

Number of 

Respondents

Time 

Estimate 

(in hours)

Number of 

Administr

ations

Total Hour

Burden

Hourly 

Rate

Estimated

Monetary 

Cost of 

Burden

Design-Focused Interviews 

with CC staff 88 100% 88 1 1 88 $45 $3,960 

Implementation-Focused

Interview with CC Staff

114 100% 114 1 2 228 $45 $10,260 

Interviews with TA

recipients

88 80% 70 1 2 141 $45 $6,336 

CC Staff Survey 264 90% 238 0.33 3 235 $45 $10,585 

TA Recipient Survey 600 80% 480 0.25 3 360 $45 $16,200 

Totals 1052 $47,341
11

16. Plans for Tabulation and Publication

16.1 Tabulation

The timeline for project data collection and reporting is in Exhibit 3. Data collection will begin in 

January 2015, or as soon as OMB approval is obtained, and conclude in the fourth quarter of 

2017. No complex analytical techniques will be used. For quantitative survey and observation

data, we will use descriptive statistics and analysis of variance. 

16.2 Publication

The evaluation will produce four reports (see Exhibit 3 for timeline). The first report will be a 

formative report focusing on the program design questions – how the Centers designed their 

work as technical assistance providers. The report will describe the Centers’ underlying theories 

of action and definitions of capacity building, and explain how the Centers assessed their 

constituents’ needs and developed work plans to address those needs. This report will be 

available in November 2015.

Two interim summative reports (projected date: March 2017) will be produced to address the 

program implementation and outcomes questions for each of the two selected priority areas. 

These reports will demonstrate how and to what extent the Comprehensive Technical 

Assistance Center program has built state capacity in these priority areas. The reports will 

include descriptions of the strategies used to build SEA capacity, common challenges faced and 

ways the Centers sought to address them, the extent to which Centers achieved their goals and 

objectives, factors that may have contributed to success (or failure) in achieving expected 

outcomes, and the extent to which Centers’ outcomes aligned with and supported their 

theories of action. The summative reports will also include six profiles of multi-year projects, 

which will be selected and analyzed to illustrate how the Centers have successfully worked to 

build capacity related to project design and planning, implementation, and outcomes. Finally, a 

final report will be produced in September 2018.

17. Approval to Not Display the Expiration Date

The OMB approval number and expiration date will be displayed or cited on all information 

collection instruments. 

18. Exceptions to the Certification Statement

There are no exceptions taken to item 19 of OMB Form 83-1.
12

2013

Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3

Administer TA recipient and CC staff surveys

Conduct interviews of TA recipients and CC staff

Conduct observations of profiled projects as 

opportunities arise

Draft report on Center designs/ theories of May

Final report on Center designs/theories of Nov

Draft reports on implementation and outcomes Aug

Final reports on implementation and outcomes Mar

Draft final summative report Apr

Final summative report Sep

Summative reports

Tasks and Activities 2014 2015 2016 2017 2018

Year 1

Surveys (TA recipient and CC staff)

Interviews (SEA and CC staff)

Formative report

Observations

Year 2 Year 3 Year 4 Year 5

Exhibit 3: Timeline of Data Collection and Reporting