IMPAQ International, LLC 1. Design

‐ Focused Interview Guide for Center Staff

September 15, 2014 

### 1. DESIGN

‐ FOCUSED INTERVIEW GUIDE FOR CENTER STAFF

The OMB Control Number for this information collection is <insert number> and the expiration date is <insert

date>.

According to the Paperwork Reduction Act of 1995, persons are not required to respond to this collection of

information unless it displays a currently valid OMB control number and expiration date. Responding to this

interview is voluntary. Public reporting burden for this collection of information is estimated to average 60

minutes per response. Send comments regarding this burden estimate or any other aspect of this collection of

information to XXXXXXXXXXXXXXX at XXXXXXXXXXXXX@ed.gov.

[NOTE TO INTERVIEWER: Below are suggested introductory remarks. While it is not necessary to

follow this as a script, it is important that you cover all of the main points contained here.]

I work for IMPAQ International, and we are evaluating the system of Comprehensive Technical

Assistance Centers. The U.S. Department of Education’s Institute for Education Sciences

contracted with us to conduct this study. You may recall that we visited last year to learn about

your Center.

As part of our study, we will be interviewing directors, managers, and staff at the Centers to

understand the work that you do. The purpose of the study is to gather data to describe the

work of the Centers and how the Centers build the capacities of state departments of education.

We will use the information we learn from our visit today to report on our results to the U.S.

Department of Education. What you have to say is important to us; we appreciate your helping

us understand your work.

Everything that we discuss during this interview will be used only for purposes of this study.

We will endeavor to keep your responses confidential by not identifying you by name. Your

responses will be de

‐ identified and will not be shared directly with individuals outside of the

research team. Findings from the interviews will be reported in summary form, but your role

and the Comprehensive Center you work with may be identified in the report. Before we

begin, do you have any questions about the purpose of the evaluation or our confidentiality

policy?

To start, do you mind if we audio record our session for the accuracy of my note taking?IMPAQ International, LLC 1. Design

‐ Focused Interview Guide for Center Staff

September 15, 2014 2 These questions will be asked of Center leadership (manager or director) and may include a

larger group including technical assistance providers. If respondents in a group interview

differ in their responses/opinion, those differences will be recorded in the interviewer notes.

Prior to conducting the interview, the interviewer will review relevant extant documents such

as proposals, annual performance review reports, evaluation reports, etc. Some of the

answers to questions asked below may be in those documents. If so, the interviewer will start

the question by referring to information gleaned from the document, and will ask the

respondent to verify the accuracy of the information.

Capacity Building Definition

1. For the purposes of your Center work with state education agencies (SEAs) [or for Content

Centers, SEAs and Regional Centers], how do you define capacity building? What is the

basis for your definition (research, experience, other Centers)?

PROBES:

How do you talk about capacity building among your Center staff?

Do you focus on different types of capacity building (e.g. human, organizational,

structural, etc.? If so, please tell us about that.

[For Content Center interviews: Do you focus on different types of capacity building for

Regional Centers as compared to SEAs? Does capacity building look different for

Regional Centers and SEAs?]

How do you discuss capacity building with your SEA constituents—to what extent do you

use the term “capacity building”?

2. When thinking about your work, how do you know when you have built capacity? What

does that “look like”? In other words, what observed changes in stakeholder behaviors

indicate capacity building? What are the outcomes that you expect to see?

3. Do you collect evidence to assess your progress in building capacity? If so, what kinds of

evidence do you collect?

Theory of Action

We are interested in the assumptions behind your work with SEAs—how you decide, out of all

of the options available, what work you will do with the SEAs (and for Content Centers, the

Regional Centers) and how you monitor and adjust that work as needed. Please describe

(and/or show us graphically) your theory of change and explain how you arrived at it.

4. How do you assess SEA needs—what information do you request, collect and review?

PROBES:

Who is involved in conversations/meetings about SEA goals? [for Content Centers, ask

also about Regional Center goals and goals that might be generated through means

other than directly by states or Regional Centers, e.g., addressing a broad, multi

‐ sector

or national need]IMPAQ International, LLC 1. Design

‐ Focused Interview Guide for Center Staff

September 15, 2014 3 To what extent do your SEAs present you with their goals, and to what extent do you

suggest goal options to them?

5. How do you determine the needs you can help the SEA meet or the goals the SEA can

pursue through collaboration with your Center?

6. How do you assess the SEA’s current level of capacity to meet their goals? [For Content

Center interviews, also ask: is the process different in your work with Regional Centers? If

yes, in what ways?]

PROBES:

To what extent do your SEAs present you with their goals, and to what extent do you

suggest goal options to them?

If different SEA personnel present you with different goals, how do you resolve this?

7. To what extent do the states’ identified needs align with the Federal priority areas? Have

you encountered many state needs or requests for TA that did not align with these

priorities? Or that you could not address for some other reason? If so, please explain. How

do you respond in these cases?

8. Once SEAs have determined their goals, how do you choose your Center objectives—the

overall results your Center hopes to achieve to support the SEA’s goals?

PROBES:

Who is involved in conversations/meetings about Center objectives?

To what extent are the Center objectives determined by the SEA’s requests of you, and to

what extent do you suggest the Center’s objectives to the SEA?

9. How do you choose which specific strategies or projects to pursue to achieve your

objectives? What is your theory about how these strategies, activities, or projects will

contribute to achieving your objectives?

PROBES:

Who is involved in conversations/meetings about strategies?

What do you take into consideration when deciding upon strategies (feasibility,

evidence, availability of expertise, experience in other projects, expectations of SEA,

likelihood of building capacity, etc.)

To what extent are your projects/strategies determined by the SEAs’ requests of you,

and to what extent do you suggest the projects to the SEAs?

10. How do you customize strategies to the SEA’s existing capacity level and needs?

11. Can you describe how the local context—political, economic, etc.—affects your choice of

strategies/projects, if at all?

PROBES:

How do you become aware of these contextual factors?IMPAQ International, LLC 1. Design

‐ Focused Interview Guide for Center Staff

September 15, 2014 4 How influential do you think context is in your decision making?

12. How does the availability of Center resources (financial, human, etc.) enter into your choice

of strategies? [ If resources do affect strategy choice: If you’re making decisions about

projects based on resources, do you involve stakeholders (SEAs/RCs) in the decision

process? If so, how?]

The next set of questions asks about practical aspects of your project work, such as how you

monitor project activities and how you determine timeframes. These questions will help us

describe the process by which you do your work, and how your theories of actions inform the

process.

13. How do you track and assess progress in your projects? What evidence do you collect or

review to do this?

14. How do you choose which outcomes to track for specific projects or the Center as a whole?

Can you give us any examples of outcomes that you’ve documented so far?

PROBES:

Who is involved in decisions about outcomes?

What implementation milestones do you track, and how?

What do you do if implementation milestones aren’t met?

How do you decide on longer

‐ term outcomes for your Center work?

How do you measure your longer

‐ term outcomes?

15. How do you determine the timeframe for a particular project? Do you have any rules of

thumb about how long projects should be?

PROBES:

How do you take stakeholder expectations and availability into account when developing

timelines? How does the SEAs’ planning process impact project timelines?

16. To what extent, if any, has your Center updated plans during a project? If so, how did you

become aware of a need to change something? Can you describe this process of changing

projects or strategies? How often does this happen?

17. As you track progress toward your Center objectives, to what extent do you also assess

SEAs’ progress toward their goals?

18. Have you modified your theory of change over time? If so, please describe how and why.

19. What else would you like to share about your approach to working with SEAs [or RCs, for

Content Centers]?