IMPAQ International, LLC    1. Design‐Focused Interview Guide for Center Staff

   

APPENDIX A

1. DESIGN‐FOCUSED INTERVIEW GUIDE FOR CENTER STAFF

The OMB Control Number for this information collection is <insert number> and the expiration date is <insert

date>.

According to the Paperwork Reduction Act of 1995, persons are not required to respond to this collection of

information unless it displays a currently valid OMB control number and expiration date.  Responding to this

interview is voluntary.  Public reporting burden for this collection of information is estimated to average 60

minutes per response.  Send comments regarding this burden estimate or any other aspect of this collection of

information to XXXXXXXXXXXXXXX at XXXXXXXXXXXXX@ed.gov.

[NOTE TO INTERVIEWER: Below are suggested introductory remarks. While it is not necessary to

follow this as a script, it is important that you cover all of the main points contained here.]

I work for IMPAQ International, and we are evaluating the system of Comprehensive Technical

Assistance Centers. The U.S. Department of Education’s Institute for Education Sciences

contracted with us to conduct this study. You may recall that we visited last year to learn about

your Center.

As part of our study, we will be interviewing directors, managers, and staff at the Centers to

understand the work that you do. The purpose of the study is to gather data to describe the work

of the Centers and how the Centers build the capacities of state departments of education, and

to report that information to the Department of Education. What you have to say is important to

us and we appreciate you helping us understand your work. We want to assure you that

participation in this interview is voluntary. We estimate that this interview will take 60 minutes.

[NOTE TO INTERVIEWER: Please read the following to the respondent(s):  

“Information collected for this study comes under the confidentiality and data protection

requirements of the Institute of Education Sciences (The Education Sciences Reform Act of 2002,

Title I, Part E, Section 183).    Responses to this data collection will be used only for research

purposes. Findings from the interview data will be reported in summary form and individuals will

not be identified by name. However, respondents’ roles and the CC they work with may be

identified in the report, which may lead to individuals’ being identified. Other than this situation

that we have made respondents aware of, we will not provide information that identifies you to

anyone outside the study team, except as required by law.”  

Before we begin, do you have any questions about the purpose of the evaluation or our

confidentiality policy?

To start, do you mind if we audio record our session for the accuracy of our note taking?
IMPAQ International, LLC 1. Design‐Focused Interview Guide for Center Staff

    2

These questions will be asked of Center leadership (manager or director) and may include a

larger group including technical assistance providers. If respondents in a group interview

differ in their responses/opinion, those differences will be recorded in the interviewer notes.

Prior to conducting the interview, the interviewer will review relevant extant documents such

as proposals, annual performance review reports, evaluation reports, etc.  Some of the

answers to questions asked below may be in those documents. If so, the interviewer will start

the question by referring to information gleaned from the document, and will ask the

respondent to verify the accuracy of the information.     

Capacity Building Definition

1. For the purposes of your Center work with state education agencies (SEAs) [or for Content

Centers, SEAs and Regional Centers], how do you define capacity building?  What is the basis

for your definition (research, experience, other Centers)?

PROBES:  

How do you talk about capacity building among your Center staff?

Do you focus on different types of capacity building (e.g. human, organizational,

structural, etc.? If so, please tell us about that.

[For Content Center interviews: Do you focus on different types of capacity building for

Regional Centers as compared to SEAs? Does capacity building look different for Regional

Centers and SEAs?]

How do you discuss capacity building with your SEA constituents—to what extent do you

use the term “capacity building”?

2. When thinking about your work, how do you know when you have built capacity? What does

that “look like”? In other words, what observed changes in stakeholder behaviors indicate

capacity building? What are the outcomes that you expect to see?

3. Do you collect evidence to assess your progress in building capacity?  If so, what kinds of

evidence do you collect?

Theory of Action

We are interested in the assumptions behind your work with SEAs—how you decide, out of all of

the options available, what work you will do with the SEAs(and for Content Centers, the Regional

Centers) and how you monitor and adjust that work as needed. Please describe (and/or show us

graphically) your theory of change and explain how you arrived at it.  

4. How do you assess SEA needs—what information do you request, collect and review?  

PROBES:

Who isinvolved in conversations/meetings about SEA goals? [for Content Centers, ask also

about Regional Center goals and goalsthat might be generated through means other than

directly by states or Regional Centers, e.g., addressing a broad, multi‐sector or national

need]
IMPAQ International, LLC 1. Design‐Focused Interview Guide for Center Staff

    3

To what extent do your SEAs present you with their goals, and to what extent do you

suggest goal options to them?

5. How do you determine the needs you can help the SEA meet or the goals the SEA can pursue

through collaboration with your Center?  

6. How do you assessthe SEA’s current level of capacity to meet their goals? [For Content Center

interviews, also ask: is the process different in your work with Regional Centers? If yes, in

what ways?]

PROBES:

To what extent do your SEAs present you with their goals, and to what extent do you

suggest goal options to them?

If different SEA personnel present you with different goals, how do you resolve this?

7. To what extent do the states’ identified needs align with the Federal priority areas? Have you

encountered many state needs or requests for TA that did not align with these priorities? Or

that you could not addressforsome other reason?   Ifso, please explain. How do you respond

in these cases?  

8. Once SEAs have determined their goals, how do you choose your Center objectives—the

overall results your Center hopes to achieve to support the SEA’s goals?   

    PROBES:

Who is involved in conversations/meetings about Center objectives?

To what extent are the Center objectives determined by the SEA’s requests of you, and to

what extent do you suggest the Center’s objectives to the SEA?

9. How do you choose which specific strategies or projects to pursue to achieve your

objectives? What is your theory about how these strategies, activities, or projects will

contribute to achieving your objectives?   

    PROBES:

Who is involved in conversations/meetings about strategies?

What do you take into consideration when deciding upon strategies (feasibility, evidence,

availability of expertise, experience in other projects, expectations of SEA, likelihood of

building capacity, etc.)

To what extent are your projects/strategies determined by the SEAs’ requests of you, and

to what extent do you suggest the projects to the SEAs?

10. How do you customize strategies to the SEA’s existing capacity level and needs?

11. Can you describe how the local context—political, economic, etc.—affects your choice of

strategies/projects, if at all?

PROBES:  

How do you become aware of these contextual factors?
IMPAQ International, LLC 1. Design‐Focused Interview Guide for Center Staff

    4

How influential do you think context is in your decision making?  

12. How does the availability of Center resources (financial, human, etc.) enter into your choice

of strategies? [If resources do affect strategy choice: If you’re making decisions about

projects based on resources, do you involve stakeholders (SEAs/RCs) in the decision

process?  If so, how?]

The next set of questions asks about practical aspects of your project work, such as how you

monitor project activities and how you determine timeframes. These questions will help us

describe the process by which you do your work, and how your theories of actions inform the

process.  

13. How do you track and assess progress in your projects? What evidence do you collect or

review to do this?

14. How do you choose which outcomes to track for specific projects or the Center as a whole?

Can you give us any examples of outcomes that you’ve documented so far?

    PROBES:

Who is involved in decisions about outcomes?

What implementation milestones do you track, and how?

What do you do if implementation milestones aren’t met?

How do you decide on longer‐term outcomes for your Center work?  

How do you measure your longer‐term outcomes?

15. How do you determine the timeframe for a particular project? Do you have any rules of

thumb about how long projects should be?

PROBES:

How do you take stakeholder expectations and availability into account when developing

timelines? How does the SEAs’ planning process impact project timelines?

16. To what extent, if any, has your Center updated plans during a project? If so, how did you

become aware of a need to change something? Can you describe this process of changing

projects or strategies?  How often does this happen?

17. As you track progress toward your Center objectives, to what extent do you also assess

SEAs’ progress toward their goals?  

18. Have you modified your theory of change over time?  If so, please describe how and why.

19. What else would you like to share about your approach to working with SEAs [or RCs, for

Content Centers]?