U. S. Department of Homeland Security

 Office for Domestic Preparedness

 Washington, D.C. 20531

ODP Information Bulletin

No. 95 January 13, 2004

TO: All State Administrative Agency Heads

All State Administrative Agency Points of Contact

All State Homeland Security Directors

FROM: C. Suzanne Mencer

Director

SUBJECT: Guidance for developing the evaluation plan for the State Homeland Security

Strategy and Urban Area Security Strategy

The evaluation plan included in the state and urban area homeland security strategies should

outline the process by which the state will review and analyze progress made toward identified

goals and objectives. The review and analysis (R&A) process should compile key management

information, track trends that emerge as the strategy is implemented, ensure that implementation

is proceeding as it should, and make mid-course adjustments to the strategy as necessary.

The evaluation plan is a critical element of the strategy and is tied to its ultimate approval by the

Department of Homeland Security. Grantees need to demonstrate that important issues

associated with evaluating progress have been contemplated thoroughly and that a plan is in

place to make sure that implementation of strategic goals and objectives is tracked from the

beginning. Consequently, the evaluation plan should provide details about the membership of

key evaluation working groups, the frequency of working group meetings, their use of evaluation

tools, and the development of corrective action plans. The evaluation plan should consider

several key categories of questions:

• Who will be part of the evaluation process?

• What meetings have been developed and scheduled to evaluate progress?

• How will progress be evaluated?

• How will mid-course adjustments be made if goals and objectives change, if progress

toward them stalls, or if mass casualty disasters occur?

The following sections provide example questions to consider when filling in the details of your

evaluation plan. This is not an exhaustive list.

1. Who will be part of the evaluation process?

• Will a new working group be established for regular R&A?

o If so, who will be part of the working group?
• Will an existing working group be tasked with R&A?

o For example, did the state create a strategic planning working group to develop

the homeland security strategy? If so, will that same working group be involved

in the plan’s evaluation?

• Will more than one working group be involved in evaluating progress toward goals and

objectives?

o For example, would one group meet more frequently than another to manage

issues at the working level?

• What emergency response disciplines will be represented in the working group(s)?

• Will local government representatives be included?

• Will private industry and voluntary organizations be included?

• Which agency(ies) will have lead and support roles in managing the meetings and

following up on action items identified by participants?

2. What meetings have been developed and scheduled to evaluate progress?

• How frequently will working groups meet to consider evaluation issues?

• Will there be monthly R&A meetings for subject matter experts?

o If so, will they be dedicated to R&A of strategy implementation or will the

meetings address routine business matters as well?

o If not, what alternative schedule has been identified?

• Will there be regularly scheduled (e.g., quarterly, biannually, annually) presentations of

R&A findings by subject matter experts to homeland security leadership and working

group members?

• What will be the expected outcomes of any monthly, quarterly, or annual meetings?

• Have appropriate funding streams been identified to support working group meetings?

3. How will progress be evaluated?

• What tools will the relevant working groups and personnel use to evaluate progress?

• How will that information be maintained and disseminated?

• Will states continue to use the ODP online tool used during the State Homeland Security

Assessment and Strategy process?

o If so, who will manage data updates and how frequently will they occur?

o If not, what alternative tool will be used?

• What types of reports will be disseminated to various working group members to

illustrate progress or risk areas?

4. How will mid-course adjustments be made if goals and objectives change, if progress

toward them stalls, or if mass casualty disasters occur?

• How will modifications to the goals and objectives included in the homeland security

strategy be identified and implemented?

• How will corrective actions be tracked in future evaluation and R&A efforts?

If you have any questions or would like assistance with the development of your evaluation plan,

please contact your ODP Program Manager at (800) 368-6498.