**CHAPTER 12**

**EMERGENCY MANAGEMENT STANDARDS AND EVALUATIONS**

As the practice of emergency management has undergone many changes over
the past half century, so too has the conception of an emergency
manager. As an emerging profession, emergency management needs to
establish standards and evaluate compliance with those standards. This
chapter begins by examining the concept of a profession and then
identifies the process by which emergency management is moving toward
professionalization. The chapter next turns to procedures for periodic
evaluation of the local emergency management agency (LEMA) and local
emergency management committee (LEMC). This section describes some
general principles for organizational evaluation and then turns to
National Fire Protection Association (NFPA) Standard 1600, Emergency
Management Accreditation Program (EMAP), and National Incident
Management System Capability Assessment Support Tool (NIMCAST). The
chapter describes procedures for evaluating drills, exercises, and
incidents and concludes with a discussion of procedures for evaluating
organizational training and community risk communication programs.

**Introduction**

Although there are some differences in definitions, social scientists
generally consider a profession to be a group of practitioners whose
specialized education and training gives them the knowledge, skills, and
abilities (KSAs) needed to perform the tasks within a specified work
domain (Blanchard, 2004; Friedson, 2001). This definition has six
important implications. First, practitioners are a group, not just a
disorganized collection of individuals. To accomplish their objectives,
practitioners organize themselves into a professional society. Second,
the members of the professional society establish consensus on their
work domain. That is, they agree on what tasks they perform that makes
their profession different from other professions. Third, members of the
profession define the KSAs that are required to perform the tasks within
their domain. This is sometimes called the profession's *body of
knowledge*. Fourth, members of the profession define the ethical
standards that govern the performance of tasks within their domain. Some
of these ethical standards are common to all professions (e.g.,
procedures for avoiding conflicts of interest), whereas other ethical
standards are specific to that profession (e.g., prescribed and
proscribed procedures for performing specific tasks). Fifth, members of
the profession establish minimum standards regarding the profession's
body of knowledge and apply these standards to both prospective members
and current members. Sixth, professions define and apply methods for
revising their body of knowledge. For example, a professional society
might establish a committee that evaluates a new theory or procedure for
handling a specific type of problem. If the new theory or procedure is
judged to be better than the one(s) in current use, it is incorporated
into a *professional standard* that is binding for all members of the
profession.

The purpose of a professional society is to ensure these six functions
are performed and promote public recognition of its existence as an
organized group with specialized expertise. In addition, a professional
society takes action to protect its task domain from encroachment by
unorganized nonprofessionals who do not adhere to the same standards of
task performance and ethical behavior. It is important to recognize that
professional societies also take action to protect their task domains
from encroachment by other professional societies that have different
bodies of knowledge, professional standards, and ethical requirements.

The foregoing definition of a profession might seem to imply that its
members must all be extremely similar in the work they do, but this is
not the case. In fact, many professions can be viewed as composed of a
variety of specific occupations (Trank & Rynes, 2003). Each occupation
is distinct to some degree, but the members of all occupations within a
profession can be meaningfully grouped in terms of a shared task domain
and, thus, their body of knowledge and ethical standards. The reason for
the emergence of specific occupations is usually the depth of knowledge
required to practice that occupation and, not coincidentally, the
prevalence of sufficient labor market demand to support that specific
occupation.

Another important aspect of this conception of a profession concerns the
ways that professions implement membership rules to exclude the
unqualified (Trank & Rynes, 2003, p. 191). The competence of prospective
members is evaluated by some combination of education, specific
training, testing, and duration of professional experience. In many
cases, a satisfactory level of competence is acknowledged by a
certificate and, in some cases, it is indicated by a (legal) license to
practice as a professional. The primary mechanism for addressing the
competence of current members is a continuing education requirement.
Another mechanism for addressing the competence of current methods is a
review board that has the power to revoke certification or licensure.
These boards usually act when someone files a complaint alleging a
member's substandard performance. Frequently, these boards review
allegations of ethical violations as well as allegations of
incompetence.

Mature professions that have developed consensual standards tend to
place a heavy emphasis on education, usually in the form of a degree
from an accredited college or university. Such degree programs are
overseen by an external accrediting boards that periodically review the
content of the coursework and the qualifications of the faculty. These
boards grant accreditation only to programs that meet the professional
society's standards. In some cases, there are separate professional
societies for a field's practitioners and educators. In such cases, the
two professional societies negotiate a mutually agreeable set of review
standards and site review committees draw members from both societies.

This point raises an important distinction between education and
training. Education imparts broad principles of subject matter that can
be applied in a wide variety of situations. Training has a narrower
focus that aims to develop competence in performing specific tasks in
well defined situations. In less mature professions, where degree
programs and specific accrediting bodies have not evolved, training
becomes the "marker" by which practitioners can be identified. Such
training might be multidimensional; that is, one may require training in
a variety of specific skills to adequately claim professional status.
However, such training is usually seen as bounded in time. This reflects
a recognition that the specific problems a professional faces might
change, so the specific KSAs a professional needs will also change.
Consequently, training programs usually demand followup or refresher
training, often at specified intervals.

Whether prospective members prepare for entry by means of a broad
education or specific training programs, certification (or, more
formally, licensure) provides assurance that an individual has acquired
the relevant knowledge and mastered the required skills to meet
professional standards of performance. In addition, certifications
routinely demand that candidates demonstrate their knowledge in some
structured format, almost always by written examination. Finally,
performance tests are often required, sometimes in the form of an
extended period of job performance supervised by a certified
professional. Thus, the legitimacy of any certification depends upon the
authority of the association or organization that grants it.

Education and training also have a significant impact on the second
marker for professions---the evolution of the knowledge system that is
used to define a field of endeavor (Trank & Rynes, 2003). Years ago,
Mosher (1968, p. 122) stated "the perspective and motivation of each
professional are shaped by the lens provided...by professional
education, professional experience, and by professional colleagues."
Hays and Reeves (1984, p. 137) emphasize that professions have an
"evolving and agreed upon body of knowledge" and worldview. The body of
knowledge is often science based, but this is not necessarily the case
for all professions (e.g., religious professions). The important point
is that the knowledge is systematic and that there are consensual rules
for generating, evaluating, and using that knowledge. The body of
existing knowledge and the rules for developing new knowledge
"constitutes the foundation from which professionals innovate and extend
the knowledge base" (Trank & Rynes, 2000, p. 191).

Finally, the third defining feature of professions rests in an
ideological and ethical component. Thus, in addition to substantive
knowledge, professions socialize members to act in terms of professional
norms and perspectives (the "lens" mentioned by Mosher) that might
differ from the views of either the public or the management of
organizations in which the professional is employed. As Friedson (2001,
p. 122) indicates, the ideology of a profession provides members with "a
larger and putatively higher goal that may reach beyond that of those
they are supposed to serve." This attitude defines the professional
identity or professional culture that supports the use of professional
discretion in identifying problems and formulating solutions.
Particularly in the past two decades, professional ideologies have been
embodied in ethical codes. These statements stand as articulations of
the values embraced by members of the profession. Ethical codes not only
encourage compliance as proof of professionalism, but typically describe
the punishments for those who fail to comply.

These three features of professions---membership certification,
organized body of knowledge, and ethical standards---provide a framework
within which to discuss emergency management as a profession. There is a
community of professionals that creates, changes, and applies a
professional body of knowledge. This community also defines the required
education and training in the body of knowledge. Finally, it enforces
its professional and ethical standards.

# Emergency Management as a Profession 

Although most emergency managers and many people in related fields would
agree that emergency management is a profession, they might disagree
about the extent to which it is a *mature* profession (Oyola-Yemaiel &
Wilson, 2005). Perhaps at this point it is most useful to view emergency
management as a developing profession, moving in the direction of
achieving the three professional benchmarks of membership certification,
organized body of knowledge, and ethical standards. After all, the
concept of emergency management has been undergoing very rapid change.
As its history was traced in Chapter 1, emergency management meant civil
defense or wartime attack preparations as recently as the 1980s. Since
then, the vision of the field has become more firmly represented in the
management of natural and technological hazards and, most recently, in
addressing terrorist threats. Over this same period, emergency
management has come to regard hazard mitigation and disaster recovery as
being as important as emergency preparedness and emergency response. In
addition to changes in the vision and practice of emergency management
have come changes in the threat environment and the tools available for
dealing with those threats. This very fluid situation has slowed the
development of consensus on the definition of the field and particularly
in defining the body of practitioners.

Furthermore, the vision of practitioners has radically changed over the
past fifty years. Perry (1985) has pointed out that, if emergency
managers were distinguished from those delivering police and fire
services, the jurisdictional emergency management role was often
embodied in the "Civil Defense Director". Incumbents in this role tended
not to have specialized training beyond some experience in the military
(usually retired), often were not college educated, and usually were not
prominent in jurisdictional administration. Thus, according to Perry
(1985, p. 135), "the vision was one of a largely invisible person,
presumably attached in some way to defense authorities (whoever they
were), charged for the most part with civil defense duties (whatever
they were)". Blanchard (2004) notes this vision has given way to a
career oriented, college educated professional who has acquired
knowledge from the physical and social sciences, planning, and
engineering. Emergency managers "apply science, technology, planning,
and management to deal with extreme events that can injure or kill large
numbers of people, do extensive damage to property, and disrupt
community life (Drabek, 1991a, p. xvii). More specifically, as previous
chapters have indicated, the knowledge, skills and abilities (KSAs)
required of an emergency manager include:

-   Knowledge about a wide range of natural and technological hazards,

-   Knowledge about a diverse set of methods for assessing their
    communities' vulnerability to those hazards,

-   Skill in managing emergency management programs in hazard
    mitigation, emergency preparedness and response, and disaster
    recovery preparedness and implementation,

-   Knowledge of the ways in which different disciplines and agencies
    can contribute to emergency management programs,

-   Sufficient knowledge of the policy process to be an effective policy
    entrepreneur, and

-   Skill in organizing, leading, and coordinating the performance of an
    emergency management organization.

Another important aspect of emergency management is that its
practitioners can be found in a wide variety of settings. The principal
distinction here is between public sector and private sector emergency
managers. Although there are many similarities in the duties of these
two groups, there are important distinctions as well. Private sector
emergency managers have a more narrow scope, usually focusing on a
single business, site, or industry (Elliott, Swartz & Herbane, 1999).
Except when mandated by government (as in the case of nuclear power
plants), private sector emergency managers are responsible for their
facilities' employees, but not the public at large. Public sector
emergency managers address the needs of governments themselves,
government employees, the jurisdiction's citizens, and private sector
organizations within their communities (Perry & Lindell, 1987).
Particularly since the 9/11 terrorist attacks, the concern of government
emergency managers with emergency preparations of private organizations
has been high (Perry & Lindell, 2003). Also, governments engage in
emergency management as an obligation and may be held legally liable for
specific failures to recognize and plan for environmental threats. This
liability is both to citizens and to other governments. Although private
sector organizations might indeed be held liable in courts for personal
and property damages generated by their operations, they have much more
limited responsibility to engage in emergency management (Lindell, in
press). Private sector and public sector emergency managers operate with
differing scope, resources, obligations, and public accountability.

Public sector emergency managers also differ in the size of the
jurisdictions in which they operate. Those in smaller jurisdictions
report more frequent contacts with elected officials and, surprisingly,
disproportionately lower frequency of contacts with Red Cross personnel
(Drabek, 1987). Emergency managers in smaller jurisdictions also tend to
have contact with higher level agency personnel (e.g., police chief
rather than a police lieutenant), have less formalized interagency
agreements, fewer joint programs with other agencies, and fewer
overlapping memberships, but higher levels of perceived
interorganizational coordination (Drabek, 1987).

The final distinction discussed here addresses public sector emergency
managers who operate at different levels of government. In large part,
this concerns issues of scope and context (intergovernmental
relationships). Overlain upon these issues is the notion that
jurisdictions and governmental levels vary in the extent to which
emergency management is officially constituted and allocated resources.
At the municipal level, emergency management rarely exists as a separate
department; is often assigned as a collateral duty, and is frequently
located in an "emergency management coordinator's office" within the
fire or police department. Sometimes much, if not all, of the municipal
emergency management function is relinquished to a county organization
(such as emergency management or Sheriff's Office). Indeed, local
emergency management functions are highly variable in their presence as
well as in their degree of success (US General Accounting Office, 2003).
Emergency management at the municipal level is closest to the disaster
impact and the people affected. At the same time as local emergency
managers are subject to many federal and state mandates, they have the
fewest resources in the intergovernmental hierarchy and must frequently
rely upon other governments, outside experts, the media, and even
private sector organizations for resources. In county government,
emergency management constraints are similar to those experienced at the
municipal level.

State and federal emergency managers occupy positions that are, for the
most part, quite different from those of local emergency managers. In
the United States, each state has an emergency management department (or
a division of a larger department), as well as other departments that
house emergency management capabilities and responsibilities. State
emergency management agencies provide technical guidance and financial
support to LEMAs, conduct statewide hazard/vulnerability analyses, and
evaluate LEMA performance. At a federal level, the FEMA works with other
federal agencies such as EPA and DOT to define policies, develop
programs, and provide technical and financial assistance. Such resources
are often passed through the states to the local level. Thus, as
discussed in Chapter 2, emergency managers at state and federal levels
tend to operate more at a policy and program level. Only a few of these
programs involve direct incident response (e.g., Coast Guard Strike
Teams). Indeed, NIMS emphasizes that federal and state resources "flow
downward" into the implementation structures created by local emergency
managers. At federal and state levels, the job of emergency manager
emphasizes management of programs and coordination of organizations.
Certainly the ability to identify experts and technology and link them
with programs and policies through the intergovernmental hierarchy
remains important. But as Drabek (1990) indicates, critical skills also
include agenda control, constituency support building, budget, and
financial analysis expertise, coalition building skills, and innovation
and entrepreneurial skills. Furthermore, time horizons may vary for such
officials in terms of their incumbency as appointees or civil servants.
Moore (1995) offers an extensive generic description of other managerial
skills important in state and federal managers.

The picture of the emergency management profession that emerges from
this discussion is multifaceted. Clearly the core notions involve
environmental hazards and their management. Emergency managers are
knowledgeable about the full range of natural and technological threats
(an "all hazard" orientation), methods of assessing community hazard
vulnerability, and methods of managing this hazard vulnerability (an
"all phases" orientation). Moreover, their "all phases" orientation
distinguishes emergency managers from fields that specialize in specific
phases. For example, urban/regional planners and civil engineers focus
on issues related to HVA, hazard mitigation, and disaster recovery.
Conversely, police, fire, and EMS focus more on emergency preparedness
and response. Finally, their focus on managing the community's response
in abnormal conditions---emergencies, disasters, and
catastrophes---distinguishes emergency managers from other types of
public administrators.

Accordingly, emergency managers must serve as generalists who integrate
the contributions from many different technical disciplines. Emergency
managers don't need to be competent in all technical skill areas (hazard
mapping using GIS, toxic plume projections, slope stability analysis,
seismic structural analysis), but instead must understand how these
different disciplines fit into the mosaic of hazard management. The
emergency manager is a generalist who knows where to find and how to
request the services of specialists. Specifically, which hazards are
managed is important but not definitive because emergency managers are
found in different regions of the country and at all levels of
government as well as the private sector. In addition, like any other
profession, emergency management requires communication and
organizational skills, including strategic planning and management,
political management, and human resources management. Blanchard (2003)
identified an extensive list of competencies and skills appropriate for
emergency management professionals that has more recently been the
subject of a survey of practicing emergency managers (Spiewak, 2006).

**Table 12-1.** Emergency Managers' Top 20 Core Competencies

  ---- ------------------------------- ---- ------------------------------
  1    Planning                        11   Operations and Procedures

  2    Hazard Identification, Risk     12   Finance and Administrative
       Assessment, and Impact Analysis      

  3    Direction, Control, and         13   Training
       Coordination                         

  4    Laws and Authorities            14   Mitigation

  5    Exercises, Evaluations, and     15   Recovery and Business
       Corrective Actions                   Continuity

  6    Communications and Warnings     16   Logistics and Facilities

  7    Hazard Mitigation               17   Crisis Communications

  8    Resource Management             18   Project Management

  9    Continuity of Operations/       19   Evacuations
       Continuity of Government             

  10   Mutual Aid                      20   Risk Management
  ---- ------------------------------- ---- ------------------------------

If one accepts this picture of the emergency management profession and
acknowledges the features of professions described by Trank and Rynes
(2003), a logical question centers on what emergency mangers need to do
to continue to develop as a profession. There are probably as many
opinions regarding "what we need" as a profession as there are emergency
managers, with no definitive answers. Indeed, the question of
professionalization has received much attention in recent years, with
the discussion led in large part by the FEMA Higher Education Project
and the International Association of Emergency Managers (Ditch, 2003).
Although these discussions have examined many dimensions of the
profession and benefits of professionalization, the following discussion
will address three avenues for professional growth that have generated a
significant amount of agreement.

The first activity that reinforces development as a profession is to
continue *to define emergency management as a profession and a
distinctive professional identity*. Paths to this outcome include
enhancing a professional ideology and ethics. Although emergency
management is inherently interdisciplinary and must serve as an umbrella
for many technical fields (an inclusive task), it also is necessary to
be somewhat exclusive in identifying the unique features of emergency
management. This can be accomplished by distinguishing emergency
managers from emergency responders, urban/regional planners, and
others---without placing any field "above" another.

An important part of cementing a professional identity is the growth of
professional associations and the encouragement of participation by
emergency managers. There are many specialized associations that are
available to emergency management professionals that range in emphasis
from narrow subject matter (for example, the Association of Contingency
Planners) to broad subject matter (National Association of Environmental
Professionals). There are at least three professional associations that
appeal directly to those engaged in what is described here as the
emergency management profession. The International Association of
Emergency Managers (IAEM) has a long history beginning as the US Civil
Defense Council and then becoming the National Coordinating Committee on
Emergency Management before adopting its present name and structure. The
IAEM sponsors meetings, continuing education opportunities, and a
professional certification program. Similarly, the National Emergency
Management Association is open to membership for state emergency
management directors and others interested in emergency management at
that level of government. Finally, the Disaster Preparedness and
Emergency Response Association is international in scope, embraces both
public and private sector, and sponsors meetings, training and a variety
of school based educational programs. An important notion is that
participation in associations provides opportunities for learning, for
networking with others, and for gaining a "sense of self" as an
emergency manager.

Defining the collective identity is also accomplished through
participating in continuing education and professional development
programs. Continuing education is available through colleges and
universities, private corporations, and government sponsorship. Many of
these programs are listed on the FEMA Higher Education Web site
(training.fema.gov/emiweb/edu). In government, professional development
opportunities are available through the programs operated by FEMA's
Emergency Management Institute. There is a *Professional Development
Series* and an *Advanced Professional Series*, each of which offers not
only specialized training, but also a certificate for completion of a
full course of study.

The concept of professional ethics for emergency managers has not as yet
received as much attention as other issues related to professional
growth. The IAEM emphasizes ethics among its members and has adopted a
formal ethical code. The IAEM code emphasizes three components. The
first part focuses upon the need to respect people (particularly
citizens), laws, regulations, and fiscal resources. The second issue
emphasizes commitment to engendering trust, acting fairly, and being
effective stewards of resources entrusted to their management. Finally,
the code asserts that members should embrace professionalism founded on
education, safety, and protection of life and property. This particular
statement of ethics is generic, but captures key issues normally
addressed in ethical codes. One interesting feature of this code is the
apparent concern for acting in concert with the regulations and
resources of the organizations served. Moore (1995) argues that such
stands place the administrator in the role of "faithful servant" of the
elected and appointed managers of the organization. As they consolidate
their position as experts, some professional groups move away from this
position to a more assertive role in management and focus ethical
aspects of service on more discipline specific principles and evaluative
rules. This issue notwithstanding, the IAEM, by adopting and publishing
an ethical code, is presently leading the discussion of ethics in the
field of emergency management. It should be noted, too, that because
emergency management is a diverse field, practitioners are likely to be
subject to a variety of ethical codes as a function of their membership
in and certification by different professional associations and
organizations. For example, emergency managers who are Certified
Environmental Professionals are subject to the Academy of Board
Certified Environmental Professionals Code of Ethics and Standards of
Practice for Environmental Professionals. The Business Continuity
Institute also maintains a code of ethics for those who accept
membership or certification. Because many emergency managers are
employed by some level of government, they are subject to the formal
ethical codes of their jurisdictions. In addition, many are members of
the American Society for Public Administration, which has an extensive
ethical code. These overlapping sets of ethical guidelines rarely
present problems and, indeed, are common for professionals. It remains
important, however, that emergency managers continue to attend to the
issue of ethics and to develop publicly visible ethical codes. This
practice stresses for the public, and also the organizations in which
professionals are embedded, that emergency management is a profession
whose members are prepared to act independently in keeping with their
professional identity and standards.

A second activity that is critical for the development of a profession
of emergency management is to *build and grow an identifiable body of
knowledge for practitioners.* Certainly as an interdisciplinary, science
based profession, emergency management will always use bodies of
knowledge generated by others. The management of specific environmental
threats is a function of the state of technology, which is a function of
the state of knowledge in a variety of scientific disciplines. For
example, the state of knowledge in seismology does not currently provide
a forecasting technology capable of supporting preimpact evacuation from
earthquakes in the way that meteorological forecasting supports
preimpact hurricane evacuations. Although such bodies of knowledge have
significant roles in the conduct of emergency management, the
professional body of knowledge described in previous sections pertains
specifically to emergency management. In particular, emergency managers
need to develop an organized body of knowledge regarding emergency
management strategies and tools. Such a body of professional knowledge
would be particularly useful in supporting research directed at
evaluating the wide variety of tools emergency managers use to
accomplish their objectives. The National Research Council (2003, p. 5)
has argued that "education and research, in addition to the role of
technology...play vital roles in the advancement of emergency
management." To date, however, the research literature evaluating the
effectiveness of programs and policies is very sparse (Lindell & Perry,
2001). Drabek (1987, 1990, 2003) has been one of the few researchers who
has consistently conducted studies of organizational strategies
available to emergency managers.

Despite their need to keep abreast of the advancing state of knowledge,
emergency managers are practitioners rather than researchers. No one
expects that, in addition to their other duties, emergency managers will
have the desire or resources to conduct their own research. However,
there is a longstanding tradition of hazards and disasters research in
the academic community. It is these researchers that conduct the needed
studies. The role of the practicing emergency manager should be (when
feasible) to endorse the need for well designed research, participate in
research when possible, identify areas needing research, critically
examine the research findings, and put the resulting knowledge to work.

Of course, the quality of the body of professional knowledge developed
from research depends upon successful linkages between practitioners and
researchers (Mileti, 1999). There has long been an awkward relationship
in the emergency management community between practitioners and
researchers. This is not unique to emergency management, but also exists
in other professions such as medicine, public health, business
administration, urban planning, and public administration. As Fischer
(1998) has pointed out, the reward system for researchers is quite
different from the reward system for practitioners. This leads each
group to have distinct goals and interests. For the professional growth
of emergency management, however, it remains critical that the knowledge
supplied through research be meaningful, explicitly assembled and
available, and used in practice.

There are many positive developments resulting from the growth of an
emergency management body of knowledge. The increasing involvement of
emergency managers in higher education and the evolution of emergency
management degree programs will shrink the gap between practitioners and
researchers. Contacts between professional associations, such as the
IAEM, with research organizations, such as the International
Sociological Association's Research Committee on Disasters, will enhance
communications regarding research priorities and research results.
Particularly as emergency management degree programs and departments
develop in colleges and universities, the problems of the "academic
legitimacy" of disaster research cited by Fischer (1998) will decrease.
The proliferation of programs that actively promote the interaction of
practitioners and researchers---such as the Natural Hazards Research and
Applications Information Center at the University of Colorado,
Boulder---will further enhance communication and cooperative projects.

It is difficult to overstate the importance of this body of knowledge in
establishing emergency management as a profession. An established body
of tested knowledge is a critical part of the claim that emergency
managers are capable of making credible contributions to decisions about
how jurisdictions should manage community vulnerability to environmental
hazards. The accepted body of professional knowledge forms a defensible
basis for practice and, when improvisation is required, it is the base
from which one innovates. The extent to which emergency managers are
skilled in using the body of professional knowledge provides a standard
for evaluating their performance. Moreover, it forms the basis for the
design and execution of training and education, including higher
education and degree programs. In asserting their credibility to the
public and other decisionmakers, the body of knowledge is what uniquely
sets emergency managers apart from other professions.

Finally, the third critical feature for advancing emergency management
as a profession involves *asserting control over the body of
professional knowledge and its dissemination*. This goal can be
accomplished through establishing specialized training, certifications,
and academic degrees. At present, there are many definitions of what
constitutes the body of knowledge as it is taught in colleges and
universities (Phillips, 2005). The IAEM and FEMA Higher Education
Project continue to be the most prominent actors with regard to this
issue. The process of training and credentialing is perhaps the
strongest public indicator to set emergency management apart as a
profession. To control a body of professional knowledge, however,
boundaries must be drawn regarding what information lies within it and,
consequently what are the essential elements of training and education.
Thus, the first step in taking control of a body of knowledge is the
presence of legitimate associations and organizations to define it. The
FEMA Higher Education Project has already begun the process of examining
how an accreditation system might be developed for academic degree
programs (Walker, 1998). Until an accrediting body is officially
established, the Higher Education Project has undertaken programmatic
activity to establish a direction for specifying the emergency
management body of knowledge.

Finally, the FEMA Higher Education Project has undertaken three methods
for defining and influencing the use of the emergency management body of
knowledge. The first method is direct study of emergency management
practice and the development of specific listings of competencies for
emergency managers (Blanchard, 2003). The second method is to develop
specific courses and course outlines with emergency management content
and to make them freely available through its Web site
(training.fema.gov/emiweb/edu) to any interested party. The content of
these courses has been developed by contract with prominent emergency
managers and disaster researchers and reviewed by panels of experts.
Such materials support teachers of individual classes in a variety of
academic departments. More important, however, the courses have
collectively become a basis for detailed curriculum development and for
the design of degree programs in emergency management.

The third method of defining the emergency management body of
professional knowledge has been to compile and update a directory of
college level programs, including those that offer undergraduate and
graduate degrees, certificates, academic minors, and diplomas. Even the
simple availability of this information encourages those who inquire to
pursue further enhancement of their expertise. The presence of the
catalog itself serves to inform both those inside and outside emergency
management that there is a basis in education for practice. Moreover,
the FEMA Higher Education Project makes available sample syllabi for a
wide range of classes on emergency management topics and has developed
full college courses with instructor guides, readings, exercises, field
trips, and student notes.

# Emergency Management Education

Certifications often include educational requirements but are not
intended to take the place of academic degrees, nor are academic degrees
intended to replace certifications. The educational requirement
establishes that the prospective professional has acquired a broad range
of KSAs upon which job specific training and experience can build. The
growth of associated academic degree programs represents significant
maturing of a profession. There are two aspects to this development.
First, professional degree programs are ways of acquiring relevant
principles and procedures from different *theoretically organized*
disciplines and developing this information into an organized body of
knowledge that can be applied to solving problems. Thus emergency
management degree programs acquire material from the physical and social
sciences, as well as from other professions such as architecture,
engineering, planning, public administration, and public health. In
turn, the emergency management programs must organize this information
into a coherent body of knowledge that addresses the problems that
confront emergency management. Consequently, there must be a depth and
breadth to collegiate education that is very difficult to acquire in
training or job environments. Furthermore, an academic education
emphasizes principles, models, and theories, whereas training focuses on
specific tasks and the appropriate methods for performing those tasks.
It is this knowledge of broad principles that develops what is often
cited as a defining feature of an emergency management
professional---the ability to *improvise* solutions to novel problems
that were not anticipated in planning and training (Drabek, 2003; Kreps,
1991). It is impossible to enumerate every conceivable problem that will
confront an emergency manager and devise specific procedures to address
each of them. On the other hand, knowledge of the principles of human
response to warning messages enables an emergency manager to, for
example, construct and disseminate warnings to residents threatened at
2:00 a.m. by a volcanic mudflow that derailed a train and spilled
sulfuric acid into a stream that feeds a lake along which a town is
situated.

The second important aspect of the development of academic degree
programs in emergency management is that they constitute an explicit
link between those who practice in the field and the academic research
community. Practitioners use the body of knowledge frequently but it is
usually the academic community that conducts research. Some university
based centers have a long history of research on emergency management,
hazards, and disasters. The most prominent of these are the University
of Delaware Disaster Research Center and the University of Colorado
Natural Hazards Research and Applications Information Center. In
addition, there are at least 28 other such university based centers
(Tierney, et al., 2001). However, to support an entire profession,
particularly one as diverse as emergency management, the academic
connection needs to be both more extensive and more complete. Degree
programs involve the academic community simultaneously in the process of
disseminating, refining, and extending emergency management knowledge.
This creates the opportunity for faculty to interact with students and
practitioners in the teaching process and to integrate that experience
into research designs.

The path to developing academic support for a profession is long and
multifaceted. There must first be sufficient demand for training and
education to capture the attention of educational institutions. For
emergency managers, this means the job market must be large enough to
accommodate the number of people who are seeking employment. The process
of obtaining credentials for professions with small staffing levels
leans heavily upon training and certifications because these types of
programs can be sustained on relatively small volume. In the past
decade, the number of jobs for emergency managers has begun to increase
to levels high enough to provide a market for academic programs.
Typically, a few institutions recognize the needs of an unserved niche
in the labor market and begin the process of serving that market niche.
In the case of emergency management, the University of North Texas was
the first to initiate an emergency management degree program (in 1983)
and was followed somewhat later by Thomas Edison University, the
Rochester Institute of Technology and the University of Wisconsin. It
took over 15 years for the majority of the other programs to respond to
the need for educating future emergency managers.

Market recognition is normally a gradual process but, in the case of
emergency management, this process has been accelerated by the FEMA
Higher Education Project. Particularly through its Web site and by
active advocacy for the profession in higher education circles, the
Higher Education Project has made it known that there is a wide market
for emergency managers and that many organizations are interested in
hiring people with emergency management degrees. The Higher Education
Project has tracked and made public the types and institutional homes of
degree programs, as well as the challenges they have faced. This
information speaks not just to the audience of colleges and
universities, but also tells practitioners of the availability of
educational opportunities and tells the public and other professionals
that there is a firm intellectual grounding for emergency management.

This visibility does not allow the profession of emergency management to
escape the gradual nature of the evolution of degree programs however.
Darlington (2000) studied a sample of 1,886 schools and found that 11.6%
offered at least one course with emergency management content. She also
reported that less than 1% of colleges offered a Bachelor's degree in
emergency management and 1.5% had an emergency management related
postgraduate degree program. To date, there is only one program offering
a Ph.D. in emergency management (North Dakota State University). When
evaluated as a proportion of all universities and colleges, these gains
seem small. However, considering that there were no emergency management
degree programs in the United States prior to 1983 provides a
perspective on how far educational opportunities have progressed.

To serve the profession as it has been described in the previous
sections, degree programs must achieve some level of standardization.
That is, if the value of an emergency management degree is to be
recognized in the marketplace, there must be some assurance that
graduates of degree programs can be reliably assumed to have acquired
the body of professional knowledge. Moreover, the body of professional
knowledge should be standardized across university and college degree
programs. As noted earlier, the body of professional knowledge is
defined by the lessons learned from practitioners' experience. This
information is disseminated to designers of academic programs in a
variety of ways, including the emergency management expertise acquired
by the faculty, review of curricular materials by professional advisory
boards, and guidelines distributed by authoritative bodies such as
government agencies and professional associations. Both the FEMA Higher
Education Project and the IAEM have taken active roles in shaping the
vision of emergency management KSAs.

At some point, these efforts to standardize and establish the content of
emergency management programs will mature into a system of accreditation
for degree programs in emergency management (Oyola-Yemaiel & Wilson,
2005). This development is analogous to the development of certification
for individual emergency managers. Thus, an independent accrediting body
is established to devise standards and create a review process for
programs. Typically, such bodies or commissions are national or
international in scope and, in addition to developing standards and
conducting evaluations, foster information exchange across a profession.
Walker (1998) has provided an extensive history and discussion of
accreditation, including comments on the development of an accreditation
body for emergency management---similar to the International Fire
Service Accreditation Congress. This type of accreditation focuses upon
academic program content and performance, as distinct from institutional
accreditation that is conferred upon colleges and universities
themselves. Program accreditation forms an explicit linkage of the body
of professional knowledge with academic institutions.

# Emergency Management Training and Certification

The development of training courses---particularly those devised and
sanctioned by professional associations---is a first step toward
consolidating the body of emergency management knowledge and
disseminating it to practitioners. The next step in this process is to
establish a program that integrates multiple courses into a certificate.
Particularly when academic degrees are unavailable, certificates are a
means of ensuring practitioners have obtained technical training and a
competent authority can verify they have acquired the necessary
expertise. Because emergency management is multidisciplinary, there are
professionally relevant certificates available in a wide variety of
substantive areas. Each of these programs has its own audience and level
of credibility. Many of these certificates are even older than the term
emergency management and some test broader skills than emergency
management whereas others test narrower skills. For example, the
National Association of Environmental Professionals (now the Academy of
Board Certified Environmental Professionals) has offered the Certified
Environmental Professional Program since 1979. This is a very broad
certification that offers five functional areas---environmental
assessment, documentation, operations, planning, and research and
education. This is a challenging certification because the minimum
requirements include a Bachelor's degree and nine years of professional
environmental experience.

Business continuity planning offers a wide range of certificates from a
variety of organizations. According to Mallet (2002), the two principal
sponsors are the Business Continuity Institute (BCI) and the Disaster
Recovery Institute International (DRII). BCI offers a series of
progressive certifications that are keyed to levels of institute
membership. DRII offers three graded certifications---Associate Business
Continuity Planner, Certified Business Continuity Professional, and
Master Business Continuity Professional---based upon experience and
skills demonstrated through testing. A number of universities and
colleges also offer certifications in contingency planning and recovery
planning. The University of Richmond (Virginia) offers four levels of
certification for a Recovery Planner: Associate, Certified, Senior and
Master.

Perhaps the greatest number of certifications is available in the area
of security. Founded in 1955, the ASIS International Foundation has
offered the Certified Protection Professional program since 1977. In
2002, it instituted two additional programs, Professional Certified
Investigator and Physical Security Professional. Other established
certifications in this area include the Certified Information Systems
Security Professional, the Systems Security Certified Practitioner, the
Certified Information Systems Auditor, the Security Certified Network
Professional, the Security Certified Network Architect, and the Global
Information Assurance Certification. The Association of Certified Fraud
Examiners offers the Certified Fraud Examiner program.

If we move to a slightly broader concept of security, Blanchard (2004,
p. 43) reports 47 certification or diploma programs offered by colleges
and universities in different aspects of emergency management. There are
also 21 programs offering recognition or certification in areas called
Homeland Security, Homeland Defense, Terrorism, and Critical
Infrastructure Protection Programs, as well as nine international
disaster management and humanitarian assistance programs. In addition,
there are dozens of new programs under development.

In the rapidly changing environment of emergency management and homeland
security, one must exercise caution regarding the pursuit of
certification programs. Kaplan (2004, p. 1) points out that with the
"pentup demand for trained staff, the certification industry---those
companies that administer or provide training for exams---has created a
bevy of new certifications." Her argument is that, although most
practitioners recognized the certifications relevant to their field only
a handful of years ago, now there are so many it is difficult to
determine which represent meaningful certification and which are
equivalent to academic diploma mills. Indeed, she found programs that
provide certification based on experience immediately after the
processing fee is paid. She warns practitioners to register only for
certification programs that have been in existence for an extended
period of time, have independent boards of examiners, have established
training and education programs, and are endorsed by relevant
professional associations. Although it is sound advice to carefully
scrutinize the quality of a certification program before registering,
one can also seek guidance from the National Fire Protection Association
Standard 1000 *Fire Service Professional Qualifications, Accreditation
and Certification Systems*. This standard identifies requirements for
accrediting bodies and certifying entities.

The critical certification and professional credential for practicing
emergency managers is the CEM (Certified Emergency Manager), offered
through the International Association of Emergency Managers. The CEM
program was established with the professional goal of increasing and
maintaining standards of knowledge, skills, and abilities among
emergency managers. In 2002, 560 individuals had received the CEM, which
is renewed on a five year cycle based on continuing education and
service (Ditch, 2003). More than 60 percent of current CEMs have held
certification more than five years. The CEM program is overseen by a
Certification Commission composed of emergency managers from a variety
of areas (government, allied fields, the military, and private
industry), as well as FEMA and several professional associations.

The process of becoming a Certified Emergency Manager involves four
phases---completing an application, fulfilling credential requirements,
passing the examination, and obtaining recertification after five years
(Ditch, 2003). The credentialing requirements address education,
training, and experience. One needs to have a Bachelor's degree to
advance to the CEM, although the substantive area of the degree is open.
Applicants are required to have a minimum of three years of employment
in emergency management (although it is possible to substitute
additional years of experience for years of college education, at the
rate of two years of experience for one year of education). One must
also have gained, independent of job experience, experience in
management of a disaster event or have assumed a substantial role in
managing a full scale disaster exercise. Each applicant must also submit
three professional references. The credential requirements also specify
that the applicant must have completed 100 hours of emergency management
training and 100 hours of general management training. There is a 25
hour limit placed on training in any single area, and the collective
training experience should cover all four phases of emergency
management: mitigation, preparedness, response, and recovery. Applicants
must also demonstrate "contributions to the profession" in at least six
specific areas of activity including teaching, publishing, serving on
boards or commissions, course development, membership in professional
associations, speaking experiences, state certifications, or assuming
leadership roles in the profession. Finally, all applicants must
complete a technical essay on comprehensive emergency management. After
meeting the credential requirements, the next step is to successfully
complete a comprehensive examination. Following the receipt of
certification, CEMs must recertify on a five year calendar.

The CEM program is the only certification available that explicitly
addresses the full range of knowledge, skills, and abilities that
embrace the profession of emergency management. Although other
certifications (including those described in this chapter) measure
important segments of emergency management knowledge, it is only the CEM
that assures competence in comprehensive emergency management and
integrated emergency management systems. Particularly in the past five
years, the CEM has come to be widely accepted by those outside the
emergency management profession as an indicator of expertise. Thus, it
is common that job advertisements for municipal, county, and state level
emergency management positions now ask that applicants possess the CEM
or acquire it within five years of accepting employment. Furthermore,
other professional associations recognize the CEM as either a criterion
for membership or evidence of expertise. These groups include the
American College of Contingency Planners, the American Society of
Professional Emergency Planners, and the US Department of Defense.

# Evaluating LEMAs and LEMCs

Evaluations of LEMAs and LEMCs are logical extensions of the procedures
advocated for conducting personnel performance appraisals (Cascio, 1998;
Schmitt & Klimoski, 1991). In addition, recent years have seen the
development of specific standards for evaluating emergency management
agencies. These efforts have culminated in the National Fire Protection
Association Standard 1600, the Emergency Management Accreditation
Program, and the National Incident Management System Capability
Assessment Support Tool. Each of these topics is discussed below.

*General Principles of Organizational Evaluation*

Periodic performance appraisals make a critical contribution to the
performance of any organization by providing a systematic review of
performance on the job (Cascio, 1998; Schmitt & Klimoski, 1991).
Evaluation of emergency management organizations can be difficult
because disasters are rare events. Consequently, it might seem that
there is rarely anything to evaluate.

In fact, there are four principal questions that should be addressed in
the evaluation of emergency management organizations. First, when should
it happen? Second, who can do it? Third, what should be evaluated?
Fourth, how should it be done? With respect to the first question, *when
should it happen*, emergency managers should review the performance of
their organizations at least annually. The most important reason for
annual evaluations is to meet the needs of the budget cycle. A positive
evaluation of organizational performance can provide substantial support
for a requested budget increase or protection against a budget cut.

With respect to the second question, *who evaluates*, emergency managers
should consider who has the information on what is desired and what has
been done. The emergency manager should always participate in the
evaluation, as should the members of the LEMA and LEMC. This is because
they know the organization's goals and have much information about its
performance. In many cases, higher level elected and appointed officials
should also play a role in the evaluation because they will make
decisions about agency budget allocations and will eventually
participate in the policymaking process during disasters and
catastrophes.

With respect to the third question, *what should be evaluated*,
emergency managers should seek to rate performance on data that meet
three conditions. First, the data must be available within the time
period in which the appraisal is being conducted, relevant to
organizational performance, and comprehensive. Data availability can be
a problem if an organization works on projects that take years to show
results. For example, a risk communication program might easily take
more than a year to conduct and even longer to produce measurable
changes in households' emergency preparedness. Consequently, performance
must sometimes be evaluated on intermediate results rather than final
outcomes. In addition, evaluation criteria should be comprehensive to
ensure all parts of the organization's performance are being measured.
In many cases, the short-term impacts of an organization's performance
are readily recognized, but the equally important long-term consequences
are not. For example, emergency managers might face short term pressure
to meet with public safety personnel (fire, police, and emergency
medical services) to ensure the EOP is updated, but meeting with land
use or community development personnel to coordinate the development of
a preimpact disaster recovery plan can easily be overlooked.

With respect to the fourth question, *how should it be done*, emergency
managers should begin by ensuring organizational performance will be
measured using an instrument that addresses the full range of an
organization's responsibilities. Many times, LEMAs can be evaluated on
their success in managing their budgets and reducing their capability
shortfalls. It also is advisable for emergency managers to collaborate
with other LEMA and LEMC members in setting objectives for the coming
year. These objectives should be specific and measurable so both of them
can determine at the end of the year if the objective was accomplished.
Some objectives should be performance-oriented (e.g., tasks performed or
projects completed) whereas others should be developmental (e.g.,
training courses taken). In addition, objectives should be set only if
they are feasible for the individual to accomplish within the period of
performance. Thus, the objective "Get an emergency operations plan
*approved* by the end of the year" should be revised to "Get an
emergency operations plan *submitted* by the end of the year" because
the LEMA can't control the approval process, which might take longer
than the end of the year to complete. Setting objectives is important
because a good development plan, based on clear objectives, shows how
the organization can achieve better ratings.

As noted in Chapter 3, local emergency managers should work with the
other members of the LEMA and LEMC to set specific, measurable
objectives they can accomplish within the period of performance. These
objectives should be developed collaboratively because such goals elicit
greater commitment than goals that a emergency manager sets
unilaterally. The goals for the LEMA and LEMC should differ from each
other for two reasons. The first reason is that the LEMA and LEMC are
different organizations with different responsibilities. The second is
that an emergency manager's ability to control resource allocation in
the LEMC is much more limited than her or his ability to control
resources in the LEMA.

Specifically, as noted in Chapter 3, the local emergency manager should
work with the LEMA staff to assess the current status of the
jurisdiction's hazard/vulnerability analysis, hazard mitigation program,
emergency preparedness program, recovery preparedness program, and
community hazard education program. Next, the LEMA staff should review
the capability shortfall identified in previous years and also the
multiyear development plan that was designed to reduce the capability
shortfall. If the goals and schedule set in that document are
inappropriate, LEMA staff should work collaboratively to set revised
goals in each of the major programmatic areas, based on an assessment of
the LEMA's current capability. The Public Entity Risk Institute's (2003)
checklist might be helpful in identifying goals. As is the case with the
multiyear plan, the local emergency manager should work with the LEMA
staff to set specific milestones (objective indicators of task
performance) to determine if they are making progress at a satisfactory
rate throughout the year. Clear assignment of authority and
responsibility for task performance will not only simplify the process
of individual performance appraisal at the end of the year, but it will
also enhance the likelihood of successful task performance.

Evaluating performance of the LEMC is somewhat more complex, but follows
basically the same procedures as are used for the LEMA. Each LEMC
subcommittee should identify the specific tasks that must be
accomplished in order to make progress in its functional area (e.g.,
hazard/vulnerability analysis; planning, training, and exercising;
recovery and mitigation; public education and outreach; LEMC
management). In some cases, this will lead subcommittee members to set
an objective that involves acquiring the resources needed to perform a
task, whereas in other cases the objective might involve actual task
performance. For example, a Hazard/Vulnerability Analysis Committee
might first set an objective of acquiring a computer program and
database, next set an objective of getting a LEMC member trained to use
these materials, and finally set an objective of having that person
conduct an HVA.

As is the case with the LEMA, the LEMC should use a collaborative
process to set specific, measurable objectives that it can achieve
within the performance period. The subcommittees should coordinate their
objectives with each other, either through the Executive Committee or in
general meetings of the LEMC. Once all of the subcommittees have set
objectives, they should monitor their performance informally throughout
the year and review it more formally at the end of the year. Once the
LEMC as a whole has reviewed its annual performance, the Executive
Committee should meet with senior elected and appointed officials to
discuss the LEMC's achievements during the previous year.

*National Fire Protection Association Standard 1600*

The National Fire Protection Association (NFPA) Standards Council
established a Disaster Management Committee in 1991 that was charged
with developing standards for preparedness, response, and recovery from
the entire range of disasters. The committee's first product, *NFPA 1600
Recommended Practice for Disaster Management,* was published in 1995.
Standards are reviewed and revised on a five-year cycle and the
committee preparing the 2000 edition concluded that the scope of the
original effort was too narrow. With representatives from FEMA, IAEM,
and the NEMA, the committee adopted what they called a "total program
approach" that is consistent with the principles of comprehensive
emergency management and integrated emergency management systems. The
new standard was written to cover both public and private sector
organizations (there are private fire services organizations such as the
nationally based Rural Metro). They also included business continuity
programs in their scope. Business continuity planning, as mentioned
above, first began in the private sector as a means of incorporating
disaster planning and consequences into business plans. Business
continuity planning follows the tradition of federal government
continuity planning used to ensure the continued delivery of government
services following nuclear attacks (Perry & Lindell, 1997c). The
standard issued in 2000 was renamed the *Standard on Disaster/Emergency
Management and Business Continuity Programs*. The third edition, issued
in 2004, retained the same name as the second edition (National Fire
Protection Association, 2004).

The current version of NFPA 1600 defines a set of criteria for disaster
and emergency management programs (including business continuity
programs). The purpose of the standard is to provide a means of
assessing and improving existing programs and guidelines for creating
new programs. Although the standard addresses issues of program
management, acknowledging that appropriate technical expertise is
required, most of the document describes specific program elements that
should be present. The standard defines an *entity* as a public or
private sector organization that is responsible for emergency/disaster
management or continuity of operations. The standard requires the entity
to have a documented emergency management program with an adequate
administrative structure, an identified coordinator, an advisory
committee, and procedures for evaluation. The program must address 14
elements identified in Table 12-3.

**Table 12-3**. Emergency Management Program Elements.

  ----------- -----------------------------------------------------------
  Element     Title

  1           Laws and Authorities

  2           Hazard Identification, Risk Assessment and Impact Analysis

  3           Hazard Mitigation

  4           Resource Management

  5           Mutual Aid

  6           Planning

  7           Direction, Control, and Coordination

  8           Communications and Warning

  9           Operations and Procedures

  10          Logistics and Facilities

  11          Training

  12          Exercises, Evaluations, and Corrective Actions

  13          Crisis Communications and Public Information

  14          Finance and Administration
  ----------- -----------------------------------------------------------

The *Laws and Authorities* element must address the legislation,
regulations, directives, and industry standards that authorize the
emergency management program. The *Hazard Identification and Risk
Assessment* element must identify the hazards to which the entity is
exposed, the probability of extreme events that would adversely affect
the entity, and the potential physical and social consequences of those
events. The *Hazard Mitigation* element requires the entity to develop a
strategy to eliminate the hazards or to limit their consequences. The
*Resource Management* element requires the entity to identify the
resources (personnel, facilities, equipment, materials and supplies, and
money) that are needed by the emergency management program, inventory
the resources that are currently available, and define the resulting
shortfall. This includes addressing the role of volunteers, as well as
creating and participating in mutual aid agreements as a means of
enhancing resources available to an entity (political jurisdiction or
organization). Also as part of resource management, programs are
directed to establish performance objectives for each threat identified
in the HVA for personnel, equipment, apparatus, and facilities. These
needs or resources are to be calculated in terms of quantity, response
times, capabilities, and other factors related to deployment.

The *Mutual Aid* element requires the entity to identify the need for
resources from other entities, establish formal agreements for
requesting those resources, and reference these agreements in relevant
sections of the entity's emergency management plans. The *Planning*
element requires the entity to develop a strategic plan, an emergency
operations/response plan, a recovery plan, a hazard mitigation plan, and
a continuity plan. In short, the standard requires a planning process to
be established that includes different functions for the planning
entity. This process should produce distinct plans for mitigation,
operations, and recovery, as well as an overall strategic plan. The
generic content of each type of plan is specified in the standard. In
addressing common plan elements, NFPA 1600 requires each entity to
specify functional roles and responsibilities within its own
organization. Moreover, roles must also be defined for external
organizations or agencies that are expected to participate in
mitigation, preparedness, response, and recovery activities.

The *Direction, Control, and Coordination* element requires the entity
to establish the authority for response and recovery operations. This
specifically includes the adoption of an incident management system and
the clear assignment of functional responsibilities to specific
organizations within the entity. The *Communications and Warning*
element requires the entity to develop and periodically test the
equipment and procedures needed to activate the emergency response
organization. The *Operations and Procedures* element requires the
entity to develop the procedures needed to respond to the hazards
identified in Element 2 (*Hazard Identification and Risk Assessment*).
These procedures must include situation assessment and resource
assessment, transition from response to recovery operations, and
continuity of operations. The *Logistics and Facilities* element
requires the entity to develop systems for identifying, obtaining, and
delivering needed resources to response and recovery personnel, as well
as handling unsolicited donations. This element also addresses the
requirement for a primary and alternate EOC.

The *Training* element requires the entity to identify training needs
(including training scope and frequency), implement a training program,
and document the delivery of training to all emergency response
personnel. The *Exercises, Evaluations, and Corrective Actions* element
requires the entity to conduct periodic evaluations of the plans and
procedures. These should include tabletop, functional, and full-scale
exercises. This element also requires the entity to develop procedures
for ensuring corrective action regarding any identified deficiencies.
The *Crisis Communications, Public Education, and Information* element
requires the entity to establish procedures for disseminating relevant
information to the news media and the public during pre-, trans-, and
postdisaster phases of operation. Finally, the *Finance and
Administration* element requires the entity to develop fiscal procedures
to ensure decisions can meet the time constraints of emergencies yet
conform to accepted accounting standards.

Ultimately, NFPA 1600 and its supporting material lay out the elements
of an emergency management program that can be adapted to any level of
government or private industry. In terms of professionalism, NFPA 1600
is significant because of the principles and assumptions upon which it
is based. That is, the program elements mandated in the standard assume
that the objectives of an emergency management program are to manage all
hazards in a multiorganizational (and, often, multijurisdictional)
environment. This marks a fundamental endorsement of the approach to
emergency management that has evolved under FEMA and that is supported
by the principal professional associations. Even more than that, the
standard defines the interest of an emergency management program as
creating a disaster resilient community. That is, there is certainly an
emphasis upon response operations, planning, and recovery, but there is
a *distinct and explicit* concern with employing effective land use
practices, codes and regulations, strategic community protection, and
sustainable urban design.

NFPA 1600 is important to the profession of emergency management for
several reasons. The first of these is that it was issued from a
respected and established authority. Virtually all major organizations
in the field of emergency management (especially in government) have
long dealt with and understood the importance of NFPA and the standards
it creates. This produces recognition among other professional groups;
elected authorities; city, county and other governmental managers; and
business organizations. Another point that underscores the importance of
NFPA 1600 is its potential for use in program assessment. The standard
provides a model that can be used in self-assessment and also by
external evaluators. Furthermore, NFPA 1600 is sufficiently detailed to
serve as a basis for planning either to create an emergency management
program or to enhance an existing program so that it meets the standard.
Government---and all organizations---must allocate limited resources.
When emergency managers must make arguments to expand, buttress, or
change their programs, NFPA 1600 serves as an authoritative basis for
claims.

*The Emergency Management Accreditation Program*

The Emergency Management Accreditation Program (EMAP) is based on NFPA
1600 (Emergency Management Accreditation Program, 2004a). However, EMAP
differs from NFPA 1600 in adding language that is specifically
appropriate for SEMAs and LEMAs (recall that the scope of NFPA 1600 also
includes private sector organizations). In addition, the EMAP
accreditation process is quite elaborate. Specifically, an EMA that
submits an application for accreditation has 18 months to conduct a
self-assessment of its compliance with EMAP's 54 standards. Consistent
with the recommendation in Chapter 3, the self assessment should begin
with a written expression of executive-level (e.g., mayor, city manager,
county supervisor) commitment to receiving EMAP accreditation. This
process should continue with the selection of an accreditation manager
who is responsible for coordinating the accreditation application. This
duty is most likely to be assigned to the local emergency manager. In
addition, the accreditation manager should coordinate the development of
a self-assessment plan that allocates responsibilities to each agency in
the emergency management organization and sets a schedule for the
self-assessment process.

The self-assessment requires the applicant to examine its emergency
management program and to provide a proof of compliance record for each
of the EMAP standards. The proof of compliance record asks the applicant
to identify the written documentation, interview sources, or
observations on which the claim of compliance is based. In addition, the
applicant is asked to explain how the documentation, interviews, or
observations support the claim of compliance. In the case of written
documentation, the proof of compliance record must be accompanied by a
copy of a relevant document or guidance in locating a copy of that
document. Equivalent documentation is required for interviews or
observations.

After completing the self-assessment documents, the applicant submits
them to the EMAP Commission for review. If the EMAP Commission judges
the applicant's materials to be satisfactory, it will schedule an
on-site assessment. During the on-site assessment, the assessor team
receives an orientation to the jurisdiction (including a tour of major
facilities) and obtains additional information about compliance. In
addition, the team conducts inspections to verify the accuracy of the
compliance claims and conducts an exit interview that describes their
preliminary findings. In particular, the members of the assessor team
verify compliance claims by examining the written plans, procedures, and
memoranda that were listed as documentary proof of compliance. Moreover,
assessor team members contact interviewees listed on the proof of
compliance record to obtain independent verification of the information
listed in the application. Finally, assessor team members inspect
facilities, equipment, materials, and supplies to verify their adequacy.

The assessor team then describes its activities and reports its findings
to the EMAP Program Review Committee at a meeting that the accreditation
applicant may attend. Based on this information, the EMAP Program Review
Committee recommends that the applicant be accredited, that the
applicant be conditionally accredited, or that the application be
denied. This recommendation is then transmitted for final action by the
EMAP Commission. If accredited, the applicant is issued a certificate
that is valid for five years. Nonetheless, accreditation must be
maintained during that time period by continuing compliance with the
EMAP Standard, continuing documentation of compliance, and filing an
annual report with the EMAP Commission. If accreditation is denied, the
applicant is informed of the reasons why this action was taken. Further
information about EMAP can be found at www.emaponline.org.

*National Incident Management System Capability Assessment Support Tool
(NIMCAST)*

Emergency managers can logon to NIMCAST (www.fema.gov/nimcast/index.jsp)
to conduct a self assessment of their jurisdiction's compliance with
NIMS. NIMCAST breaks down each chapter of NIMS into sections that
address compliance requirements. NIMCAST presents a series of questions
that can be answered either "yes" or "no". After completing the
questions, the user can identify the areas in which his or her
jurisdiction is deficient and, therefore, where it needs to improve in
order to become compliant.

# Evaluating Drills, Exercises, and Incidents

Evaluating drills, exercises, and incidents has some elements in common
with employee, LEMA, and LEMC performance appraisals, but also some
significant differences. The primary difference is that performance in
drills, exercises, and incidents is measured over a relatively short
period of time. In drills, performance often is measured over a period
of minutes. In exercises and incidents, performance is measured over a
period of hours to days. Compared to the annual reviews of individual
and organizational performance discussed in the previous sections, this
shortened time period makes evaluation easier because there is less
performance to evaluate. On the other hand, task performance usually is
measured much more intensively during drills and exercises and many
people must be observed. Finally, incidents---especially those involving
a loss of life or extensive destruction of property---have the potential
for generating lawsuits. In turn, these can stifle the free exchange of
information needed to learn from experience and improve the state of
community emergency management.

*Drills*

When planning to conduct drills, the first task is to specify clearly
what are the objectives to be tested (National Response Team, 1990).
Typically, drills are used to test people, facilities, and equipment on
tasks that are *difficult*, *critical*, and are *performed
infrequently*. The first two conditions are important because they make
failures in task performance likely and they escalate the consequences
when failure does occur. The third condition is important because
lengthy time intervals between opportunities for task performance cause
people to experience skill decay and equipment to deteriorate (e.g.,
batteries age, wires corrode, and meters lose calibration).

Drills usually involve only a single person, or at most a few people who
are located close together, who receive information about a hypothetical
situation (a *scenario*) that requires them to perform a specific task.
A task is "a distinct work activity carried out for a distinct purpose"
(Cascio, 1991, p. 190). A task might have many steps or elements, which
are defined as "the smallest unit\[s\] into which work can be divided
without analyzing separate motions, movements, and mental processes
involved" (Cascio, 1991, p. 190). In a hazmat emergency response drill,
a technician might be assigned the task of plugging a hole in a
container that is leaking a toxic liquid. An unannounced EOC activation
drill might include contacting essential personnel to the EOC, timing
their of arrival, and determining if they implement all of the
activation procedues successfully. Such tasks have multiple steps
(elements performed in sequence) that include obtaining a briefing about
site conditions, donning personal protective equipment, verifying radio
operability, collecting repair tools and materials, entering the hot
zone, performing the repair, returning to the warm zone for
decontamination, and participating in a debriefing.

The drill is conducted by an *evaluator*, who must have a level of
proficiency that meets or exceeds that of the person being evaluated. In
addition, the evaluator must identify any facilities, equipment (e.g.,
calculators, computers, or communication devices), and job performance
aids (e.g., written procedures, tables, or figures) that are specified
for use in performing the task to be tested. Frequently, an evaluator
randomly selects one or more individuals from a list of personnel listed
as the principal or alternate performer in a position to be tested. The
player is asked to "walk through" task performance by performing each
step in that task. In cases where there is a significant mental as well
as physical component, players might be asked to "think aloud" as they
proceed through the scenario by identifying the information they need,
the way they are processing the information, and the final judgment or
decision they have made (Ericsson & Simon, 1993). As noted earlier,
drills usually involve only a single person, or at most a few people who
are located close together. Consequently, a single individual can serve
as the *controller*, the person who provides the information from the
scenario, and as the *evaluator*, the person who observes the player's
performance and notes any deviations from the EOP or its procedures.

*Functional Exercises*

A functional exercise differs from a drill in the larger number of
personnel, usually from a single department, that are responsible for
performing a single function within the EOP (e.g., protective action
selection and implementation, hazmat spill control). Exercises are more
comprehensive than drills because the scenario is usually more
complex---involving more tasks and, thus, more equipment and personnel.
In turn, this means functional exercises test people's ability to
perform both *taskwork* and *teamwork*. Where the former is obviously
the ability to competently perform each separate aspect of the emergency
response, the latter is the ability to allocate resources and schedule
tasks to achieve a coordinated performance that is efficient, effective,
and timely (McIntyre & Salas, 1995).

The complexity of a functional exercise usually requires different
people to serve as controller and evaluator. Indeed, some functional
exercises might require multiple controllers (who maintain contact by
radio or cell phone) that provide information to teams of players
operating in different locations. Moreover, these complex functional
exercises will probably also require multiple evaluators, although some
remotely located teams having modest performance demands might be
assigned a single individual who serves a both controller and evaluator.

*Tabletop Exercises*

A table-top exercise differs from a drill or functional exercise because
it involves a group of senior personnel, usually branch or departmental
administrators, who serve as the directors (either the principals or
alternates) of their functions. The scenarios for table-top exercises
vary in their complexity; some are as simple as open-ended questions
designed to generate a free-ranging discussion about a particular
problem the local emergency manager (or the LEMC's planning, training,
and exercising committee) has noticed. For example, a table-top exercise
might address the criteria (e.g., minimum strike probability and storm
category) for initiating an evacuation of local hospitals in advance of
a hurricane, the resources available for providing transportation
support to the hospital evacuation, and the methods of facilitating the
return of ambulances and other vehicles against the flow of evacuation
traffic. By its very nature, a table-top exercise involves a group of
people---often in a single room. This makes it amenable to staffing by a
single controller who also serves as the evaluator.

*Full-Scale Exercises*

A full-scale exercise simulates a community-wide disaster by
simultaneously testing multiple functions and, especially, the
coordination among these functions. The complexity of full-scale
exercises requires thorough planning of the scenario, as well as
coordination among the many controllers and evaluators. There also is a
need for training the controllers, who might need to make *ad hoc*
adjustments if the players take unexpected actions during the exercise
that deviate significantly from the conditions listed in the scenario.
In addition, there is likely to be a need for training on the local EOP
and its procedures if the demand for controllers and evaluators exhausts
the available supply of qualified personnel.

The magnitude of full-scale exercises varies considerably. Small ones
might provide only a limited test of a few functions (e.g., a single
school might be selected to test the evacuation plan). However, large
exercises conducted for nuclear power plants can involve thousands of
players and as many as 50-100 controllers and evaluators. Unlike drills,
table-top exercises, and functional exercises---whose schedules are
usually announced in advance---many full-scale exercises are
unannounced. Thus, the exercise scenario and the time at which it will
be initiated are unknown to the participants. The rationale for
unannounced exercises is that this prevents agencies from scheduling
their best trained (occasionally their *only* trained) personnel for
participation in the exercise, thus providing a more accurate assessment
of community preparedness. This is especially true when exercises begin
on the evening and night "back shifts". This is certainly a valid reason
for scheduling unannounced exercises, but it is nonetheless a good idea
to work up to unannounced exercises by first verifying satisfactory
performance in announced exercises. Performing well in an unannounced
exercise is a challenging goal and, as discussed earlier, achievable
goals should be set first to build confidence and motivation to improve.
Poor performance can prove to be a demoralizing experience for
participants and also a public embarrassment to the participating
agencies. In addition, a very large number of errors might make it
difficult to identify clear cut "lessons learned" and to develop a
consensus on how to improve.

The Federal Emergency Management Agency (2003b) organizes the
development of an emergency exercise into eight steps:

1.  Needs assessment,

2.  Scope definition,

3.  Purpose statement,

4.  Objectives,

5.  Narrative,

6.  Major and detailed events,

7.  Expected actions, and

8.  Messages.

The *needs assessment* addresses the community's primary and secondary
hazards, focusing on testing the solutions that were implemented to
correct past problems in the EOP, procedures, staffing, and training, as
well as new facilities, personnel, and equipment.

The exercise's *scope definition* includes the type of emergency, its
location, the functions to be exercised, the members of the emergency
response organization who will participate, and the type of exercise. By
contrast, the *purpose statement* explains why the exercise is being
conducted. The exercise *objectives* are specific task statements
describing the actions participants are expected to demonstrate. That
is, the objectives should make clear *who* should take *what action* in
response to *which conditions* and to *what standard of performance*.

The *narrative* provides a specific description of relevant conditions,
including a chronology of events. These include the initiating event
(e.g., specific details about a train derailment causing a fire and
release of hazardous materials), and contextual conditions in the
physical (e.g., weather conditions) and social (e.g., proximity to human
occupancies, level of emergency response organization activation)
environments. There should be a description of all *major events*
requiring actions to meet the exercise objectives and of *detailed
events* if they are expected to initiate expected actions.

*Expected actions* are ones that achieve the exercise's stated
objectives. These include *assessment* actions that obtain or verify
information about the existence of environmental conditions (e.g.,
obtaining information about wind speed, wind direction, and atmospheric
stability) or that verify the performance of organizational actions
(e.g., verifying the establishment of traffic control points to restrict
access to evacuated areas). Expected actions also include *preventive*
(e.g., adding sandbags to increase the height of a levee) or
*corrective* (e.g., patching a leaking chemical tank) actions that
reduce the magnitude of the event, *protective* actions that reduce the
effects on people (e.g., sheltering in-place), and *management* actions
that consider alternative actions, make decisions, allocate resources,
or coordinate the actions of responders and the public (e.g., selecting
and implementing an evacuation within one mile of a train derailment).

The key to effective exercises is the transmission of messages
communicating specific events to exercise participants. Messages should
indicate who (source) says what (content), how (channel) and to whom
(recipient). They also should be designed so they generate the expected
actions that meet exercise objectives (with what effect).

*Incidents*. The evaluation of performance in an incident can be
extremely informative because it provides an unscheduled test of the
emergency response organization. This has the advantage of providing a
realistic test of many incident management functions, such as
organizational activation and notification, in a way that would not be
achieved in an announced exercise. Of course, the disadvantage of actual
incidents as evaluations of the emergency response organization is that
they also are uncontrolled. That is, both the magnitude of the event and
the response functions that are tested are matters of chance. Of course,
incidents also have no controllers or evaluators, so respondents must
frequently rely on their memories and any documentation they have
produced to establish who did what, where they did it, when they did it,
and why they did it. This underscores the need for emergency managers to
ensure there is adequate documentation of incident conditions and the
community response. As the next section indicates, such documentation
will be needed to support a critique.

*Critiques*

All three forms of exercises (tabletop, functional, and full-scale) and
incident responses will benefit from an oral critique by the players,
controllers, and evaluators (National Response Team, 1990). However,
some full-scale exercises have so many participants that a smaller
number of representatives must be selected from each responding unit. In
an exercise critique, the discussions should address whether the
exercise objectives were met. In an incident critique, the question is
whether the response was consistent with the EOP and procedures. If
there were deviations from the EOP and procedures, the participants
should discuss why this occurred. In some cases, the deviation is
adaptive (i.e., responders improvised a more effective method of
protecting public health, safety, property, and the environment) and the
conclusion will be that the EOP or procedures need to be revised. In
other cases, the deviation will be judged to be maladaptive, so the
solution will be to reassign personnel, improve personnel training, or
upgrade facilities and equipment. Whatever problems are identified, the
results of the critique should be documented in a written report that
contains specific recommendations for action, assignment of
responsibility for implementation, and a schedule for completion of each
element of the action plan.

# Evaluating Training and Risk Communication Programs

The procedures for evaluating training and risk communication programs
are a logical extension of the previous types of evaluations in their
requirement for emergency managers to define, in advance, what are the
criteria for defining the success. Many emergency managers already
evaluate the success of training and risk communication programs in
terms of *input criteria*. For example, CERT training can be measured by
the number of training participants, the number of contact hours of
training, and the percentage of a community's neighborhoods covered.
Risk communication programs can be measured by the number of brochures
distributed, the number of presentations at community meetings, and the
number of newsmedia stories. Ultimately, however, programs need to be
evaluated in terms of *output criteria*---the extent to which the
programs changed people's attitudes and behavior. As a result, the best
procedures for evaluating training and risk communication programs are
distinctly different from the previous types of evaluations because they
involve assessing the effects of a treatment on a sample of individuals
from a larger group (Cook, Campbell & Peracchio, 1990; Schmitt &
Klimoski, 1991).

*Criteria for Defining Program Success*

The criteria for judging the success of training programs are often
classified into four groups: reaction, learning, behavior, and results
(Goldstein, 1993). *Reaction criteria* consist of trainees' judgments
about the training program. These usually include trainees' evaluations
of the trainers, the facilities, and equipment; their perceptions of
their mastery of the material; their enjoyment of the class; and their
willingness to take another class from the instructor. *Learning
criteria* are defined by trainees' performance on written tests of
knowledge or performance tests of skills within the context of the
training program. *Behavior criteria* refer to trainees' ability to
apply knowledge and skills they learned during their training when they
return to their jobs. In the context of emergency management, this means
performance during drills, exercises, and incidents that take place
after training is completed. Finally, *results criteria* refer to the
consequences of trainees' performance on the job. That is, did the
training make a difference in the overall performance of the
organization? Someone who can flawlessly demonstrate a skill that is
never used on the job (e.g., knowing how to conduct radiological
monitoring in a community that has no exposure to radiation hazard) will
probably never have an impact on the community's safety.

The same four criteria can be used in the assessment of a community risk
communication program. Reaction criteria are measured by participants'
reactions to the speaker, setting, communication medium, and message
content. Learning criteria are measured by the participants' beliefs
about the hazard and hazard adjustments. Behavior criteria are measured
by households' and businesses' implementation of hazard adjustments, and
results criteria are measured by reductions in casualties, damage, and
disruption from disasters. Clearly, reaction criteria are the easiest to
collect, whereas learning and behavior criteria are more difficult to
obtain. The infrequency of disasters makes the collection of results
criteria extremely difficult. Moreover, it can be difficult to draw the
correct conclusions from the impact of a single disaster. For example, a
chemical spill might produce few casualties and damage because of
specific event conditions (a small quantity was released) or situational
conditions (the wind was blowing away from populated areas) rather than
because the emergency organization responded effectively.

**Case Study: Evaluation of the Seaside Tsunami Awareness Program**

The city of Seaside is located near the northwest tip of the Oregon
Coast about 80 miles from Portland. The city has a permanent population
of 6000 that swells to 40,000 in the summer. Most of the city is located
in the tsunami runup zone, so community leaders initiated a nine month
tsunami awareness program in December, 2004 (Connor, 2005). The program
began by creating a tsunami website
(cityofseaside.us/html/tsunami_info.html) and placing tsunami awareness
brochures at city hall, local grocery stores, the post office, the
library, and other public locations. Program staff also posted tsunami
evacuation maps in community buildings (e.g., city hall, community
development department, library, swimming pool, community center,
visitors' bureau, police station, and fire station) and arranged for a
series of feature articles and a full-page notice in the local
newspaper. These publicity efforts were supplemented by a program
comprising five outreach events:

-   Neighborhood Educator Project

-   Business Workshop

-   School Outreach Program

-   Public Workshop

-   Tsunami Evacuation Drill

The *Neighborhood Educator Project* divided the city into 88 blocks,
with approximately 40 households per block. Project staff mapped the
blocks and recruited a volunteer Neighborhood Educator for each block.
The Neighborhood Educators participated in a one hour class based on
information from the International Tsunami Information Center, the
National Oceanographic and Atmospheric Administration, the Oregon
Department of Geology and Mineral Industries (DOGAMI), the Red Cross,
and the city of Seaside. In addition, the Neighborhood Educators were
given Seaside's tsunami evacuation brochure, a calendar of outreach
events, information in Spanish, and a resource guide for families. They
were asked to develop their own approaches to neighborhood education and
to disseminate the tsunami awareness information to the residents of
their blocks.

The *Business Workshop* invited all local businesses to attend a
workshop containing basic tsunami information, as well as instruction on
business emergency preparedness and disaster recovery. The workshop
began with a lecture that was followed by a small group exercise in
which groups of ten participants identified appropriate preparedness
measures for their businesses and mapped their tsunami evacuation
routes. A facilitator at each table recorded the group's major ideas
and, shortly afterwards, presented them to the entire workshop. The
entire workshop was videotaped and covered by the local news media.
Participants were given educational materials (brochures, a business
guide to disaster preparedness, and stickers) to take as they left.

The *School Outreach Program* was conducted in the middle school and two
elementary schools. A DOGAMI staff member presented a one hour lecture
to the elementary school children that explained the differences between
locally- and distantly-generated tsunamis. The lecture also explained
how to develop family emergency plans and identify appropriate
evacuation routes. After the lecture, the city's tsunami program
coordinator conducted a quiz in which stickers, pens, and rulers were
awarded for correct answers. The middle school program was implemented
as 22 discussion groups, each of which involved about 30 students and
lasted a half hour. Group facilitators (the tsunami program coordinator
and three high school volunteers) moderated discussion about locally-
and distantly-generated tsunamis and how to respond to each type of
tsunami. Each discussion group located their school on the tsunami
evacuation map and identified their families' evacuation routes. All
students were invited to submit entries to a tsunami awareness program
poster competition.

The *Public Workshop*, held at the city's convention center, began with
a half hour presentation by Oregon Emergency Management (OEM) that was
followed by a half hour question and answer period with an expert panel.
Next, participants were divided into small groups for discussion
sessions that were similar to those in the middle school program. Group
facilitators helped participants map their evacuation routes and then
each group decided how to respond to different types of tsunami
scenarios. Each group then developed a list of what households can do to
prepare and reported back to the other groups on their recommendations.
As with the Business Workshop, the proceedings were videotaped and
covered by the local news media. Participants were provided with hazard
awareness materials (brochures, a family resource packet, and Red Cross
information) as they left.

The *Tsunami Evacuation Drill* allowed people to apply what they had
learned by implementing their family evacuation plans. The drill was
organized in cooperation with DOGAMI, OEM, Oregon DOT, Red Cross, police
and fire departments, the local transit organization, and amateur radio
operators. When the tsunami sirens sounded at noon on a Saturday after
all five outreach events had been completed, people followed the six
major tsunami evacuation routes to predetermined safety zones. In
addition, local hotels and restaurants partially implemented their
evacuation plans (one staff member per facility). The city government
activated its EOC and conducted a table top exercise for a distantly
generated tsunami.

The tsunami awareness program was evaluated in four ways---financial
cost, activity measures, outreach event evaluations (which were mostly
reaction criteria), and surveys (which were mostly learning criteria).
The financial cost of the program was small; other than the program
coordinator, the budget was only \$50 per month for supplies. The reason
for the low cost was the 51 volunteers who contributed 1300 hours of
their time and other citizens who donated materials and services. The
activity measures included the

-   Number of community and organizational meetings attended by the
    program coordinator;

-   Number of presentations at local meetings;

-   Number of Neighborhood Education training classes held;

-   Percentage of community blocks (and city households) contacted by
    Neighborhood Educators;

-   Number of businesses participating in the Business Workshop;

-   Number of people attending the Public Workshop;

-   Number of submissions to the poster contest;

-   Number of schools visited, students reached, and small-group
    sessions held; and

-   Number of people participating in the Tsunami Evacuation Drill.

The outreach event evaluations were conducted by asking participants to
fill out forms that assessed reaction criteria for each of the five
outreach events---the Neighborhood Educator Project, Business Workshop,
School Outreach Program, Public Workshop, and Tsunami Evacuation Drill.
These questionnaires identified specific strengths and weaknesses of the
individual events. In addition, volunteers conducted mail surveys of
local residents before and after the program. Based on this survey,
program staff estimated 2,200 people participated in outreach
activities. In addition, the survey clearly documented the program's
success in achieving *learning criteria* such as the:

-   Location of tsunami risk areas,

-   Cues for evacuating (leave if you feel ground shaking from an
    earthquake; don't wait for a siren),

-   Appropriate mode of evacuation (walk to high ground because bridges
    over the two rivers might collapse in an earthquake), and

-   Locations of safe zones.

The survey also confirmed the program's effectiveness in increasing the
percentage of households with emergency plans and identified the
Neighborhood Educator (44%) and Tsunami Evacuation Drill (30%) as the
activities with the highest levels of participation. By contrast, the
Public Workshop (7%), School Outreach (11%), news media coverage (5%),
Business Workshop (3%), and Web site (2%) elicited lower levels of
participation. The survey might have underestimated the impact of the
School Outreach Program because it targeted children who, of course,
would not have filled out the questionnaires. Similarly, survey might
have also underestimated the impact of the Business Workshop because
this program targeted a very limited population segment. The survey also
revealed that some respondents thought the program was not beneficial
(l4%) and a few respondents were unaware of the program (7%). Finally,
the survey identified areas in which the program needed to be improved.