A Longitudinal Study of the Impact of an Emergency Preparedness Curriculum

The body of knowledge required for professional practice evolves as new challenges are presented and as specific competency requirements of a profession flex and change. The increasing incidence and potential for terrorist events, ongoing natural disasters impacting larger and more diverse communities, and the advent of a novel pandemic influenza represent only a few of the challenges confronting public health professionals. Accordingly, new competencies are identified and existing competency sets are updated to better represent the knowledge, skills, and attributes (KSAs) needed to keep communities healthy and safe. This article describes a study conducted during a five-year time period to evaluate a curriculum designed to develop the competency of public health professionals responsible for responding to these challenges. In 2002, in response to the aforementioned issues, the University of Minnesota School of Public Health (UMNSPH) initiated conversations with key partners to identify how best to respond to emerging learning needs. Working through the University of Minnesota Center for Public Health Preparedness (UMNCPHP), this group conducted a comprehensive, learning needs assessment and developed a standardized curriculum focusing on preparedness, response, and recovery competencies.1 UMNSPH decided to create and implement two academic certificate programs organized around the Bioterrorism and Emergency Readiness (BT/ER) Competencies2 to meet the needs of the large numbers of public health workers who were without formal academic preparation. These competencies represented basic knowledge and skills needed by public health practitioners to respond to a wide range of emergencies (Figure 1). The first certificate, the Public Health Certificate in Preparedness, Response, and Recovery,3 was designed to support acquisition of knowledge and skills necessary to effectively participate in planning, executing, and evaluating community public health preparedness activities. The second certificate, the Public Health Certificate in Food Safety4 (later retitled Food Protection), was designed to support those engaged in protecting the health and safety of the food supply from “farm to table.” Each of the curriculum plans shared course offerings targeted to the same BT/ER Competencies and also offered a larger selection of elective courses based on KSAs specific to the focus of the certificate. Generally, more than one course contributed to the development of a given competency. Figure 1. Bioterrorism/Emergency Readiness Competencies for public health professionalsa The curriculum was grounded in an educational model proposed by Benner5 that framed learning as a developmental process. Benner believed that as the depth and breadth of KSAs develop and combine with experiential practice, each novice learner becomes increasingly proficient in the specific competencies under development. UMNSPH created a model for Lifelong Learning in Public Health Education1 based on this premise to guide the creation of learning activities targeted to various levels of competency. The Lifelong Learning Model was used to shape educational activities in the certificate programs. To shape evaluation of the curriculum, we used the Kirkpatrick Model.6 As with the Lifelong Learning Model, Kirkpatrick's Model is hierarchical in nature, moving from basic evaluation of learning to evaluating application of learning and subsequent outcomes. Level 1 evaluation, “Reaction” (referred to as “the happiness index”), provided activity designers and instructors with baseline information regarding the degree to which learners were satisfied with the activity, but did not provide information on learning, application of learning, or outcomes. This level of evaluation was routinely employed for UMNSPH courses using a standardized evaluation tool. Level 2 evaluation strategies were designed to determine the degree to which KSAs were learned. A variety of evaluation strategies were used to measure student learning, including course presentations, group work, written papers, and pre- and posttest measures. Level 3 evaluation, characterized by a change in behavior, required contact with those who had completed courses to determine what, if any, behavior changes had occurred as a result of the newly acquired competencies. Kirkpatrick's Level 4 evaluation addressed outcomes resulting from changes in behavior. An evaluation yielding data regarding achievement of desired outcomes represents an example of Level 4 evaluation. Competency reflects the ability of the learner to appropriately apply what has been learned in professional practice.7 It was, therefore, important to incorporate Level 3 and, to the degree possible, Level 4 measures into the evaluation plan. With this in mind, we decided that one of the strategies for evaluating the curriculum would be to include a longitudinal study to gather data about changes in behavior, based on what had been learned in courses comprising these two certificate programs, and to determine if and how new learning was applied as part of everyday practice. Because changes in behavior signaling integration of new KSAs occur over time, we decided feedback would be solicited at six and 12 months following completion of coursework.