# Presentation: 210398

## “PHI”Public Health Investigation   e-ExerciseDeb Nickels RN,C, BSNAmanda Hodle, MPHElizabeth Ablah, PhD, MPH

- Kansas’ e-Exercise approach to increasing public health competency in disease investigation & emergency readiness

## History

- Partnership between Kansas Association of Local Health Departments & Kansas Department of Health & Environment to assess County Health Departments (locals) competency in disease investigation
- _Planning partners_
- Kansas Association of Local Health Departments (KALHD)
- Local Health Departments in Kansas

**Notes:**
Wanted to evaluate/assess the training provided on the protocols actually taught locals how to use the protocols correctly/efficiently how to investigate/follow-up infectious disease cases

## History, cont

- Kansas Department of Health and Environment (KDHE)
- Office of Local and Rural Health
- Bureau of Epidemiology and Disease Prevention (BEDP)
- Office of Communications
- Bureau of Consumer Health (BCH)
- KDHE Laboratory
- St. Louis University Heartland Center for Public Health Workforce Development/Bioterrorism Preparedness (SLU)
- University of Kansas School of Medicine—Wichita (KUMC-W)

**Notes:**
St Louis University Heartland Center assisted with exercise design and the University of Kansas School of Medicine is the contracted entity for evaluation.

## Goals

- _Create pilot e-Exercise that would test_:
- KALHD disease investigation guidelines
- Risk Communication skills
- Surge capacity skills
- Coordination skills
- _Design e-Exercise that would utilize core competencies as basis for questions_

**Notes:**
KALHD commissioned the guidelines; KDHE reviewed them.
Risk Comm training was provided in 2004.  Multiple opportunities for live training; multiple resources (CDCynergy, tools from BT toolkit)

## Goals, cont

- Develop infectious disease e-Exercise that assists local public health workers in building disease investigation skills
- Develop opportunities for new and/or enhanced partnerships related to infectious disease investigation and public health preparedness
- Increase local public health workers’ level of preparedness for any public health event

**Notes:**
Coordination locally, as well as with regional and state partners (other Co, KDHE, BCH)
Coordination with potential partners for this and future exercises (EM, Hospital, Law Enforcement)

## Pilot Region

**Notes:**
They volunteered to be our “guinea pigs”; 15 BT regions

## Justification

- Formatted to address concerns of locals about being consistently pulled away from home office
- Create opportunity to have all relevant disease investigation staff involved
- Evaluate retention of training provided in ‘03-’04
- Identify training needs

## Timeline

- Planning initiated Aug 2004
- e-Exercise development began Oct 2004
- e-Exercise piloted in Feb 2005
- _Proposed future actions_:
- Take e-Exercise to regions around state
- Develop other e-Exercise scenarios

## Exercise Timetable

| Task | Date | Time | By | Special Details |
| --- | --- | --- | --- | --- |
| Rules of the Game | 1-28-05 | Before 10AM | PHIX |  |
| Week 1, Scene 1 | 1-31-05 | Before 10 AM | PHIX | Results back 2-2 |
| Week 1, Scene 2 | 1-31-05 | 2-4 PM | PHIX | Results back 2-2 |
|  |  |  |  |  |
| Live Simulator tc | 2-3-05 | Before 10 AM | Daniel N. |  |
| Week 1, Scene 3 | 2-3-05 | Noon-2 PM | PHIX | Results back 2-8 |
| Live Simulator tc | 2-3-05 | 1-2 PM | Linda F. |  |
| Week 1, Scene 4 | 2-3-05 | 2-4 PM | PHIX | Results back 2-8 |

## Exercise Design

- _Test resources on_:
- Accessibility
- Usability
- Accuracy
- _Utilize new & existing communication systems_
- PHIX (Kansas Health Alert Network System)
- HAWK (electronic disease surveillance and reporting system)
- Fax
- E-mail
- Phone

## Exercise Design, cont

- e-Exercise done in real-time
- e-Exercise divided into injects and questions
- Divided by week
- Created clues by underlining or using quotes for key prompts
*****Please refer to “Rules of the Game” handout for further information

**Notes:**
We wanted to exercise to be played in a time frame similar, but not identical to, a real outbreak.  In the past in Kansas, exercises were done in one day as a tabletop exercise.  While this is a good format, we wanted to address issues of team work on the local level.  We wanted to find out how locals worked at home, with their staff and co-workers, as well as with regional partners.
Also, needed to break it up so that we could get some information back before providing further information (some information in following scenes gave “answers” to questions asked in previous scenes).

## Exercise Design, cont

- _Injects distributed via PHIX_
- Responses to questions submitted through PHIX
- Only e-Exercise participants & selected observers could access exercise materials on PHIX
- _Questions derived from injects_
- Completed from local office
- Completed as a disease investigation team
- _Forms used were existing disease investigation tools in KALHD guidelines_

**Notes:**
[add list of observers here]

## Scenario

- Infectious disease outbreak
- Transmission is fecal-oral
- Each county had at least one case
- Included a food worker
- Food worker worked while infectious
- Regional control activities indicated

**Notes:**
We’re not getting specific b/c we want to use this exercise, with some minor details changed, for the rest of the state in the future.  
Food worker had multiple occasions, due to where he lived and where he worked, to infect the public.

## Scene Example

- Week 1: Co A finds out they have an infectious disease, confirmed by lab report
- Week 2: Co A finds out that the contact of this case is a food worker
- Week 3: Co A finds out that this food worker continued to work during infectious period
- Surprise!

**Notes:**
The stress from the locals began (according to us), when there was some indication that there was more than one case and that this might be a regional issue, as well as the realization that risk communications would be needed.  
They had to be encouraged to work as a region initially, though worked wonderfully together after they “got off the ground”.
Surprise = different, unexpected forms of communication, such as the phone call from the infection control RN simulator, community member with concerns about recent media coverage of the disease, faxes, etc that they didn’t know were coming.

## Calendar

| Sun | Mon | Tue | Wed | Thu | Fri | Sat |
| --- | --- | --- | --- | --- | --- | --- |
|  |  | 1
Contacts infectious | 2
WK 1, Scene 1 & 2 responses due | 3
Scene 3, 1:00 & 2:00 PM
Scene 4, 3:10 & 3:45 PM | 4 | 5 |
| 6 | 7
Food worker@ Salina | 8
WK 1, Scene 3 & 4 responses due | 9
Scene 1, 8:10 AM
Scene 2, 12:00 AM
Scene 3, 2:25 & 3:00 PM
Scene 4, 3:30 PM
Food worker @ Salina | 10
Food worker symptomatic 
Food worker@ Salina | 11 | 12
Food worker@ Great Bend |
| 13
Other contacts symptomatic
Food worker@ Great Bend | 14
Scene 1 & 2, 10:05 AM 
WK 2, Scene 1, 2, 3, 4 responses due | 15 | 16 | 17

Scene 3, 3:22 PM
WK 3, Scene 1 responses due | 18 | 19 |
| 20 | 21
Last day for Ig to food worker 2/7 contacts | 22
WK 3, Scene 3 responses due | 23
Last day for Ig to food worker 2/9 contacts

Scene 1, 9:01 AM | 24
Last day for Ig to food worker 2/10 contacts | 25 | 26
Last day for Ig to food worker 2/12 contacts |
| 27
Last day for Ig to food worker 2/13 contacts | 28 |  |  |  |  |  |

## Participant Guidance 

- _e-Exercise prompting/guidance provided_
- as requested by counties
- as deemed appropriate per facilitator’s review of county answers
- to participants by state, regional, and local facilitators
- per uniform facilitator designed tool to determine level of guidance

**Notes:**
Gave guidance given during course of the exercise, usually by phone but also by e-mail
Includes: DN, AH, Mike Cameron/Mike H, BCH State and regional supervisors, regional BT coordinator, local food inspectors 
Because help given by so many, wanted a common tool to determine what type/how much help was given to participants
Designed scale to have some objectivity no matter who provided the coaching/prompts

## Exercise Guidance Scale

_**6-point scale**_
- 0: No assistance needed
- 1: Review material and action, no support needed
- 2: Required minimal support of action
- 3: Needed support w/ prompts or hints w/ both process & details
- 4: Full support w/ review of action & assurance action is corrected, key prompts on both process & details
- 5: Maximum support, gave answer; provided instructions on both process & details

**Notes:**
Example 0: When a counties calls in to say they are taking positive action with no feedback from facilitators
Ex 1: News releases reviewed w/ no supportive action/corrections needed
Ex 2: No help with process but needs assistance w/ detail questions “Am I understanding this question correctly?”
Ex 3: Reviewing a message map and giving suggestions for change
Ex 4: Helping with a line list...I think the tool is a line list, but what should be on it? Yes, you do need a line list, a, b, and c should be on it.
Ex 5: If event occurred today, what is your role in taking action for a similar event?  It would be x, y, z.

## Evaluation

**Notes:**
We hired an outside evaluator from the University of Kansas School of Medicine – Wichita, Department of Preventive Medicine and Public Health, to conduct an extensive evaluation.

## Evaluation Components

**Pre- & Post-exercise survey** via Internet
- 76 **Questions from Scenes** via PHIX
**County** **focus groups** (n=6)
**Regional** **focus group**
**Interviews** with KDHE staff & regional coordinator

**Notes:**
We hired an outside evaluator from the University of Kansas School of Medicine – Wichita to conduct an extensive evaluation.  There were several components that made up the evaluation for this exercise.

First, a pre and post- exercise survey was implemented via an Internet survey tool (Zoomerang) and faxes.

Second, questions (relating the the scenarios) were injected through Kansas’ Health Alert Network (PHIX).  Participants’ responses were evaluated for relevance and accuracy.

Third, the evaluators conducted six county focus groups post-exercise to assess participants’ perceptions of the exercise in the privacy of their own health departments.

Then, the evaluators conducted a regional focus group to emphasize themes identified from the earlier focus groups.

Finally, the evaluators informally conducted interviews with KDHE staff facilitating the event, and the area’s regional coordinator for their assessments of the exercise.

## Pre-Exercise Survey

## Pre-Exercise Survey

- Included all health department staff with e-mail addresses
- 65 surveys sent through Zoomerang (and fax) 56 responded
- 86% response rate
- Most questions asked respondents to rate their abilities from very poor to excellent
- Scale: very poor, poor, average, good, excellent

**Notes:**
Included all staff because we didn’t know who would be involved in the disease investigation; we didn’t want to suggest who should be involved in the exercise by to whom we sent the survey
Also wanted to be sure to capture opinions/ideas of all staff, not just administrator
Finally, respondents who were not involved in disease investigation serve as a control group

## Pre-Exercise SurveyResults

- 	_Exercise participants reported __**higher**__ abilities than non-participants in:_
-   Recognizing infectious disease cases
-   Alerting the public health system
-   Understanding Incident Command
- 	  System
-   Implementing risk communication skills
-   Use of computer software

**Notes:**
Participants reported attending a greater number of public health trainings than did non-participants
[comment in red] Liz, what software?  PHIX?  Microsoft Office? (“computer software” was asked, not specific)

## Pre-Exercise Survey Results

- _Participants Self-Rated “__**Average**__” Abilities_
- Recognize infectious disease cases (56%)
- Alert to potential for an outbreak (60%)
- Participated in a coordinated response (52%)
- Alerted the public health system (40%)
- Understood Incident Command structure (40%)
- Need for & implementation of surge capacity (56%)
- (Scale: very poor, poor, average, good, excellent)

**Notes:**
These were the survey items for which most exercise participants rated themselves as average
Non-participants’ scores were also average, but also leaned more toward “poor” and “very poor”
Coordinated response = multidisciplinary response to infectious disease cases
Alert PH System at community, state, national levels in event of public health emergency
Most exercise participants rated themselves as average
Non-participants’ scores leaned more toward “poor” and “very poor”
Coordinated response = multidisciplinary response to infectious disease cases
Alert PH System at community, state, national levels in event of public health emergency

## Pre-Exercise Survey Results

_**Participant Goals**_
- “[Develop] Familiarity and proficiency using available resources to address epidemiology-related issues.”
- “The capacity of our health department to handle large scale emergency events.”

**Notes:**
Those who ended up participating indicating they wanted ... and they wanted to understand ...

## Pre-Exercise Survey Results

_**Non-Participants Goals**_
- “I need to learn to recognize a communicable disease and then be able to know how to handle or at least help in the process of investigation.”
- “Whatever it takes to help control infectious diseases from spreading.”

**Notes:**
Some who ended up not participating also wanted to get a lot out of the training...

## Post-Exercise Survey

## Post-Exercise Survey Results

- _Participants Self-Rated Abilities as “__**Good**__”_
- Recognized infectious disease cases (59%)
- Alerted to potential for an outbreak (59%)
- Participated in a coordinated response (77%)
- Alerted the public health system (50%)
- Understood Incident Command Structure (50%)
- Need for & implementation of surge capacity (68%)
- (Scale: very poor, poor, average, good, excellent)

**Notes:**
Non-participants’ scores were also average, but also leaned more toward “poor” and “very poor”
Participants’ scores moved from “average” at pre-exercise to “good” at post-exercise

## Post-Exercise Survey Results

- _Participant Self-Rated Abilities as “__**Good**__”_
- Coordinated with state partners (82%)
- Coordinated with regional partners (91%)
- Coordinated internally (64%)

**Notes:**
Non-participants’ scores were also average, but also leaned more toward “poor” and “very poor”

## Level of Prompting/Coaching

**Notes:**
This is the level of prompting/coaching participants reported they needed during the exercise.
The participant rated levels of coaching are higher than (on the next slide) the levels of prompting/coaching facilitators actually provided.

## Slide 30

**Notes:**
This is shows you the level of prompting/coaching from the facilitators’ perspective

## Overall Rating of Exercise

## Pre vs. Post

## Pre vs. Post

- _Among those who participated, increased_:
- Ability to participate in a coordinated response
- Ability to identify the need for and implement surge capacity
- _Among those who did not participate_:
- No significant differences

**Notes:**
There was a statistically significant increase in participants’ abilities to provide a coordinated response and implement surge capacity.

## County Focus Groups

## County Focus GroupsResults

_**Communication**_ - challenges (PHIX, e-mails), positives (phone conference)
_**Design**_ - Preferred this design over tabletop or other off-site exercises
_**Activities**_ - Made changes you liked; meeting more often

**Notes:**
Some themes emerged from the county focus groups.

There were benefits and challenges with various communication methods (phone conference vs. PHIX, Zoomerang, e-mails)

The design was preferred and considered innovative

Participants reported changing their behaviors – they liked working together, and therefore decided to meet and discuss cases and scenarios more often

## County Focus GroupsResults

_**Impact**_ - Powerful; sometimes seemed real; of value to you personally and to agencies
_**Length**_ - Seemed long at Week 4; but it would have lasted longer if real
_**Lesson**_ – Can work together, relied on partners

**Notes:**
Nearly all participants reported the exercise had a powerful impact on them (how seemed real)

(self-explanatory)

## Participant Impressions

_**Exercise was**_:
- Reasonable
- Manageable
- Provided good internal and partner challenges
- Recommend e-exercise in this format to be conducted at least every 15 months
_**Participants had fun**_

**Notes:**
According to the counties, this exercise was characterized as...  These were comments made by several different people from different counties
Were told by one participant that they would like more clues so they could finish this

## Facilitator Impressions

- Counties were exceptionally cooperative
- Counties were prepared
- Received positive feedback from:
- Within KDHE
- External Stakeholders
- Counties gained confidence in disease investigation and coordination skills

**Notes:**
All the counties sent responses in on time or coordinated with facilitators when they couldn’t get it in by the deadline.
Counties took the time to review resource material before the exercise.  This was strongly suggested but not required
Though some guidance, overall ready
Pleased that no major changes...after roll out/went live.
Positive feedback from Office of Comm, OLRH, BCH, ESS/BEDP
Positive: KUMC, Wald center, SLU, KALHD—The format is “unique and breaking new ground”
Others want to use this format, EM wants to use this format and assist when we move to larger and intentional events

## Challenges

- Facilitators lacked formal exercise design education
- Designing e-Exercise consistent with disease process
- Determining mode of communication during e-Exercise
- Developing and implementing new training
- Facilitators’ time not 100% devoted to exercise development

**Notes:**
Problems with any project...had planned on getting training out via KS-TRAIN and other routes for counties prior to beginning of exercise, but several training opportunities didn’t come together.  However, in process is a disease investigation course on TRAIN, Risk Comm plan development using CDCynergy through TRAIN, developing a live public health Incident command course, presenting Epi 101 at multiple sites and through multiple organizations (KDHE, Public Health Certificate Program [Hospitals?]).  All of these are in process for 2005-2006.
Mode communication, e-mail? Fax? Snail mail? Phone? New secure Emergency preparedness system?  PHIX?  Needed something that was timely that all participants could use/were familiar with and would have access to. 
Facilitators had other work duties
Facilitators wanted to find out if participants knew to take in an outbreak/ph emergency.  Wanted to get at the process, not just the outcome.
Facilitators wanted to prevent bias in participant answers by requiring scene responses by a certain day and time.
Facilitators wanted to keep everyone involved and to test regional cooperation
Evaluation of the outcomes was tricky, difficult to determine what a success was or what a failure was.  Is it a success to find out that no one is prepared or is it a failure of the systems being tested?  If everyone is answering correctly, is the exercise too easy or are we providing too much help?  Etc.

## Challenges, cont

- Keeping exercise details a secret
- Creating questions that had participants explaining process
- Timing of injects & responses to avoid suggesting the correct course of action
- Developing a scenario that would effect each county

## Conclusions

- Improved utilization of regional BT coordinators
- Re-evaluation of regional roles & responsibilities
- Identification of new external partners for large-scale public health emergency
- Suggested modification of disease investigation resources & guidelines

**Notes:**
Learned through the pre-test (Zoomerang.com bouncing) that e-mail may not be the most reliable way to communicate with the counties.  Messages sent by e-mail may not be received due to firewalls, limited mailbox capacity (Storage space), etc. [technology issues]
Developing a unified message, how to share existing tools with regional partners, how to troubleshoot as a group, reinforce support for each other, how to collate data to see trends/patterns,  
Having security at Ig clinics, how to transport Ig, re-established connections with BCH, 
Both lack tools for patron recalls

## Future Actions

- Incorporate feedback to improve             e-Exercise
- Consider other electronic systems for interactive e-Exercise participation
- Move to next level of e-Exercise design
- Intentional events vs. unintentional events
- Create library of scenarios for future        e-Exercises
- Develop strategies for sustainability

**Notes:**
From conversations w/ locals and other stakeholders
Utilizing a state’s health alert network reduces participants ability to communicate with each other during an exercise

## Questions?

## Additional Information

## Modifications to Exercise

- _Had to correct minor errors_
- Corrected a fax number
- Corrected information on food worker sheet
- Corrected date on food worker lab report
- Risk Communication observer was ill
- Clarified food worker information during conference call w/ counties

**Notes:**
These were found immediately after roll out.  The were corrected before they had a large impact on future scenes.
Had to bring “back-up” risk comm person in as a trusted agent
Original nurse simulator had army duty and was replaced by another KDHE staff member.
Had to provide information about the shift worked by the infectious food worker during a conference call with region b/c we didn’t provide this information in any other documents/resources.

## Pre-Training Survey Results

- _Participants Reporting “__**Never**__” Using Available Technology/Resource_
- PHIX (24%) – lack of access
- KALHD disease investigation protocols (28%)
- Foodborne Illness and Outbreak Investigation Manual (24%)
- HAWK (68%)

## Post-Exercise Survey Results

- 	_Participants Reported Needing More Training Resources:_
- Foodborne Manual (46%)
- Kansas TRAIN (59%)

## Decisions - Actions

## New Partnerships