# Presentation: 714744

## Criteria for Prioritizing and Characterizing Statistical Anomalies Identified in BioSense

- Colleen Martin, MSPH1, Jerome Tokars, MD MPH1, Gabriel Rainisch, MPH2, Jacqueline Burkholder, PhD MS2, Kali Crosby, BSN RN2
- 1 Division of Emergency Preparedness and Response
- National Center for Public Health Informatics
-  Centers for Disease Control and Prevention
- 2 Constella Group, LLC

## BioSense System

- National near real-time biosurveillance system to enable public health situational awareness
- Current data sources: >370 hospitals, >1100 ambulatory care Departments of Defense (DoD) and Veterans Affairs (VA) medical facilities, large national laboratory
- Data stored, analyzed, and visualized in a web-based application

## CDC BioIntelligence Center (BIC)

- Team of analysts that:
- Monitor, analyze, and interpret national BioSense data
- Examine trends and statistical anomalies
- Identify problems and needed system enhancements
- Provide training and technical support for users

## Statistical Analysis

- Algorithms analyze syndrome and sub-syndrome counts and rates to identify statistically significant increases.
- Hospital data:
- Modified C2 (W2)
- Unit of analysis: facility, all facilities in a jurisdiction
- VA and DoD data:
- Modified C1, C2, C3 and SMART model
- Unit of analysis: zip code, all visits in a jurisdiction
- A large number of statistical anomalies are identified daily at the national level!

## Rationale for a Standard Process

- Identify a subset of potentially important anomalies
- Apply objective criteria for characterizing each anomaly
- Decrease the amount of time a BIC analyst spends characterizing anomalies
- Decrease the variability in decision-making among BIC analysts
- Ensure that the BioSense application can support anomaly characterization and epidemiologic decision-making

**Notes:**
WORK IN PROGRESS!!!!!!!!!!!!

## Objectives

- To describe a set of objective criteria and standard process for identifying and characterizing potentially important anomalies
- To compare the criteria with several recent public health events

## Positive Criteria

**Patient Class Clustering**: Anomalies for >1 patient class (outpatients, ED, inpatients)
**Temporal Clustering**: Anomalies for _>_ 2 of past 3 days
**Recent Maximum**: Maximum rate or count within the last 6 months (or since the facility began sending data for facilities with < 6 months of data)
- Seasonal Maximum: Exceeds rate or count and rate during the same 4 week enclosing time period the previous year
**Severe Illness**: _>_ 1 patient also maps to the severe illness/death syndrome

## Positive Criteria Caveats

- Patient class clustering criterion cannot be met by:
- DoD and VA that send outpatient data only
- Hospitals that send ED data only
- Seasonal maximum criterion cannot be met by:
- Any facility that has not been sending data for _>_ 13 months
- Non-seasonal syndromes

## Descriptive Criteria

- 50% of patients have chronic co-morbidities
- 50% of patients have chief complaints and/or diagnoses that map to the same sub-syndrome
- Distribution of _>_ 1 age strata is 25% higher than that strata during the 4 weeks prior to the anomaly
- Distribution of _>_ 1 sub-syndrome is 25% higher than that strata during the 4 weeks prior to the anomaly

## Negative Criteria

- Inconsistent data across application modules
- Rate-based anomaly no longer statistically significant due to denominator increase
- Holiday effect
- Data feed disruption depressed the expected value during the algorithm’s baseline period
- Coding and/or data entry errors
- Flaws in syndrome/sub-syndrome mapping methods
- Anomaly flagged by working diagnosis but negated by final diagnosis

## Identify anomalies based on rate, ED or inpatients

- Does the anomaly meet _>_1 positive criteria?

- Yes

- No

- Characterize the anomaly

- Do not characterize the anomaly

- Does the anomaly meet _>_1 negative criteria?

- Yes

- Rule out the anomaly

- No

- Does the anomaly have potential PH significance?

- No

- No action taken

- Contact state and/or local PH per jurisdictional protocols

- Yes

- CONTEXT
- News, Epi-X, partners

- CONTENT
- Clinical, epidemiologic

**Notes:**
Anomaly dataset that feeds the BioSense application is provided to the BIC using automated processes.
Anomaly dataset is de-duplicated, filtered, and entered into the Anomaly Database.
	For specific facilities
	For ED and inpatients, use outpatient data for corroboration
	That are statistically significant based on rate
Anomalies are prioritized and characterized based on a set of standard criteria.
Each criteria has an objective definition and can be applied using modules within the BioSense application.
Analysts make decisions regarding an anomaly’s potential public health importance and the appropriate corresponding action.
Statistical anomaly information and associated criteria, decisions, and actions are entered into the Anomaly Database.
If the anomaly is deemed to be of potential importance, the analyst communicates their findings to the team.
If deemed appropriate, the analyst contacts the state and/or local health department per jurisdictionally defined protocols, gives a summary of relevant findings, and provides any needed analytic support.

## Anomaly Database

## Methods

- January to July 2007, 6 public health events were confirmed through communications with state and local health departments.
- We examined data anomalies identified during the event periods to determine which of the positive criteria were met on each anomaly day.
- We applied the criteria prospectively during 4 events.
- We applied the criteria retrospectively for 2 events.
- Data as they would have appeared at the time of the event were used.

## Events

- Gastrointestinal outbreak at DoD facility triggered 9 data anomalies during 1/25/07 to 2/10/07.
- Fever clusters at several hospital EDs triggered 9 data anomalies during 1/24/07 to 2/11/07.
- Fever clusters at several hospital EDs triggered 5 data anomalies during 1/30/07 to 2/8/07.
- Rash cluster at one hospital ED triggered 1 data anomaly on 3/15/07.
- Gastrointestinal cluster due to synthetic data for a drill at one hospital ED triggered 1 data anomaly on 6/2/07.
- Respiratory outbreak at DoD facility triggered 8 data anomalies during 6/20/07 to 7/9/07.

## Slide 15

| Event
(# Anomaly Days) | Patient Class Clustering | Temporal Clustering | Recent Maximum | Seasonal Maximum | Severe Illness | No Positive Criteria |
| --- | --- | --- | --- | --- | --- | --- |
| GI (9) |  | 6 days | 2 days | 7 days |  | 1 day |
| Fever (9) |  | 5 days | 5 days |  |  | 2 days |
| Fever (5) |  | 3 days | 3 days |  |  |  |
| Rash (1) |  |  | 1 day |  |  |  |
| GI drill (1) |  |  | 1 day |  |  |  |
| Respiratory (8) |  | 4 days | 2 days | 5 days |  | 1 day |

## Results Across Events

- 6 events triggered anomalies on 33 days.
- 29 (88%) of the 33 days, _>_1 positive criteria was met.
- 0 anomalies met any of the negative criteria.

## Discussion

- Although data anomalies are common, it is rare that any are associated with true public health events.
- The BIC is primarily interested in prioritizing anomalies that are large in magnitude compared to historical data, involve multiple jurisdictions, or demonstrate unusual illness severity or other characteristics.
- Our positive criteria were useful for identifying statistical anomalies associated with six confirmed events that involved sudden, large increases above baselines.
- If deemed useful, it might be possible to enable users to filter and characterize anomalies in the BioSense application based on these or other criteria.
- This is a work in progress!

## Acknowledgements

- CDC
- Roseanne English, Paul McMurray, Kaipo Akaka
- Constella Group, LLC
- Jacqueline Burkholder, Kali Crosby, Matthew Miller, William Lei, Nimeshkumar Patel, Gabriel Rainisch, Courtney Rodi, Ericka Sinclair
- Science Applications International Corporation
- Steve Bloom
- State and local public health partners
- BioSense Monitoring Protocols Working Group

## Thank You

- Colleen Martin, MSPH
- _[Cmartin5@CDC.GOV](mailto:Cmartin5@CDC.GOV)_
- 404-639-7612
- _[www.](http://www.cdc.gov/biosense)__[cdc](http://www.cdc.gov/biosense)__[.](http://www.cdc.gov/biosense)__[gov](http://www.cdc.gov/biosense)__[/](http://www.cdc.gov/biosense)__[biosense](http://www.cdc.gov/biosense)_
- _[BioSenseHelp](mailto:BioSenseHelp@CDC.GOV)__[@CDC.GOV](mailto:BioSenseHelp@CDC.GOV)_