# Presentation: 137409

## The Facts Don’t Speak For Themselves: 

- AHRQ 2007

- HS Kaplan
- R Levitan
- B Rabin Fastman
- CUMC/NYPH

- Getting the Story from Aggregate Data

- AHRQ 2007

## Event Reports as Rumble Strips

- Both can help increase safety by revealing danger
- Neither is reliably quantitative
- Both may create some unwanted noise in the environment

**Notes:**
How do you help a drowsy or distracted driver avoid crashing? One strategy is to install rumble strips on the roadway shoulders. The sound and vibration can wake a sleeping driver-giving him or her time to regain control of the vehicle.

## Counting: A Means to an End

## Steinbeck on Counting

## Focus of Today’s Presentation

-    Discuss real-time database queries using built-in tools:
**RAI**: Risk Assessment Index
- “**Single Click**” Standard reports
**QBF**: Query by Field
**Data Mining: Clustering, CBR, and HAWK** –
- Constraint - **Access**: Permissions and roles

## Access: Permissions and Roles

- User-based
- Role within organization
- Role within MERS system
- Overarching access rule set
- Location
- Service line/Department
- Employee events/Patient complaints
- Type: falls, meds, transfusion, equip, etc.
- Read-only
- HIPAA

**Notes:**
MERS at Columbia

- Filter no-harm reports to improve signal-to-noise ratio (SNR)

- Risk Assessment Index

## Two Filters to Enhance SNR

**Impact**

**Frequency**

**RISK**

**Low**

**High**

**Medium**

**Notes:**
These two steps, or  tools, to enhance the signal and diminish the noise are use of a risk matrix and barrier analysis. This is a process of simplification to ”eliminate the unnecessary so that the necessary may speak” (Hans  Hoffman, Abstract expressionist). The Risk Assessment Matrix is derived from MIL-STD-882B. The matrix provides a systematic method for assigning a risk level to a failure event based on the severity of the danger and likelihood of the event’s occurrence.. 
Hazard is a danger, whereas risk takes into account the danger and the likelihood.

Barrier analysis:  Initial screen is based on matrix value (index) for the initiating event – assuming no recovery step occurred. This value, however, remains unchanged if the recovery step was an unplanned one.
Conditional on another variable?
Dependant on single human performance?
If computer supported, is it readily over-ridden?
Failure readily detected?
Planned or Unplanned?

## “Single Click” Standard Reports

- Ad-hoc reporting in real-time
- MERS has a comprehensive list of reports

## “Single Click” Standard Reports

## Medication Events by Specific Type

## Query By Field (QBF): Exact field matches

- QBF’s filtered results can be fed into any report, graph, or spreadsheet

## Generation of Graphicson data subset using QBF filter

## Medication Drill-Down: Categories of Ordering Errors

## Benchmark Against Total and Other Reporting Sites/Hospitals

**Notes:**
DELETE?

## Real-Time, Formatted Spreadsheets

## User-Customized Spreadsheets

## Data Mining

- Mining Association Rules

- Decision Trees

- ID

- Clustering

- Statistical Clustering

**Data Mining**

- CBR

- Similarity Matching

- Textual
- Numeric
- Semantic

- Neural Networks

- © 2007 by The Trustees of Columbia University in the City of New York.

- CBR

**Notes:**
What Is Data Mining? Attempts to identify interesting information and patterns in large databases
Uses complex tools

## Why Cluster?

- Clusters show us event reports that are similar across predefined dimensions
- They may represent:
- frequency of a type of event
- event trends in time
- potential prevention, etc

- © 2007 by The Trustees of Columbia University in the City of New York.

**Notes:**
Read first bullet.
Frequency of a type of event: Many of the same type of incidents can be a bad sign. A rare incident may also be noticed.
Event trends over time: You could plot similar events in a cluster over time to see if things are getting better or worse.
Potential prevention: You can see what has worked in previous, similar incidents in the past and try to apply it in new events.
One way to create statistically significant clusters is to start with all the data in one cluster, and then systematically (using an algorithm) add to or remove elements from the cluster, so that its statistics improve.

## Case-Based Reasoning (CBR)

- What is case-based reasoning?
- Case-based reasoning is another methodology for, among other things, identifying clusters of *similar* events in large databases

- © 2007 by The Trustees of Columbia University in the City of New York.

**Notes:**
Case-based reasoning is a problem solving paradigm which provides the foundation for a technology for intelligent systems. It is based on the notion that human expertise is not based on rules or other formalized structures, but on experiences. 
With CBR, we try to relate new problems to previous ones. By reasoning based on analogies between current and old problems, you can draw solutions from old experiences. In order to use case Based Reasoning, you need to retrieve similar events from the past.  
Read bullet.

## Clustering/CBR

- Clustering divides large data sets into coherent subsets that can be studied more easily
- Given an event report, CBR will
- go through all event reports in database
- compute similarity between them
- find all reports within a certain distance or similarity (defined by the user)
- These reports form a **cluster**

- © 2007 by The Trustees of Columbia University in the City of New York.

**Notes:**
Clustering Algorithms
There are many algorithms used to create clusters
Here we will discuss :
case-based reasoning

As an overgeneralization, all clustering algorithms basically do what was described in the previous slides: they divide the data into subsets based on some criterion of “distance.” The two techniques presented here use different definitions of “distance.”  Statistical clustering uses numerical distance, while case-based reasoning uses distance between semantic concepts.

## CBR and Similarity Matching

- Using CBR, the *computer system* can establish the closest matches to any target event
- It can *cluster* based on *similarity*
- It can also identify unique events

- © 2007 by The Trustees of Columbia University in the City of New York.

- CBR and Similarity Matching

**Notes:**
Using CBR, the system compared report #5 to all other reports in the database, and using mathematical algorithms, computed the degree of similarity to the other events.  The system looked at numerical as well as semantic similarities.  It established that report #52 had the highest match (85%) to event #5 (the target event).

## HAWK

- MERS’ similarity function, HAWK, uses a vector of pre-assigned weights that corresponds to the vector of variables in an event report record.
- HAWK provides information that can be used to evaluate trends

**Notes:**
In case-based reasoning, a casedescribes a datum, i.e., a list of attribute-value pairs
An attribute is an element of an event
Its value is the actual value for the attribute in the specific case
For a medical event report, case attributes/values could be
who discovered the event/MD, RN, QA, Manager
medication involved/analgesic, antibiotic, anti-hypertension
point in the process it was discovered/admission, perioperative, radiology
etc.

## CBR: Another Use

- CBR can be extended to provide solutions to problems based on past experiences in the database
- e.g., a help desk

**Notes:**
Read slide.
When you call a help desk, it is probable that the operator enters your event information into a CBR-based computer program. The database will be searched for similar events, and solutions to these past similar events will be presented as possible solutions to your current problem.  

## The Facts Don’t Speak For Themselves:

*“**Knowledge resides in the user and not in the collection.”*
*	*

- C. West Churchman* *in* The Design of Inquiring Systems*