# Presentation: 142160

## AHRQ Quality Validation PilotProcess and Tools

- Patricia Zrelak, RN, PhD
- UC Davis Center for Healthcare Policy and Research
- AHRQ QI Users Meeting
- Bethesda, MD; September 28, 2007

## AHRQ Validation Pilot: Goals

- Gather evidence on the scientific acceptability of the patient safety indicators (PSIs)
- medical record reviews, data analysis, clinical panels, evidence reviews
- Consolidate the evidence base
- Improve guidance on the interpretation and use of the data
- Evaluate potential refinements to the specifications

## Pilot goals continued

- Develop medical record abstraction tools
- review potentially preventable adverse events
- identify potential opportunities for improvement
- Develop mechanisms for conducting validation studies on a routine basis
- collaborating with other organizations
- data collection and analysis
- ongoing evaluation and refinement

## Patient Safety Indicators

| Phase I | Phase II |
| --- | --- |
| Accidental puncture and laceration | Foreign body left in during procedure |
| Iatrogenic pneumothorax | Postoperative hemorrhage or hematoma |
| Postoperative pulmonary embolism (PE) or deep vein thrombosis (DVT) | Postoperative physiologic and metabolic derangement |
| Postoperative sepsis | Postoperative respiratory failure |
| Selected infection due to medical care | Postoperative wound dehiscence |

## Pilot timeline

- Collaborator recruitment
- September 2006 to October 2006
- Collaborator training; protocol and tool development and testing (Phase 1)
- November 2006 to March 2007
- Data collection application development and testing; data reporting (Phase 1)
- April 2007 to September 2007
- Analysis and assessment (Phase 1)
- July 2007 to November 2007

## Data Collection

- Chart abstraction by collaborators
- Trained via webinar
- Administrative data
- Cases were assigned based on a sampling probability using AHRQ QI software
- Medical record abstraction tools & guidelines
- Accidental puncture and laceration
- Iatrogenic pneumothorax
- Postoperative pulmonary embolism (PE) or deep vein thrombosis (DVT)
- Postoperative sepsis
- Selected infection due to medical care

##  Guidelines in tool development 

- Literature search – evidence based
- Alignment with related QI projects & initiatives
- Ex. Surgical Care Improvement Project (SCIP)
- Professional & regulatory guidelines
- Ease of use

## Guidelines in tool development

- Ongoing expert review process
- Healthcare and medical practitioners
- Quality experts
- Consultation with other experts as needed
- Local alpha testing and refinement
- Feedback from collaborator training
- Learnings from national pilot testing

## Recognizing limitations

- Chart review
- Not all data elements of interest are available
- Ex. hand washing, mask use & environmental factors
- Time constraints (burden on collaborators)
- Some items of interest are too time consuming to abstract  (e.g., lowest urine output)
- Reliability of certain data elements/differences in practice
- Ex. incentive spirometry & sequential compression devices
- Variability between healthcare systems
- Admission weights, temperature documentation in OR

- Section 1: Abstractor details
- Section 2: Record identification/validation
- Section 3: Ascertainment of the event(s)
- Was the patient eligible for the indicator?
- Did the indicator event happen?

**Generic structure of the data abstraction tool**

## Structure of collection tool continued 

- Section 4: Risk factors
- Section 5: Evaluation and management
- Characterization of the event
- Potential preventability of the event
- Section 6: Outcomes
- Impact of the event on the patient

## Section 1:  Abstractor details

- 1.1 	Date abstraction completed
- 		_ _ / _ _ / _ _ _ _
- 1.2  Abstractor identifier
- 		_P_ _A_ _Z_ _ _ _ _

## Section 2: Record identification/validation 

- Demographics
- AHRQ study ID, patient identification code, DOB, gender, dates of admission and discharge
- Criterion validation
- Was the correct chart abstracted (correct PSI, patient and admission)?
- Link to administrative data

## Section 3: Ascertainment of event

- _Criterion validation_ - whether cases flagged did or did not have the clinical event
- Inclusion and exclusion criteria from the  Patient Safety Indicator Technical Specifications
- Confirmation of the event and date
- Ascertainment of multiple procedures/events per discharge

## Ex.  Iatrogenic pneumothorax

- 3.1 Was the patient’s admission associated with any of the following conditions or procedures (before the date of the pneumothorax diagnosis)?  Check all that apply.
- Pregnancy, childbirth or puerperium
- Chest injury or trauma
- Pleural effusion
- Diaphragmatic surgery or repair
- Thoracic surgery (excluding bronchial procedures)
- Cardiac surgery
- Lung or pleural biopsy
- Operations on the esophagus
- Anterior thoracic spinal fusion or thoracic duct surgery
- None of the above
-       If YES to any of the above conditions, please describe the condition or procedure that apparently led to a pneumothorax in the TEXT BOX below and then END the abstraction.

## Pneumothorax continued

- 3.2 Did the patient have a pneumothorax or suspected pneumothorax at the time of admission?
- Yes
- No
-      If YES, STOP as this is an exclusion criterion.  Please describe the circumstances surrounding this pneumothorax at admission in the TEXT BOX below and then END the abstraction.

- 3.3  Did the patient experience a pneumothorax during this admission?
- Yes
- No
-        If NO, STOP as this is required for study inclusion. Please describe any abnormality or condition that might have been misinterpreted as a pneumothorax, such as pneumomediastinum or subcutaneous emphysema in the in the TEXT BOX below and then END the abstraction.
- 3.4  Document the date the pneumothorax was diagnosed.  Use the earliest date in the event of multiple pneumothoraces.
- 	_ _ /_ _ /_ _ _ _

**Ex.  Postoperative Pulmonary Embolism or Deep Vein Thrombosis**

## Section 4: Risk Factors

- Confounding - whether there are confounding factors that might be important for improving indicator specifications and for interpreting and using the AHRQ PSI rates

## Example: Selected Infections due to Medical Care 

- 4.1  Did the patient have any of the following immunosuppressive conditions on admission?  Check all that apply.
- Cancer
- HIV/AIDS
- Severe malnutrition
- Lupus or other autoimmune disease
- Sickle cell disease
- Nephrotic syndrome or chronic renal failure
- Short gut syndrome
- Immunoglobulin deficiency
- Transplant
- Other immunodeficiency (specify)

## Section 5: Evaluation and Management 

- Processes of care/process improvement
- Eligibility for interventions
- Did the patient receive interventions?
- How was the patient diagnosed?
- How was the complication managed once it occurred?

## Ex.  Postoperative Deep Vein Thrombosis or Pulmonary Embolus 

**5.7  How was the venous thrombosis diagnosed? **
**Duplex ultrasonography **
**CT scan **
**Contrast venography**
**By clinical suspicion alone**

## Ex.  Postoperative Deep Vein Thrombosis or Pulmonary Embolus

**5.8  What specific segment(s) of the venous system was/were identified to have thrombus?  Check all that apply. **
**Inferior vena cava**
**Iliac veins**
**Femoral veins **
**Popliteal vein **
**Deep lower extremity veins distal to the popliteal **
**Superficial lower extremity vein(s)**
**Superior vena cava **
**Brachiocephalic (innominate) veins **
**Internal jugular vein **
**Superficial neck vein **
**Subclavian or axillary vein **
**Deep upper extremity veins distal to the axillary vein **
**Superficial upper extremity vein **
**Critical documentation missing **
**Other (specify):****   **

## Ex.  Postoperative Deep Vein Thrombosis or Pulmonary Embolus

- 5.9  Was the thrombus occluding or non-occluding?
- 5.10  Based on diagnostic test documentation, how is the acuteness or age of the DVT reported.
- 5.12  ‘.....was the PE or venous thrombosis detected as a result of routine screening?’
- 5.13  .... ‘signs and symptoms present 48-hours prior to diagnostic studies’.

##  Section 6: Outcomes

- Impact on the patient
- The abstractor is asked to assess the documentation and render a judgment on the impact of the complication
- Ex.  Causative factor related to death,  readmission, increased length of stay, and/or transfer to a higher level of care.

## Ex.  Selected Infection due to Medical Care 

- 6.1  Does the chart suggest that the patient suffered any adverse effects or consequences from this infectious or inflammatory process?  Check all that apply.
- Additional pain or discomfort
- Extended length of hospital stay
- Underwent an operating room procedure to treat infection (e.g., incision and drainage, excision)
- Residual disability or impairment of normal function (at discharge)
- Readmission
- Death
- None or the above  or not specified

## Last question of every tool

- If there are special circumstances or comments related to this case that you feel are important that were not captured in the survey, please state in the TEXT BOX.
- DVT/PE example:  The patient experienced a postoperative PE-but is was from a septic emboli.

##          Guidelines

**Guidelines for Validation **
**of ****Selected AHRQ Quality Indicators**
**(Version 6.2_04/5/2007) **

- PSI 6:  **Guideline for Iatrogenic Pneumothorax  **