# Presentation: 492331

## Overview of the Pediatric Indicator Module

- Presenters: Kathryn McDonald and Sheryl Davies,
- Stanford University
- AHRQ QI User Meeting
- September 26-27, 2005

## Acknowledgements

- Pediatric Module Development:
- Kathryn McDonald, Stanford University
- Patrick Romano, UC-Davis
- Sheryl Davies, Stanford University
- Amy Ku, Stanford University
- Kavita Choudhry, Stanford University
- Jeffrey Geppert, Battelle Health and Life Sciences
- Corinna Haberland, Stanford University
- Support for Quality Indicators II (Contract No. 290-04-0020):
- Mamatha Pancholi, AHRQ Project Officer
- Marybeth Farquhar, AHRQ
- Mark Gritz and Jeffrey Geppert, Project Directors, Battelle Health and Life Sciences

## Children’s Hospitalizations, US 2000

- spinningwheelalpacas.com

- chkd.com/images/HospitalVisit.jpg

- _Children’s Hospitalizations, US 2000_
-   6.3 million
-   $46 billion
-   36% of 1-17 yr olds in Children’s hospitals

## Unique Population

- Dependent on adults
- Constantly developing
- Demographics
- Epidemiology
- Coding in pediatrics
- Simpson LA, al DDe. Measures of Children's Health Care Quality: Building towards Consensus. Manuscript in preparation: Background paper prepared for National Quality Forum; 2003 September 19.

**Notes:**
Tailoring indicators for children’s health care is also important due to the unique aspects of the pediatric population.

Dependent on adults – require adults to provide home care, take to doctor, and for basic needs. Adds another link to the care chain.
Constantly developing – physiologically, psychologically and intellectually changing in relatively short period of time. 
Demographics – variety of ages, with different physiologic and social needs. Also, children are more likely than adults to be poor.
Epidemiology -  children are generally well, very few get hospitalized. Those that do get hospitalized are complex cases.
Coding – because not overseen by CMS, less priority for updating codes, less specificity, potentially poorer coding practices. 

## Current Measurement State

- Simpson and colleagues search

- Simpson LA, et al. Measures of Children's Health Care Quality: Building towards Consensus. Manuscript in preparation: Background paper prepared for National Quality Forum; 2003 September 19.

**Pediatric indicators**

**Inpatient**

**Small subset (~10) feasible with restricted data**

**Notes:**
WILL NEED TO UPDATE THIS

XXX total indicators identified
XXX inpatient indicators
 10 feasible using administrative data

Few feasible indicators already developed. 

Most indicators are focused on outpatient care. This makes sense because of the wellness of the pediatric population, but pediatric hospitals have then to rely on adult indicators. Special population of hospitalized children require specialized indicators. 

## Pediatric Applications of AHRQ QIs

-  Miller et al., Sedman et al., NACHRI chart reviews
**Lessons learned**
- Complications DO occur in children
- Some complications clinically different
- Some indicators perform differently in kids or rare with current exclusions
- Death related PSIs seemed less useful as defined in kids

**Notes:**
A few studies have applied the AHRQ PSIs to children (NOTE: Use of the PQIs for children are documented in the literature, but no studies have specifically used the AHRQ QI versions)

Miller and Sedman compared rates with outcomes and found that errors do occur and for most indicators, more resource use (LOS) occurs. NACHRI chart review underway with a coalition of hospitals, feedback from these efforts have highlighted some considerations. 

DO occur: The complications are important for kids, especially since mortality is rare
Some comps were found to be clinically different in kids, like Postoperative DVT/PE. Really these are line DVTs/PEs
A few indicators, because of limiting the population to low risk patients, are more rare in children (decubitus ulcer). Almost all adults admitted to hospital are at risk and prophylaxis considered for many comps. For kids prophylaxis is not recommended but for high risk patients (e.g., PO hemorrhage/hematoma) so the high risk kids is where the action is...
A few comorbidities are more important in kids. For instance, patients with urinary tract abnormalities make up a much larger proportion of UTIs than in adults, so while leaving these patients in adult indicators doesn’t do much to rates, really affects pediatric indicators. 
Death indicators are not very useful as defined. Kids don’t die, and cases identified were very high risk cases. 

## Indicator Module Development

**Literature**

**Actual Use**

**Concept**

**SOURCES**

**Candidate **
**Indicators**

**Evaluation**

**Selection**

## Framework for Assessing Pediatric Indicator Validity

- Face validity/consensual validity
- Does the indicator capture an aspect of quality that is important and subject to provider control?
- Precision
- Is there substantial “true” provider-level variation?
- Minimum bias
- Is it possible to account for differences in severity of illness that could potentially confound comparisons across providers?
- Construct validity
- Does the indicator identify quality of care problems that are flagged or suspected using other methods?
- Fosters real quality improvement
- Is the indicator unlikely to be gamed or cause perverse incentives?
- Application/experience
- Is there reason to believe the indicator will be feasible and useful?

**Notes:**
We are applying the same framework used to develop the PQIs, IQIs and PSIs to assess potential pediatric quality indicators.

## 
Literature review
To identify quality concepts and indicators 
To determine previous work on indicator validity

Hospital ICD-9-CM coding review
To ensure proper definition (correspondence between clinical concept and coding practice)

Clinical panel reviews 
To refine indicator definition and risk groupings
To establish face validity when minimal literature 

Empirical analyses
To explore alternative definitions
To assess nationwide rates, hospital variation, relationships among indicators
To develop appropriate methods to account for differences in underlying risk

- Literature review
- To identify quality concepts and indicators
- To determine previous work on indicator validity
- Hospital ICD-9-CM coding review
- To ensure proper definition (correspondence between clinical concept and coding practice)
- Clinical panel reviews
- To refine indicator definition and risk groupings
- To establish face validity when minimal literature
- Empirical analyses
- To explore alternative definitions
- To assess nationwide rates, hospital variation, relationships among indicators
- To develop appropriate methods to account for differences in underlying risk

**Indicator Development**

**Notes:**
Similarly, we are using the same general research protocol for developing indicators.

## Phased Evaluation

- Phase I
- Current AHRQ QIs
- Eliminate QIs covering adult only chronic illnesses or those with questionable validity for kids
- Phase II
- Novel indicators
- Require development or updating

**Notes:**
Because of interest in from the field for adapting current indicators quickly, and because new indicators needed more development time (not too many we can take off the shelf), planned a 2 phase process. 

## Example Indicator Evaluation

**Notes:**
Indicators go through many iterations to obtain final definition

## Slide 12

## 

Decubitus ulcer

Patients with secondary dx 707.0 per 1000 patients

Exclude high risk patients: Transfers from long term care facility, paralysis

***Decubitus ulcer***
- Patients with secondary dx 707.0 per 1000 patients
- Exclude high risk patients: Transfers from long term care facility, paralysis

- EXCLUDE SPINA BIFIDA PATIENTS

- Literature Review and User Data

**Notes:**
We will go through the development of decubitus ulcer as an example. 

We started with the current AHRQ QI definition. From user feedback we found that the indicator included many spina bifida patients in pediatrics (example of a comorbidity being more important in kids), so because risk is similar to paralysis we added exclusion for these patients. 

## Initial Empirical Results

- Rates by age group and high risk groups
- Higher rate in higher age groups
- Ulcers occur more frequently in high risk groups but some occur in traditionally low risk
- Lower rate in premature neonates
- Rates are provided without commentary to panelists prior to conference

**Notes:**
We calculated rates to make sure the indicator behaved as desired. Noticed that rate in premature infants with fragile skin is low – first clue to terminology differences. 

## Medical/Surgical Panel Composition

**	****Specialty****	****	****	****	****	****Location**
- Pediatric Emergency Medicine			Dallas, TX
- Thoracic Surgery, Congenital Heart Surgery	Washington, DC
- Neonatology					Seattle, WA
- Neonatal & Pediatric Nursing			San Francisco, CA
- Pediatric Surgery, Surgical Critical Care		New Haven, CT
- Pediatric Critical Care				Louisville, KY
- Pediatric Infectious Disease			Augusta, GA
- Pediatric General Surgery			Nashville, TN
- Pediatrics					Valhalla, NY
- Pediatric Radiology, Diagnostic Radiology	Seattle, WA
- Pediatric Oncology				New York, NY
- Hospitalist					Philadelphia, PA

**Notes:**
Went to a panel with diverse specialty and geographic characteristics. 

## Panel Evaluation

- Expand population to INCLUDE high risk populations
- Prefer stratification scheme
- Skin breakdown in neonates

**Notes:**
The panel felt that the indicator was most useful when tracking high risk populations, including patients with hemiplegia, paraplegia, quadriplegia (e.g., due to cerebral palsy), spina bifida, muscular dystrophy or glycogen storage diseases and neurodevastation due to trauma. These patients are at high risk for developing ulcers due to neurologic impairments, and as a result may have ulcers present on admission. Despite the inability to easily distinguish ulcers present on admission, panelists felt they desired to have separates rates available for high risk and lower risk patients.
Panelists noted that “skin breakdown” or “decubiti” in newborns rarely stem from gravity related causes, but rather from friction from equipment and other processes. These sores are rarely identified as decubiti, but rather as skin breakdown and panelists felt these are likely to be coded differently. Coding consultation will help inform the inclusion of newborns in this indicator.
Given the need to accurately identify premature infants and light for gestational age infants as requested, additional definitional work is required and this indicator was not re-rated by panelists. It will be re-examined in the second round of pediatric indicator development. 

## Post-Panel Investigation

- Empirical analyses
- Examine rates of decubitus ulcer in potentially high risk groups.
- Identify similar risk strata
- Coding consult
- Understand coding guidelines for infants with “skin breakdown” or decubiti

**Notes:**
NEED TO ADD SOME NOTES

## Example Evaluation

- Revised Definition for Decubitus Ulcer
- Patients with secondary dx of 707x per 1000 patients
- Exclude patients transferred from long term care facility and _another acute care facility_
- _Stratify by:_
- _Low Risk_
- _High risk (paralysis, spina bifida, anoxic brain damage)_

**Notes:**
As a result of all evidence, we changed definition to following: 

And because of the massive shift in the concept for this indicator, this one will go back for a second panel review. 

## Slide 19

## Slide 20

## ResultsOverarching Themes

- High risk populations are important in children
- Bias and risk groups
- Expanded data
- Application of indicators key
- Feedback and validity testing key

**Notes:**
High risk – complications are very unlikely to occur in uncomplicated pediatric cases, unlike adult cases. Therefore, prophylactics are reserved and aid in complicated cases. Should expand.

Bias – expanding increases bias. Stratification preferred (more discussion later)

Expanded data – desired to have more data to improve indicators (linked, present on admission), expand indicators (readmission)

Application – The actual use of the indicators must be kept in mind during evaluation. Different thresholds

Feedback – user feedback and validity testing must continue and is invaluable. 

## Types of Modifications Made to QIs

- Expand population at risk
- Decubitus ulcer, postoperative sepsis
- Restricted age range
- Transfusion reaction, Diabetes, Asthma, Perforated appendix
- Exclusion of normal newborns
- Stratification/split
- Iatrogenic pneumothorax, Accidental puncture laceration, Post-op hemorrhage/hematoma
- Added exclusion criteria
- Post-op wound dehiscence, Post-op respiratory failure, UTI
- Modified numerator
- Gastroenteritis

**Notes:**
Not all indicators and types of changes to each are represented on this slide. We’ve selected several examples to show the range of reasons for modifications.

Decubitus ulcer, postoperative sepsis – we expanded to high risk groups
Transfusion reaction – eliminate newborns because of coding validity issues
Diabetes – eliminate under 6 to reduce number of initial admissions included
Asthma – difficult to diagnose as asthma in younger patients
Perforated appendix – difficult to prevent in young patients because they can’t tell you symptoms
Excluded normal newborns when this group diluted the denominator
Iatrogenic pneumothorax – clinically different in neonates and older patients
Accidental puncture laceration –stratify by procedure type because of different risk
PO hemmorhage/hematoma – stratify by high risk vs. low risk patients by diagnosis
PO wound dehiscence – wanted to exclude immunocompromised state, but not really at higher risk
PO respiratory failure – exclude patients with neuromuscular disorders, prolonged state on respirator
UTI – exclude patients with structural problems that make more severe UTIs likely – not measure of outpatient care
Gastroenteritis – combined with dehydration indicator

## Indicators Not Recommended

- Clinically different in children
- Likely to occur in complex cases in children/ preventability questionable
- Coding concerns
- Bacterial pneumonia
- PO physiologic and metabolic derangement
- Combined with other indicator, remaining cases not useful
- Dehydration

**Notes:**
Some potentially applicable AHRQ indicators are not being recommended for inclusion in the pediatric module. For example, FTR is an unproven concept in pediatrics. Similarly, ... NEED TO ADD TEXT

## Rates per 1000Procedure-related Complications

**Notes:**

These are indicators which occur during procedures. Because of difficulty identifying minor procedures (may not be coded), these indicators include all patients). 
Points:  QIs occur at various frequency. 

## Rates per 1000Complications in All Patients

**Notes:**
These are QIs that could occur in any patient
Most are under development (stratification scheme).

## Rates per 1000Postoperative Complications

**Notes:**
Post-operative complications
PO hemorrhage highlights the importance of stratification. For PO hemorrhage the overall rate is 2.12. However, when stratified, you see that the rate is 21.46 is the high risk group, but only 1.92 in the low risk group. 

PO wound dehiscence is more rare in children than in adults. Because of this (likely) the clinically coherent high risk group this time around has basically the same risk as the intermediate risk (we don’t need to say this, but just in case the question comes up).

## Rates per 100,000 populationPotentially Avoidable Hospitalizations

**Notes:**
Potentially preventable hospitalizations. Note that cannot isolate the at risk population for diabetes or asthma. 
Perforated appendix is per 100 appendicitis.

## Rates (%) Mortality Indicators

## Dealing with Bias

- Stratification
- Clinically transparent, actual numbers
- Low numbers, overwhelming number of results
- Risk adjustment
- Allows for comparisons
- Full adjustment impossible, black box
- Exclusions
- Easy comparisons, complex cases avoided
- Low numbers, leaves out cases important to prevent

**Notes:**
There are alternative for dealing with bias that can be used alone or in combination. Each approach has pros and cons.

## Risk Adjustment

- Reason for admission/ type of procedure
- DRGs
- Comorbidity
- Must develop de novo
- SES risk adjustment
- Not unique to kids, but may over-adjust

**Notes:**
Three things looking to adjust for...

## Phase II: Novel Indicators

- Literature review
- Organization contact
- Federal agencies, professional organizations, advocacy groups, provider organizations
- 100+ contacted
- Most indicators submitted not feasible given data constraints

## Indicators Under ConsiderationAmbulatory Care

- Cellulitis hospitalization rate
- Hospital admissions for influenza-related conditions, age 6-23 months
- Immunizable condition hospitalization rate

## Indicators Under ConsiderationNeonatal

- Intraventricular hemorrhage
- Respiratory distress syndrome
- Chronic respiratory disease
- Meconium aspiration syndrome rate
- Nectrotizing enterocolitis
- Neonatal mortality
- Nosocomial bacteremia
- Proportion of VLBW infants born at Level III centers
- Retinopathy of prematurity

## Indicators Under ConsiderationPatient Safety and Mortality

- Aspiration pneumonia
- Postoperative pneumonia
- Catheter-associated venous thrombosis
- Other postoperative metabolic derangements (hyponatremia, hypernatremia)
- Trauma mortality

## Phase II: Next Steps

- Literature reviews
- Update existing definitions
- Develop and test definitions using administrative data
- Panel review
- Reformulation of indicators
- Development and release of new software

## Timeline

- January 2006
- PedQI software release with current AHRQ QIs adapted for pediatric cases
- Fall/Winter 2005
- PQI, IQI, PSI updates converted to adult population focus
- Early 2007
- PedQI update with new indicators

## Implications

- AHRQ PSIs, IQIs and PQIs
- No longer apply to children, though concepts retained in PedQI
- Children’s vs. community hospitals
- Focus on strata for stratified indicators
- Compare results within peer groups
- Request to users
- Monitoring of coding practices essential
- Communication to AHRQ about early experiences

## Acknowledgments

**Funded by AHRQ**
- Support for Quality Indicators II (Contract No. 290-04-0020)
- Mamatha Pancholi, AHRQ Project Officer
- Marybeth Farquhar, AHRQ QI Senior Advisor
- Mark Gritz and Jeffrey Geppert, Project Directors, Battelle Health and Life Sciences
**Data used for analyses:**
- Nationwide Inpatient Sample (NIS), 1995-2000. Healthcare Cost and Utilization Project (HCUP), Agency for Healthcare Research and Quality
- State Inpatient Databases (SID), 1997-2002 (36 states). Healthcare Cost and Utilization Project (HCUP), Agency for Healthcare Research and Quality

## Acknowledgements

- We gratefully acknowledge the data organizations in participating states that contributed data to HCUP and that we used in this study: the Arizona Department of Health Services; California Office of Statewide Health Planning & Development; Colorado Health & Hospital Association; Connecticut - Chime, Inc.; Florida Agency for Health Care Administration; Georgia: An Association of Hospitals & Health Systems; Hawaii Health Information Corporation; Illinois Health Care Cost Containment Council; Iowa Hospital Association; Kansas Hospital Association; Kentucky Department for Public Health; Maine Health Data Organization; Maryland Health Services Cost Review; Massachusetts Division of Health Care Finance and Policy; Michigan Health & Hospital Association; Minnesota Hospital Association; Missouri Hospital Industry Data Institute; Nebraska Hospital Association; Nevada Department of Human Resources; New Jersey Department of Health & Senior Services; New York State Department of Health; North Carolina Department of Health and Human Services; Ohio Hospital Association; Oregon Association of Hospitals & Health Systems; Pennsylvania Health Care Cost Containment Council; Rhode Island Department of Health; South Carolina State Budget & Control Board; South Dakota Association of Healthcare Organizations; Tennessee Hospital Association; Texas Health Care Information Council; Utah Department of Health; Vermont Association of Hospitals and Health Systems; Virginia Health Information; Washington State Department of Health; West Virginia Health Care Authority; Wisconsin Department of Health & Family Services.

## Questions?