# Presentation: 724767

## Development & Evaluation of the Forthcoming AHRQ Neonatal Quality Measures

**AHRQ 2007 Annual Conference, Session 3C**
**Overview of the AHRQ Quality Indicators**
**Patrick S. Romano, MD MPH**
**On behalf of the AHRQ QI Support Team**

## AHRQ Quality Indicators (QIs)

-  **Pediatric Quality Indicators – **
-  Phase I
- - Adapted pre-existing AHRQ measures
- - Released February 2006
-  Phase II
-  Novel indicators
-  Researched existing pediatric measures
-  Focused on neonatal measures

**Notes:**
Given the uniqueness of the pediatric patient population, Pediatric (PDI) Module was developed.  During phase I applicable measures from the PQIs, IQIs & PSI were chosen & adapted to pediatric population
For Phase II we sought to develop a group of novel, pediatric specific, indicators.  After research on existing measures used by other groups & organizations the neonatal measures we found were felt, as a group, to have the best potential for quality measurement work & research

## Neonates: important & unique

- In CA (in 2000)
- 437,500 hospital births
- Over $1.5 billion spent on hospital births
- 2.1% of newborns weighed <2000g (highly 	probable NICU admissions)
- Over $730 million spent on infants weighing 	<2000g
- Medical concerns, risks of mortality & 	morbidity different in Neonates/NICU

- Schmitt, Sneed & Phibbs, “Costs of Newborn Care in California: A Population-Based Study”,
*Pediatrics*, 2006, 117; 154-160.

**Notes:**
Almost 2/3s of all childhood hospital stays are for newborns & neonates (up to 30 days old), though vast majority are for uncomplicated births
Even though NICU stays make up only a small part of the total number of neonatal hospitalizations, the costs they incur involve a much greater percentage of the total costs.
Remember, these NICU cost estimates are only for those infants <2000g.  If other issues such as congenital anomalies, respiratory problems, or infections are included the numbers of admissions and costs quickly climb, and obviously become much greater when looked at nationally.
Neonates, especially those that are critically ill, are a unique patient population, even within pediatrics (ask any pediatric resident working in an NICU).  The medical issues and risks they are faced with (in terms of survival, respiratory issues, brain development (and potential damage), nutrition, etc.) are unique to this group and thus the quality issues involved in their care are also very different

## Literature review to identify previously developed quality measures from various researchers and organizations (e.g., the California Perinatal Quality Care Collaborative, JCAHO, CHCA, and the National Perinatal Information Center) 
Grade III & IV intraventricular hemorrhage (IVH)
Retinopathy of prematurity (ROP)
Necrotizing enterocolitis (NEC)
Meconium aspiration syndromes (MAS)
Nosocomial blood stream infections (BSI)
Neonatal mortality

- Grade III & IV intraventricular hemorrhage (IVH)
- Retinopathy of prematurity (ROP)
- Necrotizing enterocolitis (NEC)
- Meconium aspiration syndromes (MAS)
- Nosocomial blood stream infections (BSI)
- Neonatal mortality

**AHRQ Neonatal QI development**

**Notes:**
We are using the same general research protocol for developing these indicators as in previous phases
Lit Review / QIs:
These are all measures used by organizations involved in healthcare quality (Child Health Corporation of America (CHCA), National Perinatal Information Center (NPIC), California Perinatal Quality of Care Collaborative (CPQCC),  
& The Joint Commission on Accreditation of Healthcare Organizations (JCAHO))

## ICD-9-CM coding review
To ensure correspondence between clinical concept and coding practice

Empirical analyses
To explore alternative definitions
To assess nationwide rates and hospital variation
To develop methods to account for differences in risk

Dealing with Bias

Exclude patients at risk for:
Complications present on admission
Non-preventable complications
Stratification – risk groupings

- To ensure correspondence between clinical concept and coding practice
**Empirical analyses**
- To explore alternative definitions
- To assess nationwide rates and hospital variation
- To develop methods to account for differences in risk
**Dealing with Bias**
- Exclude patients at risk for:
- Complications present on admission
- Non-preventable complications
- Stratification – risk groupings

**AHRQ Neonatal QI development**

**Notes:**
ICD-9:
Each indicator was adapted for use with administrative data using ICD-9 codes for diagnoses and procedures.  

Bias:
In addition to having an appropriate patient sample, e.g. limiting IVH to 500-1500g infants, the measures excluded patients at risk for. . . 
Non preventable complication: Excluded infants with birthweights < 500g as these have such extremely high rates of complications and mortality despite exceptional care
Stratification / risk groupings – rates stratified by birthweight groupings, or inborn vs. outborn status

## Clinical panel review

- Intended to establish consensual validity
- Modified RAND/UCLA Appropriateness Method (“nominal group” or “modified Delphi”)
- Panel included 3 neonatologists, 1 neonatal nurse, 2 perinatologists, 1 family physician nominated by national provider organizations
- All panelists rated all assigned indicators on:
- Overall usefulness for quality improvement
- Overall usefulness for public reporting
- Likelihood of being preventable
- Likelihood of being due to medical error or negligence
- Likelihood of being clearly charted in medical records
- Extent to which indicator is subject to case mix bias

## Clinical panel process 	

- Pre-conference ratings; comments and suggestions solicited
- Individual ratings returned to panelists with distribution of ratings and other panelists’ comments/suggestions
- 2-hour conference call moderated by non-clinician and attended by note-taker, focusing on high-variability items and panelists’ suggestions
- Suggestions adopted only by consensus
- Post-conference ratings; comments and suggestions solicited

## AHRQ Neonatal QIs – Brief Summary

**Notes:**
The following measures were the result of this development & were presented to the panel for final evaluation. . .
Some highlights from the table. . .
Birthweight limits: As mentioned before all indicators exclude less than 500g infants.  IVH, ROP and NEC focus on the 500-1499g infants (as this the population at greatest risk for these complications, and is consistent with the existing measures for these diagnoses).   Nosocomial BSIs include 500-1499g, and those >=1500g if they experienced death, a major surgery, mechanical ventilation or transfer in or out to another acute care hospital.
Inclusions: IVH & MAS includes inborns only as potentially preventative interventions are mostly pre- or ante-natal.  The rest of the measures include transfers within the first 2 days of life.
Exclusions: IVH: <1 week transfers exclusion due to risk of no head U/S being done to identify IVH,  and Neonatal mortality excludes transfers to another hospital, and extremely high mortality risk diagnoses.
Of note, in the first round of ratings the expert panelists were asked to choose between two versions of a neonatal mortality measure – one that closely approximately an available JCAHO measure (not shown) and the one shown here.  This modified version (shown) – which had stricter inclusion and exclusion criteria, was preferred.
Also, panelists were informed that a risk adjustment method for these measures was being developed.

## AHRQ Neonatal QI Rates: 2003 KID

- Events per 1000 population at risk

- Kids’ Inpatient Database 2003. AHRQ Healthcare Cost and Utilization Project.

**Notes:**
Overall, observed rates from the 2003 KID for the 6 potential measures.  

## AHRQ Neonatal QI Rates: 2003 KID

- Events per 1000 population at risk – with birthweight groupings for <1500g

- Kids’ Inpatient Database 2003. AHRQ Healthcare Cost and Utilization Project.

**Notes:**
As expected, the highest rates are in the <1000g infants and we have declining rates as the infants get larger. (Rates are comparable to those reported in the literature)

## AHRQ Neonatal QIs: Ratings Process

- Endorsement requirements - Median score of ≥7 (on 1-9 scale),
- without significant disagreement, on either of two questions:
-  Useful for quality improvement?
-  Useful for comparative reporting?

| Measure | Quality Improvement? | Comparative Reporting? |
| --- | --- | --- |
| IVH | 7* | 6.5 |
| ROP | 4 | 3 |
| NEC | 6 | 6 |
| MAS | 3 | 3 |
| Nosocomial BSI | 8 | 8 |
| Neonatal Mortality | 6 | 7 |

- * Significant disagreement on ratings amongst panelists

**Notes:**
Only Nosocomial BSI (based on the CPQCC measure) and the stricter version of Neonatal Mortality were endorsed by the expert panel.
IVH, ROP, NEC, & MAS were not endorsed

## Unendorsed Neonatal QIs: Concerns

- Uncertain preventability of the outcome
- Lack of specificity of existing ICD-9 codes
- “Necrotizing enterocolitis” (777.5)
- “Retrolental fibroplasia” (362.21)
- Limitations of administrative data (lack of detailed clinical information linked to uncertain diagnosis)

**Notes:**
Concerns: 
Preventability: Panelists were concerned about the degree of preventability of certain indicators.  Even though there is good evidence that certain interventions can prevent or attenuate these complications – e.g. appropriate use of ante-natal steroids decrease the rate & severity of IVH in preterm infants – it was felt that even with excellent, appropriate care many infants still would develop these complications. 
Lack of specificity of ICD-9 codes:  While there were codes for all of the complications used in the measures, in some, e.g. NEC, there is currently only one code, & there is no differentiation for grades of severity.  This would make quality assessments very difficult for these measures.

## Risk Adjustment

- Risk adjustment under development
- Testing models based on:
- Gender
- Birthweight
- Singleton vs. multiple
- Congenital abnormalities
- Grouped by risk

**Notes:**
Borrowing from work by Ciaran Phibbs and colleagues. Adapted by the CPQCC.

Congential abnormalities are grouped by risk determined emprically. 

## Conclusions & Implications

- AHRQ measures for nosocomial BSI and neonatal mortality are forthcoming
- Potential to help prioritize quality improvement efforts for neonates
- Non-endorsed measures have potential as research tools to identify and investigate “best practices”
- Coding changes are needed to improve the acceptability of other potential indicators

**Notes:**
Given their use of relatively easily accessible administrative data, the endorsed measures have the potential to help prioritize quality improvement efforts for neonates at both local and national levels.

## Future Work

- Further development on risk adjustment
- Official release anticipated Winter 2008
- Validation work using partnerships with providers
- Submission for NQF endorsement
- Propose coding changes

## Acknowledgments

- Data used: KID 2003. Healthcare Cost and Utilization Project (HCUP), Agency for Healthcare Research and Quality.
- We gratefully acknowledge the data organizations in participating states that contributed data to HCUP that we used in this study: the Arizona Department of Health Services; California Office of Statewide Health and Development; Colorado Health and Hospital Association; CHIME, Inc. (Connecticut); Florida Agency for Health Care Administration; Georgia Hospital Association; Hawaii Health Information Corporation; Illinois Health Care Cost Containment Council; Iowa Hospital Association; Kansas Hospital Association; Maryland Health Services Cost Review Commission; Massachusetts Division of Health Care Finance and Policy; Missouri Hospital Industry Data Institute; New Jersey Department of Health and Senior Services; New York State Department of Health; Oregon Association of Hospitals and Health Systems; Pennsylvania Health Care Cost Containment Council; South Carolina State Budget and Control Board; Tennessee Hospital Association; Utah Department of Health; Washington State Department of Health; and Wisconsin Department of Health and Family Services.