**MEASURES OF PEDIATRIC HEALTH CARE QUALITY BASED ON HOSPITAL
ADMINISTRATIVE DATA**

**THE PEDIATRIC QUALITY INDICATORS**

**TECHNICAL REPORT PREPARED BASED ON CONTRIBUTIONS BY:**

**KATHRYN MCDONALD, PRINCIPAL INVESTIGATOR**

**PATRICK ROMANO, CO-INVESTIGATOR**

**SHERYL DAVIES, PROJECT MANAGER/SENIOR RESEARCH ASSISTANT**

**CORINNA HABERLAND, PEDIATRIC HEALTH SERVICES RESEARCHER**

**JEFFREY GEPPERT, LEAD ANALYST**

**AMY KU, ANALYST**

**KAVITA CHOUDHRY, RESEARCH ASSISTANT**

**We gratefully acknowledge the assistance of our clinical panels, peer
reviewers, AHRQ project officers and our Support for Quality Indicators
colleagues.** **This work also builds on previous work by our team along
with many other contributors to development of the AHRQ Quality
Indicators. We thank our previous collaborators, whose names appear on
previous technical reports.**

February 20, 2006

Table of Contents

[Orientation to the Report 1](#orientation-to-the-report)

[Introduction 2](#introduction)

[Background 2](#background)

[Pediatric Quality Indicator Development
2](#pediatric-quality-indicator-development)

[Methods 5](#methods)

[Operationalization of indicators 9](#operationalization-of-indicators)

[Clinician Panel Review Methods 10](#clinician-panel-review-methods)

[Peer Review Methods 14](#peer-review-methods)

[Empirical Methods 15](#empirical-methods)

[Purpose of Analyses 15](#purpose-of-analyses)

[Analysis Approach 15](#analysis-approach)

[Results 18](#results)

[Summary of Results 18](#summary-of-results)

[Overall Results from Clinician Panel Review
20](#overall-results-from-clinician-panel-review)

[Overall Results from Peer Review 23](#overall-results-from-peer-review)

[Detailed Results by Indicator 25](#detailed-results-by-indicator)

[ACCIDENTAL PUNCTURE OR LACERATION (PSI)
26](#accidental-puncture-or-laceration-psi)

[DECUBITUS ULCER (PSI) 32](#decubitus-ulcer-psi)

[FOREIGN BODY LEFT IN DURING PROCEDURE (PSI)
38](#foreign-body-left-in-during-procedure-psi)

[IATROGENIC PNEUMOTHORAX (IN NEONATES AT RISK) (PSI)
42](#iatrogenic-pneumothorax-in-neonates-at-risk-psi)

[IATROGENIC PNEUMOTHORAX IN NON-NEONATES (PSI)
47](#iatrogenic-pneumothorax-in-non-neonates-psi)

[POSTOPERATIVE PHYSIOLOGIC AND METABOLIC DERANGEMENT (PSI)
52](#postoperative-physiologic-and-metabolic-derangement-psi)

[POSTOPERATIVE HEMORRHAGE AND HEMATOMA (PSI)
57](#postoperative-hemorrhage-and-hematoma-psi)

[POSTOPERATIVE RESPIRATORY FAILURE (PSI)
63](#postoperative-respiratory-failure-psi)

[POSTOPERATIVE SEPSIS (PSI) 69](#postoperative-sepsis-psi)

[POSTOPERATIVE WOUND DEHISCENCE (PSI)
77](#postoperative-wound-dehiscence-psi)

[SELECTED INFECTION DUE TO MEDICAL CARE (PSI)
83](#selected-infection-due-to-medical-care-psi)

[TRANSFUSION REACTION (PSI) 89](#transfusion-reaction-psi)

[ASTHMA ADMISSION RATE (PQI) 93](#asthma-admission-rate-pqi)

[DIABETES SHORT-TERM COMPLICATIONS ADMISSION RATE (PQI)
97](#diabetes-short-term-complications-admission-rate-pqi)

[DEHYDRATION ADMISSION RATE (PQI) 100](#dehydration-admission-rate-pqi)

[GASTROENTERITIS ADMISSION RATE (PQI)
103](#gastroenteritis-admission-rate-pqi)

[BACTERIAL PNEUMONIA ADMISSION RATE (PQI)
107](#bacterial-pneumonia-admission-rate-pqi)

[PERFORATED APPENDIX ADMISSION RATE (PQI)
111](#perforated-appendix-admission-rate-pqi)

[URINARY TRACT INFECTION ADMISSION RATE (PQI)
115](#urinary-tract-infection-admission-rate-pqi)

[CRANIOTOMY MORTALITY RATE (IQI) 118](#craniotomy-mortality-rate-iqi)

[PEDIATRIC HEART SURGERY MORTALITY RATE (IQI)
122](#pediatric-heart-surgery-mortality-rate-iqi)

[Conclusion 125](#conclusion)

[References 128](#references)

**Index of Appendices**

Appendix A. Specifics of Literature Search Methods

Appendix B. Clinician Review Panels

Appendix C. Example Panel Instructions and Questionnaires

Appendix D. Peer Reviewers

**Index of Tables**

[Table 1. Current AHRQ QIs not considered for inclusion in pediatric
patients 5](#__RefHeading___Toc128231352)

[Table 2. Organizations contacted for nominations of panelists and
potential indicators (for Phase II) 6](#__RefHeading___Toc128231353)

[Table 3. Multi-specialty Panel Composition
11](#__RefHeading___Toc128231354)

[Table 4. Criteria for Agreement Status
13](#__RefHeading___Toc128231355)

[Table 5. Definitions for Overall Appropriateness of Indicator for
Internal QI 14](#__RefHeading___Toc128231356)

[Table 6. Indicators recommended for inclusion in the Pediatric Quality
Indicator Set 19](#__RefHeading___Toc128231357)

[Table 7. Indicators no longer being considered for inclusion
20](#__RefHeading___Toc128231358)

#  {#section .unnumbered}

# Orientation to the Report

This report documents the work undertaken in Phase I of a two-phase
process to develop the Pediatric Quality Indicators as part of the
Agency for Healthcare Research and Quality (AHRQ) contract, "Support for
Quality Indicators II" under subcontract with Battelle Memorial
Institute by Stanford University and the University of California at
Davis. This work was initiated in response to a charge to develop
indicators of children's health care utilizing inpatient administrative
data. These indicators examine both the quality of inpatient care, as
well as the quality of outpatient care that can be inferred from
inpatient data, such as potentially preventable hospitalizations.

The report contains three main sections:

1\. The introduction section launches the actual technical report and
provides background regarding pediatric indicator development and the
current effort to develop an indicator set based on administrative data.

2\. The methods section outlines the approach used to gather evidence to
identify and evaluate potential patient safety indicators, including the
literature review, empirical analyses, and clinician panel review, as
well as the operationalization of indicators and evaluation of risk
adjustment approaches.

3\. The results section is divided into two parts. The first part
highlights general themes and summarizes the overall results. The second
part provides detailed results for each AHRQ QI examined.

Several appendixes provide additional detail regarding methods and
results.

# Introduction

## Background 

The demand for information on quality in healthcare has risen sharply
over the past several years. In response to this demand, the Healthcare
Cost and Utilization Project (HCUP) Quality Indicators (QIs) were
developed at the Agency for Healthcare Research and Quality (AHRQ) in
1994. The 33 initial indicators, based on inpatient hospitalization
data, were designed to highlight quality concerns and to target areas
for further analysis.

From 1998 to 2002, Stanford University and the University of California
(UC), under contract with AHRQ, reviewed the HCUP quality indicators and
recommended revised and new indicator sets. The indicators, named the
AHRQ Quality Indicators (AHRQ QIs), are divided into three indicator
sets: the Inpatient Quality Indicators, the Prevention Quality
Indicators and the Patient Safety Indicators.

The Inpatient Quality Indicators (IQI) and Prevention Quality Indicators
(PQI) were developed together, and include all mortality indicators and
potentially preventable admission indicators. Much of the scientific
evidence for these indicators is based on reports in the peer reviewed
literature. Structured literature review and empirical analyses were
used to establish the validity of these indicator sets. Details
regarding the development process are presented in the publication
"Refinement of the HCUP Quality Indicators" available at
[www.qualityindicators.ahrq.gov](http://www.qualityindicators.ahrq.gov/).

The Patient Safety Indicators (PSIs) provide rates for potentially
preventable complications of care. Building from a base of indicators
reported in the literature (e.g., the Complications Screening Program
developed by Lisa Iezzoni and colleagues), indicators developed
internally at AHRQ, and a detailed review of the ICD-9-CM code book, the
Stanford-UC project team aimed to identify a set of patient safety
related indicators. Given the relative lack of literature outlining or
validating such indicators, a structured clinical review process was
developed and conducted to refine indicator definitions and establish
face validity. Clinical panelists were nominated by professional
organizations for the review, and consisted of generalist, specialist,
and subspecialist physicians, nurses, and midwives. A few indicators
required additional input from surgical subspecialties, and as a result
underwent a second review. Details regarding the development of the PSIs
are presented in the publication, "Measures of Patient Safety Based on
Administrative Data: The Patient Safety Indicators" available at
[www.qualityindicators.ahrq.gov](http://www.qualityindicators.ahrq.gov/).

## Pediatric Quality Indicator Development

In 2000, children accounted for 18 percent or 6.3 million of the
hospitalizations in the U.S. The vast majority of these stays were for
newborn infants, with children and adolescents (one to 17 years old)
accounting for 1.8 million of the hospital stays (5%).^1^

There are few measure sets exclusively designed to measure quality of
care for children, and none for hospital care.^2^ Recently, AHRQ has
responded to the need for research on potential indicators of pediatric
hospital quality by commissioning this project to focus on children's
health care quality using routinely collected hospital discharge data as
the basis for indicator specification.

Development of quality indicators for the pediatric population involves
many of the same challenges associated with the development of quality
indicators for the adult population. These challenges include the need
to carefully define indicators using administrative data, establish
validity and reliability, detect bias and design appropriate risk
adjustment, and overcome challenges of implementation and use. However,
the special population of children invokes additional, special
challenges.

A draft briefing paper, presented at the recent National Quality Forum
(NQF) meeting on pediatric quality indicators^3^, outlined these
challenges as the four 'Ds':

*Differential epidemiology of child healthcare relative to adult
healthcare* **--** In general, children are a relatively healthy
population. Except for a subpopulation of children with special
healthcare needs, children seldom have multiple concurrent illnesses and
have relatively few encounters with the healthcare system. Many
encounters children have are for preventive care in an outpatient
setting, and most children are rarely hospitalized. Therefore, some may
suggest that hospital-based indicators are of limited importance to
measuring the overall quality of children's healthcare. Advocates of
this view may prefer population-based measures of outpatient care,
focusing on the appropriate delivery of preventive care (e.g.,
immunizations) or outpatient care for chronic diseases (e.g., asthma) or
common childhood illnesses (e.g., viral respiratory infections).
However, as user requests to the AHRQ QI support service illustrate,
hospitals that care for children still need measures for quality
improvement purposes, just as others may need such measures for consumer
education and informed purchasing.

*Dependency* **--** A second challenge in children's healthcare is their
dependency on parents and other adults for financing, accessing,
receiving, and evaluating their care. Many aspects of healthcare from
clinical decision-making to patient instructions to actual care delivery
depend on involvement by an adult caregiver, as well as the patient
(i.e. the child). Evaluating care may further depend on children's
caregivers to submit accurate information in a timely manner.

*Demographics* **--** Children are a diverse group, ranging from
premature neonates to adolescents. Children are more likely to live in
poverty than adults (resulting in higher dependence on Medicaid), and
are more likely than persons in any other age group to belong to a
racial or ethnic minority group. Adolescents, along with young adults,
are less likely to have health insurance than older adults. Delivering
healthcare and evaluating the quality of that care is especially
challenging for such a diverse and hard-to-reach population.

*Development* **--** Children are in a constant state of physical,
emotional and cognitive development. A child's physical and mental
health depends on the success of all of these developmental processes.
Thus, quality indicators appropriate for one age group may be
inappropriate for another. Different measures may be needed for each age
group (e.g., neonates, young children, older children, adolescents).

Since these four factors can pervade all aspects of children's
healthcare, simply applying adult indicators to younger age ranges is
insufficient. For example, many quality indicators dealing with common
chronic diseases in adults simply do not apply to children, whereas
other indicators derived from the adult setting require careful
consideration of their validity due to different causative factors in
the pediatric population. Others require modified definitions due to
different coding practices for children. Therefore, the development of
the Ped QI module requires careful consideration of each of these
factors.

# Methods 

**General Approach to Pediatric Indicator Development**

The development of the AHRQ Pediatric Quality Indicators utilizes a four
pronged approach: identification of candidate indicators, literature
review, empirical analyses, and panel review. Candidate indicators were
identified through both published literature and a brief survey of
national organizations. Literature review provided descriptions and
evaluations of some candidate indicators and the underlying relationship
to quality of care. Empirical analyses were conducted to explore
alternative definitions; to assess nationwide rates and hospital
variation; and to develop appropriate methods to account for variation
in risk. Clinical panel review helped to refine indicator definitions
and risk groupings, and to establish face validity in light of the
limited evidence from the literature for most pediatric indicators.
Information from these sources was used to specify indicator definitions
and make recommendations to AHRQ regarding the best indicators for
inclusion in the pediatric indicator set.

*Phase I versus Phase II*

The development of the Ped QI module is expected to occur in two phases.
Phase I, documented in this report, evaluated current AHRQ QIs and their
potential adaptation to the pediatric population. Phase II of the
pediatric indicator development will examine novel indicators (i.e., not
part of the current AHRQ QI set) and any AHRQ QIs that require extensive
re-definition and clinical input.

*Identification of potential indicators*

Current AHRQ Quality Indicators were reviewed for applicability to the
pediatric population, including both hospital-level indicators of
inpatient care and area-level indicators of access to quality outpatient
care (utilizing inpatient admission data). Not all current indicators
were considered for inclusion in the pediatric indicator set. Indicators
that address chronic or acute diseases that primarily affect the adult
population (e.g. COPD, CHF, AMI), or are clinically different in
children (e.g., hip fracture), were eliminated. A few other indicators
were eliminated due to early concerns about validity in the pediatric
population, based on users' chart reviews and validation projects (e.g.
Complications of Anesthesia, Failure to Rescue, Death in Low Mortality
DRGs).^4^ See Table 1 for a list of indicators that were not considered
for adaptation to the pediatric population.

[]{#__RefHeading___Toc128231352 .anchor}Table 1. Current AHRQ QIs not
considered for inclusion in pediatric patients

+---------------------+------------------------------------------------+
| **Reason for        | **Indicator**                                  |
| exclusion**         |                                                |
+---------------------+------------------------------------------------+
| Primarily adult     | **Inpatient Quality Indicators:**              |
| diseases            |                                                |
|                     | -   Acute myocardial infarction (AMI)          |
|                     |     mortality                                  |
|                     |                                                |
|                     | -   Congestive heart failure (CHF) mortality   |
|                     |                                                |
|                     | -   Stroke mortality                           |
|                     |                                                |
|                     | -   Gastrointestinal (GI) hemorrhage mortality |
|                     |                                                |
|                     | -   Hip fracture mortality                     |
|                     |                                                |
|                     | -   Hip replacement mortality                  |
|                     |                                                |
|                     | -   Abdominal aortic aneurysm (AAA) repair     |
|                     |     mortality and volume                       |
|                     |                                                |
|                     | -   Carotid endarterectomy mortality and       |
|                     |     volume                                     |
|                     |                                                |
|                     | -   Esophageal resection mortality and volume  |
|                     |                                                |
|                     | -   Pancreatic resection mortality and volume  |
|                     |                                                |
|                     | -   Coronary artery bypass graft (CABG) volume |
|                     |                                                |
|                     | -   Percutaneous coronary angioplasty (PTCA)   |
|                     |     volume                                     |
|                     |                                                |
|                     | -   Bilateral catheterization                  |
|                     |                                                |
|                     | -   Incidental appendectomy in the elderly     |
|                     |                                                |
|                     | -   Laparoscopic cholecystectomy rate          |
|                     |                                                |
|                     | -   CABG area rate                             |
|                     |                                                |
|                     | -   Hysterectomy area rate                     |
|                     |                                                |
|                     | -   Laminectomy area rate                      |
|                     |                                                |
|                     | -   PTCA area rate                             |
|                     |                                                |
|                     | **Patient Safety Indicators:**                 |
|                     |                                                |
|                     | -   Post-operative hip fracture                |
|                     |                                                |
|                     | **Prevention Quality Indicators**              |
|                     |                                                |
|                     | -   Long term diabetes complications area rate |
|                     |                                                |
|                     | -   Chronic obstructive pulmonary disease      |
|                     |     (COPD) area rate                           |
|                     |                                                |
|                     | -   Angina area rate                           |
|                     |                                                |
|                     | -   CHF area rate                              |
|                     |                                                |
|                     | -   Lower extremity amputation among diabetics |
|                     |     area rate                                  |
+---------------------+------------------------------------------------+
| Rare and often      | -   Pneumonia mortality                        |
| occurs in           |                                                |
| clinically complex  |                                                |
| patients or         |                                                |
| patients in end     |                                                |
| stage disease       |                                                |
+---------------------+------------------------------------------------+
| Chart review from   | -   Failure to rescue                          |
| pediatric           |                                                |
| institutions raised | -   Death in low mortality Diagnostic Related  |
| validity concerns   |     Groups (DRGs)                              |
| for pediatrics      |                                                |
|                     | -   Complications of anesthesia                |
|                     |                                                |
|                     | -   Post-operative pulmonary embolism or deep  |
|                     |     vein thrombosis                            |
+---------------------+------------------------------------------------+
| Obstetric           | -   Obstetric trauma                           |
| indicators --       |                                                |
| clinical issues are | -   Cesarean delivery rate                     |
| similar for teen    |                                                |
| and adult mothers   | -   Vaginal birth after cesarean (VBAC)        |
|                     |     delivery rate                              |
|                     |                                                |
|                     | -   Birth trauma (an indicator of obstetric    |
|                     |     care)                                      |
|                     |                                                |
|                     | -   Low birth weight (an indicator of          |
|                     |     obstetric care)                            |
+---------------------+------------------------------------------------+

[]{#__RefHeading___Toc128231353 .anchor}Table 2. Organizations contacted
for nominations of panelists and potential indicators (for Phase II)

  -----------------------------------------------------------------------
  **Organizations Contacted for General Input and Potential Indicators**

  California Perinatal Quality Care Collaborative

  Center for Research for Mothers and Children

  Centers for Disease Control and Prevention

  Centers for Medicare and Medicaid Services

  Centers for Medicare and Medicaid Services

  Children\'s Medical Center

  Joint Commission on the Accreditation of Healthcare Organizations

  Leapfrog Group

  National Center for Chronic Disease Prevention and Health Promotion

  National Center on Birth Defects and Developmental Disabilities

  National Initiative for Children\'s Healthcare Quality

  National Institute of Child Health and Human Development

  National Institute of Mental Health

  National Institute on Alcohol Abuse and Alcoholism

  National Institute on Drug Abuse

  National Patient Safety Foundation

  National Quality Forum

  Parents of Infants and Children with Kernicterus

  Substance Abuse and Mental Health Services Administration

  Texas Children\'s Hospital

  The Child and Adolescent Health Measurement Initiative

  United States Pharmacopeia

  Vermont Oxford Network

  Zero to Three: National Center for Infants, Toddlers and Families
  -----------------------------------------------------------------------

  -----------------------------------------------------------------------
  **Organizations contacted for Potential Indicators and Panelist
  Nominations**

  Ambulatory Pediatric Association

  American Academy of Allergy Asthma and Immunology

  American Academy of Child and Adolescent Psychiatry

  American Academy of Family Physicians

  American Academy of Neurology

  American Association for Pediatric Ophthalmology and Strabismus

  American Association for the Surgery of Trauma

  American Association of Neurological Surgeons

  American Association of Pediatrics

  AAP, Section on Adolescent Health

  AAP, Section on Allergy and Immunology

  AAP, Section on Anesthesiology and Pain Medicine

  AAP, Section on Cardiology and Cardiac Surgery

  AAP, Section on Critical Care

  AAP, Section on Emergency Medicine

  AAP, Section on Endocrinology

  AAP, Section on Gastroenterology and Nutrition

  AAP, Section on Hematology/Oncology

  AAP, Section on Hospital Care

  AAP, Section on Infectious Disease

  AAP, Section on Nephrology

  AAP, Section on Neurological Surgery

  AAP, Section on Neurology

  AAP, Section on Ophthalmology

  AAP, Section on Orthopaedics

  AAP, Section on Otolaryngology/ Head and Neck Surgery

  AAP, Section on Pediatric Pulmonology

  AAP, Section on Perinatal Pediatrics

  AAP, Section on Radiology

  AAP, Section on Surgery

  AAP, Section on Urology

  American College of Cardiology

  American College of Chest Physicians

  American Hospital Association

  American Pediatric Society/Society for Pediatric Research

  American Pediatric Surgical Association

  American Pediatric Surgical Nurses Association

  American Society of Clinical Oncology

  American Society of Nephrology

  American Society of Pediatric Hematology/Oncology

  American Society of Pediatric Nephrology

  American Society of Pediatric Neurosurgeons

  American Society of Pediatric Otolaryngology

  American Thoracic Society

  California Association of Neonatologists

  Children and Adults with Attention-Deficit/Hyperactivity Disorder
  (CHADD)

  Child Health Corporation of America

  Child Neurology Society

  Congenital Heart Surgeons\' Society

  Lawson Wilkins Pediatric Endocrine Society

  National Association of Children's Hospitals and Related Institutions
  and National Association of Children's Hospitals, the policy affiliate

  National Association of Neonatal Nurses

  National Association of Pediatric Nurse Practitioners

  North American Society for Pediatric Gastroenterology, Hepatology and
  Nutrition

  Pediatric Emergency Medicine Interest Group, Society for Academic
  Emergency Medicine

  Pediatric Infectious Diseases Society

  Society for Adolescent Medicine

  Society for Maternal-Fetal Medicine

  Society for Pediatric Dermatology

  Society for Pediatric Radiology

  Society for Pediatric Urology

  Society of Clinical Child and Adolescent Psychology

  Society of Critical Care Medicine, Section on Pediatrics

  Society of Pediatric Anesthesia

  Society of Pediatric Nurses

  Society of Thoracic Surgeons
  -----------------------------------------------------------------------

Literature review

Literature review provided evidence for potential indicators. The
results of the literature were presented to panel members to help inform
their ratings. Literature review involved searching for pertinent
articles on both PubMed and the *Pediatrics* web site. Searches were
done using keywords contained in or synonymous with the title of each
quality indicator. References to applicable journal articles in
bibliographies of retrieved articles were also reviewed. Articles that
provided any specific evidence, either confirming or arguing against
indicator use, were reviewed. Examples of applicable evidence included
articles utilizing the indicator or similar indicators, articles
describing the concept of the indicator as a measure of quality of care,
and articles evaluating the sensitivity and specificity of the
indicators and/or codes utilized by the indicators. For the most part
indicators that had evidence not supporting their use were not
considered for use in the indicator set. See Appendix A for literature
review search terms and limits.

## Operationalization of indicators

Applicable current AHRQ QIs were reviewed by two pediatrician health
services researchers before panel review, and potential modifications
were discussed and implemented (by consensus) in some cases. Empirical
analyses of specific codes and alternative indicator definitions further
informed draft indicator definitions. All analyses were performed using
the 2003 KIDs' Inpatient Sample(NIS) from the Healthcare Cost and
Utilization Project (HCUP), Agency for Healthcare Research and Quality.
For example, when diagnoses codes for patients with transfusion reaction
were analyzed it was determined that this event is often miscoded in the
neonatal population. The defining of indicators outside of the current
AHRQ QI set (i.e., "novel" indicators) is ongoing, and will be completed
as part of Phase II. When possible, definitions begin with an
established operationalized definition, and then adaptations are
incorporated based on application to a pediatric population, adaptation
of the indicator for administrative data or changes in clinical
practice.

A structured review of each indicator was undertaken to evaluate face
validity (from a clinical perspective). This process mirrored that
undertaken during the initial development of the Patient Safety
Indicators. Specifically, the panel approach established *consensual
validity,* which "extends face validity from one expert to a panel of
experts who examine and rate the appropriateness of each item...."^5^
The methodology for the structured review was adapted from the RAND/UCLA
Appropriateness Method^6^ and consisted of an initial independent
assessment of each indicator by clinician panelists using an initial
questionnaire, a conference call among all panelists, followed by a
final independent assessment by clinician panelists using the same
questionnaire. The panel process served to refine definitions of some
indicators, add new measures, and dismiss indicators with major concerns
from further consideration.

A similar standardized panel approach was previously used to evaluate
potential indicators of primary care quality^7,\ 8^ as well as
ambulatory care sensitive conditions.^9^

## Clinician Panel Review Methods

*Panel selection*

Forty-four distinct professional clinical organizations and hospital
associations were invited to submit nominations. These organizations
were selected based on the applicability of the specialty or
subspecialty to the candidate quality indicators. Nineteen organizations
submitted nominations: Ambulatory Pediatric Association, American
Academy of Allergy Asthma and Immunology, American Academy of Family
Physicians, American Academy of Pediatrics, American College of Chest
Physicians, American College of Nurse-Midwives, American Society of
Pediatric Hematology/Oncology, American Society of Pediatric Nephrology,
California Academy of Family Physicians, Child Health Corporation of
America, National Association of Children\'s Hospitals and Related
Institutions, National Association of Pediatric Nurse Practitioners,
Pediatric Infectious Diseases Society, Society for Academic Emergency
Medicine, Society for Adolescent Medicine, Society for Pediatric
Anesthesia, Society of Critical Care Medicine, Society of Pediatric
Nurses, and Society of Thoracic Surgeons.

These professional organizations nominated a total of 125 clinicians.
All nominees were invited to participate, if eligible, in the evaluation
of indicators available in Phase I and Phase II. In order to be eligible
to participate, nominees were required to spend at least 30% of their
work time on patient care, including hospitalized patients. From the 70
nominees accepting the invitation; five clinicians were ineligible to
participate. Nominees were asked to provide information regarding their
practice characteristics, including specialty, subspecialty, and setting
(i.e., urban vs. rural location, region of country, and service to
underserved populations), primary hospital of practice (i.e., funding
source), and involvement in education (i.e., clinical training, academic
affiliation).

To ensure appropriate clinical expertise on each panel, we identified
the specialties that would be required to properly evaluate the
indicators assigned to that panel. Panelists were selected so that each
panel had diverse membership in terms of practice characteristics and
setting. Thus, when a specific geographic area or type of clinician
(e.g. academic) was over-represented by the pool of eligible nominees,
randomly drawn members from that specific sub-group were contacted first
to fill the panels. In addition, conference call scheduling logistics
influenced assignments. From the 65 eligible nominees, 45 individuals
accepted our invitation to participate on a specific panel.

*Panel composition*

Four panels were formed to evaluate indicators grouped as follows:
Medical and surgical indicators, surgical only indicators, neonatal
indicators and prevention indicators. Participants in the panels are
listed in Appendix B. All panels had diversity in the geographic
location of panelists, and their type of practice (see Table 3).

[]{#__RefHeading___Toc128231354 .anchor}Table 3. Multi-specialty Panel
Composition

  -------------------------------------------- --------------------------
  Characteristic                               \% (N)

  Gender                                       

  Female                                       33% (15)

  Academic Affiliation                         

  Yes                                          91% (41)

  No                                           9% (4)

  Not reported                                 0% (0)

  Geographic Region                            

  East                                         29% (13)

  West                                         20% (9)

  South                                        27% (12)

  Midwest                                      24% (11)

  Community                                    

  Urban                                        71% (32)

  Suburban                                     36% (16)

  Rural                                        29% (32)

  Not reported                                 13% (6)

  Funding of Primary Hospital                  

  Private                                      51% (23)

  Public                                       13% (6)

  Both                                         18% (8)

  Not Reported                                 18% (8)

  Part of Patient Population Considered        
  Underserved                                  

  Yes                                          80% (36)

  No                                           7% (3)

  Not reported                                 16% (7)
  -------------------------------------------- --------------------------

1Clinical and/or research affiliation

*Initial evaluation*

After agreeing to evaluate each indicator presented in Phase I and Phase
II, panelists were sent information (see Appendix C) regarding
administrative data, ICD-9-CM coding, assignment of Diagnostic Related
Groups (DRGs) and Major Diagnostic Categories (MDCs), and specific
definitions for "adverse events or complications," "preventability," and
"medical error." Panelists were presented with six to seven indicators
(except the neonatal panel, which only reviewed two indicators) in the
Phase I review. The standardized text used to describe each ICD-9-CM
code was presented along with the specific numeric code. Exclusion and
inclusion criteria were also given, as well as the clinical rationale
for the indicator and the specification criteria. A summary of
literature-based evidence and empirical rates based on the 2000 NIS were
provided for reference. Finally, panelists were provided potential
questions regarding the indicator definition that the study team planned
to explore during the conference call.

Each of the 8 to 13 panelists from a given panel provided input for a
given indicator by completing a 10-item questionnaire (see Appendix C
for the two versions used: *hospital-based* for complications and
mortality indicators, and *prevention* for ambulatory care sensitive
area level indicators). The hospital-based indicator questionnaire asked
panelists to consider the ability of this indicator to screen out
conditions present on admission, to identify conditions with high
potential for preventability, and to identify medical errors. The
prevention indicator questionnaire asked panelists to evaluate the
ability of this indicator to assess access to high quality outpatient
care. In addition, both versions of the questionnaire asked panelists to
consider potential sources of bias, reporting or charting problems,
potential ways of gaming the indicator, and possible adverse effects of
implementing the indicator. Finally, panelists were invited to suggest
changes to the indicator.

*Conference call*

Following the submission of the initial evaluation questionnaires, all
panelists participated in a 90-minute conference call for their panel to
discuss the indicators. The purpose of each conference call was to allow
panelists to discuss their opinions regarding each indicator. Following
the instructions in the RAND/UCLA method where the primary goal of
interaction among panelists is to allow room for varied opinions about
the appropriateness of an indicator, panelists were explicitly told that
consensus was not the goal of discussion. In cases when panelists agreed
on proposed changes to the indicator definitions, such consensus was
noted and the definition was modified accordingly before the final round
of rating. Each call was moderated by a team member (KM), who directed
the structure of the call, and ensured that all panelists had a chance
to share their opinions. Also present was a technical expert, who
answered questions regarding administrative data and coding (PR), and
silent observers, who maintained comprehensive notes of the call (SD,
CH, KC, AK, JG). All team members refrained from offering opinions
regarding indicators during the call. Agenda items were set based on the
feedback received from the initial evaluation and in general focused on
points of disagreement among panelists. Panelists were prompted
throughout the process to consider the appropriate population at risk
for each indicator (specifically inclusion and exclusion criteria) in
addition to the complication or condition of interest. However, if
panelists wished to discuss other aspects of the indicator, this
discussion was allowed within the time allotted for that indicator. The
calls were recorded and transcribed for purposes of summarizing themes
and determining definitional changes.

*Final evaluation*

Following each conference call, changes to each indicator were made
where suggested by panelists. In each case, every panelist present on
the call must have either endorsed the proposed change or indicated
neutrality for the change to be implemented. The indicators were then
redistributed to panelists along with questionnaires used in the initial
evaluation. Each indicator description included explication of any
definitional changes that were adopted and the reason. Panelists were
asked to re-rate each indicator based on their current opinion. They
were asked to keep in mind the discussion during the conference call.
Four indicators were not re-distributed due to ongoing extensive
revisions. These indicators underwent a second round of review by the
same panel, following revisions.

*Tabulation of results*

To examine the results of the panels, we applied a modified version of
the "appropriateness" criteria outlined in the RAND/UCLA Appropriateness
Method. Results from the final evaluation questionnaire were used to
calculate median scores from the 9 point scale for each question and to
categorize the degree of agreement among panelists (see Table 4). Median
scores determined the level of acceptability of the indicator, and
dispersion of ratings across the panel for each applicable question
determined the agreement status. Therefore the median and agreement
status were independent measurements for each question. The following
six criteria covered in the questionnaire were used to summarize the
panel's opinions (i.e., median, agreement status category) on the
following aspects of each indicator:

> 1\. Overall usefulness of the indicator, both for internal quality
> improvement purposes and comparisons between hospitals,
>
> 2\. Likelihood that indicator measures a complication and not a
> comorbidity (specifically, present on admission),
>
> 3\. Preventability of complication,
>
> 4\. Extent to which complication is due to medical error,
>
> 5\. Likelihood that complication is charted given that it occurs, and
>
> 6\. Extent that indicator is subject to bias (systematic differences,
> such as case mix that could affect the indicator, in a way not related
> to quality of care).

For area based indicators panelists provided feedback on the following
aspects:

> 1\. Overall usefulness of the indicator, both internally within an
> area and for comparisons between areas
>
> 2\. Extent to which event reflects poor access to quality outpatient
> care
>
> 3\. Consistency in terminology for charting principal diagnosis
>
> 4\. Extent that indicator is subject to bias

These evaluations are included in the summary of results for each
indicator.

[]{#__RefHeading___Toc128231355 .anchor}Table 4. Criteria for Agreement
Status

  --------------- -------------- ------------------------------------------
  **Category**    **Panel size** **Criteria**

  Agreement       8-13 panelists Two or fewer members rated indicator
                                 outside specific three-point range (1-3.9,
                                 4-6.9, 7-9) in which the median falls.

                  5-7 panelists  One or fewer panelists rated indicator
                                 outside specific three-point range (1-3.9,
                                 4-6.9, 7-9) in which the median falls.

  Disagreement    8-13 panelists Three or more panelists rated indicator in
                                 each of the extreme three-point ranges
                                 (1-3.9, 7-9), demonstrating a split in
                                 opinion.

                  5-7 panelists  Two or more panelists rated indicator in
                                 each of the extreme three point ranges
                                 (1-3.9, 7-9), demonstrating a split in
                                 opinion.

  Indeterminate   All panel      Any panel rating not qualifying as either
  Agreement       sizes          "agreement" or "disagreement" by above
                                 criteria.
  --------------- -------------- ------------------------------------------

We used the ratings regarding the overall appropriateness of the
indicator for internal quality improvement (i.e., criterion number 1
above based on question #8a on questionnaire in Appendix C) to assess
the overall usefulness as a screen for potential quality problems at the
hospital or area level (see Table 5). This score mirrored the criterion
used for selection of the PSIs during the initial development process.
The median score and agreement category for this usefulness question
were combined into modified RAND groupings. Akin to the RAND
"Appropriate" levels, we created two categories, "Acceptable" and
"Acceptable (-)." "Acceptable (-)" refers to indicators that were
considered acceptable because the median rating was 7 or higher (on a
1-7 scale), but there was at least one participant (or two, in the case
of larger panels) whose rating fell below this range. The RAND
"Uncertain" level was likewise divided into two sub-levles, "Unclear,"
and the slightly worse category, "Unclear (-)." The RAND "Inappropriate"
level was defined identically but named "Unacceptable." These
designations, along with some initial administrative data testing and
subsequent coding clarifications, were used to form recommendations
regarding inclusion in the pediatric indicator set.

[]{#__RefHeading___Toc128231356 .anchor}Table 5. Definitions for Overall
Appropriateness of Indicator for Internal QI

  ---------------- ------------------------------------------------------
  Acceptable       Median falls between 7 and 9 (inclusive of both),
                   agreement

  Acceptable (-):  Median falls between 7 and 9 (inclusive of both),
                   indeterminate agreement

  Unclear:         Median falls between 7 and 9 (inclusive of both),
                   disagreement, OR

                   Median falls between 5 and 7 (inclusive of neither),
                   agreement or indeterminate agreement

  Unclear (-):     Median between 4 and 5 (inclusive of both), agreement,
                   indeterminate agreement or disagreement, OR

                   Median falls between 1 and 3.9 with disagreement

  Unacceptable:    Median falls between 1 and 3.9, agreement or
                   indeterminate agreement
  ---------------- ------------------------------------------------------

## **Peer Review Methods**

We received 40 nominations from federal agencies, advocacy groups and
health care quality associations for peer reviewers. In addition, a few
physicians who were nominated for the clinician review panels agreed to
participate in the peer review process instead. Twenty-six of the peer
reviewers we invited have expressed an interest in participating and
were sent materials. Among the peer reviewers are clinicians, policy
advisors, professors, researchers, and managers in quality improvement.
Participants in the review process are listed in Appendix D.

#  

## 

## Empirical Methods

### Purpose of Analyses

Empirical analyses were conducted to provide the clinical panels and
peer review participants with additional information about the
indicators. These analyses were also used by the development team to
test the alternative specifications and the relative contribution of
indicator components in the numerator and denominator. The results are
included in the "detailed results by indicator" section. These analyses
were not intended to inform issues of precision, bias and construct
validity, which will be addressed separately.

### Analysis Approach

**Data Source**

The data source used in the empirical analyses was the 2003 Kids'
Inpatient Sample (KID). The KID contains all-payer data on hospital
inpatient stays from States participating in the Healthcare Cost and
Utilization Project (HCUP). The 2003 KID provides information on 3
million inpatient stays from about 3,400 hospitals. The KID sampling
frame included all pediatric discharges from community,
non-rehabilitation hospitals in the HCUP State Inpatient Databases (SID)
that could be matched to the corresponding American Hospital Association
(AHA) survey data (subject to state-specific restrictions). The KID
includes a sample of pediatric discharges from all hospitals in the
sampling frame. For the sample, the pediatric discharges were stratified
by uncomplicated in-hospital birth, complicated in-hospital birth, and
pediatric non-birth and a random sampling taken of 10 percent of
uncomplicated in-hospital births and 80 percent of other pediatric cases
from each frame hospital. To obtain national estimates, discharge
weights using the AHA universe as the standard based on six
characteristics contained in the AHA hospital files: geographic region,
control, location, teaching status, bed size and hospital type. In this
report, we used the discharge level weights and PROC SURVEYMEANS in SAS
(cite) to compute the weighted national rates and variances and
indicator denominators (i.e., the sum of the discharge weights). For
more information, see *Design of the Hcup Kids' Inpatient Database
(Kid), 2003*
(http://hcup-us.ahrq.gov/db/nation/kid/reports/KID_2003_Design_Edited_013006.pdf).

**Definition of Neonate and Newborns**

Several of the indicators require a definition of "neonate" and
"newborn" in the specification as inclusion or exclusion criteria or to
stratify the rate. For this report, we used the following definitions to
define these populations.

[]{.mark}

A "neonate" is any discharge record with an admission date during the
neonatal period (birth to 28 days). To determine the neonatal period, we
use the age in days (AGEDAY) data element. That is, a neonate is any
discharge record with AGEDAY\<=28. If that data element is missing, and
age in years (AGE) equals zero, then a neonate is any discharge record
with ANY one of five conditions: 1) MDC 15 (Newborns & Other Neonates
with Condition Originating in the Perinatal Period); or 2) DRG 385-391;
or 3) Admission Type of "newborn" (ATYPE=4); or 4) a diagnosis code of
V29.xx (Observation and evaluation of newborns for suspected condition
not found); or 5) a diagnosis code indicating a live birth (see below)..
The latter definition is slightly too broad, as it includes some
discharges occurring outside the neonatal period.

A "newborn" is a neonate discharge record originating from a live birth.
To identify a live birth, we use discharge records with EITHER 1) any
diagnosis code of V3x.0x (i.e. V3x codes -- Liveborn Infants accoring to
Type of Birth - with a "0" in the fourth digit) OR 2) an admission type
of "newborn" (ATYPE=4) and age in years equal to zero, excluding
discharges with any diagnosis code of V3x.1x or V3x.2x (i.e. V3x codes
with a "1" or "2" in the fourth digit). These latter codes indicate live
births that occurred outside the hospital or prior to admission.

A "normal newborn" is a newborn without significant complications. To
identify normal newborns we use the newborn definition above along with
DRG 391 (Normal Newborn).

**Population Denominators**

The area level indicators use a population denominator. Although
hospital zip code may be used, it is recommended that patient zip code
be used to calculate area level indicators. This reduces effects of
tertiary referral centers. Our intercensal population estimates come
from the U.S. Census Bureau. The default age categories reported by
Census for the pediatric population are 0 to 4, 5 to 9, 10 to 14 and 15
to 19. Several of the indicators require a more refined age definition
in the specification as inclusion or exclusion criteria or to stratify
the rate. In addition, we report age categories used by AHRQ in their
child health publications (see, for example *Care of Children and
Adolescents in U.S. Hospitals HCUP Fact Book No. 4* at
http://www.ahrq.gov/data/hcup/factbk4/factbk4.htm). To estimate these
more refined age categories, we allocated the population estimates
uniformly within the census five-year age categories. In other words, we
assumed no growth in population or cohort size within these five year
age categories. The specific calculation is described below.

  ----------------------- ----------------------- -----------------------
  PedQI Age Category      U.S. Census Age         Allocation
                          Category                

  0 to 28 days            0 to 4 years            (28/365) \* (1/5)

  29 to 60 days           0 to 4 years            (32/365) \* (1/5)

  61 to 90 days           0 to 4 years            (30/365) \* (1/5)

  91 to 365 days          0 to 4 years            (275/365) \* (1/5)

  1 to 2 years            0 to 4 years            (2/5)

  3 to 5 years            0 to 4 years            (2/5)

                          5 to 9 years            (1/5)

  6 to 12 years           5 to 9 years            (4/5)

                          10 to 14 years          (3/5)

  13 to years             10 to 14 years          (2/5)

                          15 to 19 years          (3/5)
  ----------------------- ----------------------- -----------------------

**Birth Weight Categories**

For exclusions based on birth weight, or to stratify rates based on
birth weight, we use the following ICD-9-CM diagnosis codes:

  ------------------ ----------------------------------------------------
  Birth weight       ICD-9-CM Diagnosis Codes
  category           

  \<500g             76401, 76411, 76421, 76491, 76501, 76511, V2131

  500-999g           76402, 76403, 76412, 76413, 76422, 76423, 76492,
                     76493, 76502, 76503, 76512, 76513, V2132

  1000-1499g         76404, 76405, 76414, 76415, 76424, 76425, 76494,
                     76495, 76504, 76505, 76514, 76515, V2133

  1500-1999g         76406, 76407, 76416, 76417, 76426, 76427, 76496,
                     76497, 76506, 76507, 76516, 76517, V2134

  2000-2500g         76408, 76418, 76428, 76498,76508, 76518, V2135
  ------------------ ----------------------------------------------------

**Hospital Type Categories**

The analyses report rates separately by hospital type (children's vs.
non-children's). The KID data contains the American Hospital Association
(AHA) identifier for a subset of hospitals (about 79% of hospitals and
73% of discharges). For those hospitals that we could link to the AHA
Annual Survey, we identified children's hospitals as those hospitals
with either 1) a service type of children's hospital (see below) or 2)
that answered "yes" to the question: do you restrict admissions
primarily to children?.

AHA Service Type

  ------------ ----------------------------------------------------------
  50           children\'s general

  51           children\'s hospital unit of an institution

  52           children\'s psychiatric

  53           children\'s tuberculosis and other respiratory disease

  55           children\'s eye, ear, nose and throat

  56           children\'s rehabilitation

  57           children\'s orthopedic

  58           children\'s chronic disease

  59           children\'s other specialty
  ------------ ----------------------------------------------------------

#  

# Results

## Summary of Results

All current AHRQ QIs were considered for adaptation to a pediatric
population. Four indicators were already designed specifically for use
in a pediatric population: Pediatric Heart Surgery Mortality/Volume
(IQI), Pediatric Asthma Admission Rate (PQI) and Pediatric
Gastroenteritis Admission Rate (PQI). In a preliminary review, most
Inpatient Quality Indicators were eliminated from further consideration
because the conditions in question are primarily adult conditions: These
include mortality (and volume for those starred) for AMI, CHF, Stroke,
GI hemorrhage, Hip fracture, Hip replacement, Esophageal resection\*,
Pancreatic resection\*, AAA repair\*, Carotid endarterectomy\*, PTCA\*
and CABG\*. Pneumonia mortality was eliminated since death from
pneumonia is rare among children and typically occurs only in the
setting of multiple or severe chronic diseases. Rates of Incidental
Appendectomy, Bilateral Cathterization (validated for an elderly
population), Laparoscopic cholecystectomy, CABG, PTCA, Laminectomy and
Hysterectomy were also excluded for lack of relevance to children.
Prevention Quality Indicators excluded for similar reasons included Long
term diabetes complications, COPD, Angina, CHF, and Lower extremity
amputation among diabetics. Among the PSIs, two indicators were
eliminated due to either the absence of the event in children
(Post-operative hip fracture), or because the event was felt to be
clinically different in children (Post-operative PE or DVT). Three other
indicators---Failure to rescue, Death in low mortality DRGs and
Complications of anesthesia---were eliminated due to serious validity
concerns when applied as defined in the pediatric population. This
evidence included chart reviews and information obtained through user
reports. Use of these indicators would require extensive redefinition
beyond the scope of this project. For Low Mortality DRG, the validity of
this indicator would need to be investigated further, since mortality is
so rare in children and DRGs are generally not specific for children.
Failure to rescue will require identification of complications in high
risk populations and testing to establish a link with outcomes in
patients with those complications and quality of care. Finally,
Obstetric trauma, Cesarean delivery rate, and VBAC delivery rate were
excluded because little evidence demonstrates a meaningful clinical
difference between adolescent and adult obstetric patients. Adolescents
could remain in the existing obstetric indicators without compromising
the value or integrity of those indicators. Panels reviewed a total of
23 current AHRQ QIs.

Tables 6 -- 7 summarize the recommendations for each indicator based on
our review of the evidence, including literature review, empirical
analyses and clinician panel review. Based on final definitions tailored
to the pediatric population, 19 indicators are recommended for inclusion
in the Ped QI module at this time. Fourteen of those indicators are
intended for use at the hospital level, while the other five are area
level indicators, intended to measure access to high quality outpatient
care. Finally, 3 indicators are not considered suitable for inclusion in
the Ped QI module. One indicator will be considered in Phase II.

[]{#__RefHeading___Toc128231357 .anchor}Table 6. Indicators recommended
for inclusion in the Pediatric Quality Indicator Set

  ----------------------- ------------------ --------------- -----------------------
  **Indicator name**      **Panel                            **Special notes**
                          recommendation**                   

                          **Internal QI      **Comparative   
                          purpose**          reporting       
                                             purpose**       

  ***Inpatient                                               
  Indicators***                                              

  Accidental puncture and Acceptable (-)     Not recommended 
  laceration                                                 

  Decubitus ulcer         Acceptable (-)     Acceptable (-)  

  Foreign body left in    Acceptable (+)     Acceptable (+)  
  after procedure                                            

  Iatrogenic pneumothorax Acceptable (+)     Acceptable (-)  Denominator for
  in neonates at risk                                        community hospitals
                                                             will be very low.

  Iatrogenic pneumothorax Acceptable (+)     Acceptable (-)  Barotrauma and
  in non-neonates                                            procedure related
                                                             pneumothoraxes captured
                                                             together.

  Pediatric heart surgery Acceptable (+)     Acceptable (+)  Ratings based on
  mortality                                                  preliminary ratings,
                                                             dependent on adequate
                                                             risk adjustment.

  Pediatric heart surgery Acceptable (+)     Acceptable (+)  Ratings based on
  volume                                                     preliminary ratings,
                                                             dependent on adequate
                                                             risk adjustment.

  Postoperative           Acceptable (+)     Acceptable (+)  Some acquired
  hemorrhage and hematoma                                    coagulapathies will
                                                             only be diagnosed in
                                                             patients who have
                                                             bleeding complications,
                                                             leading to
                                                             uncorrectable bias.

  Postoperative           Acceptable (+)     Acceptable (-)  
  respiratory failure                                        

  Postoperative sepsis    Acceptable (-)     Not recommended 

  Postoperative wound     Acceptable (+)     Acceptable (+)  
  dehiscence                                                 

  Selected infection due  Acceptable (-)     Not recommended 
  to medical care                                            

  Transfusion reaction    Acceptable (+)     Acceptable (-)  

  ***Area Level                                              
  Indicators***                                              

  Asthma admission rate   Acceptable (+)     Acceptable (-)  Socioeconomic Status
                                                             (SES) risk adjustment
                                                             recommended. Data does
                                                             not capture admission
                                                             to short stay or
                                                             extended emergency
                                                             department (ED) stays.

  Diabetes short term     Acceptable (+)     Not recommended SES risk adjustment
  complication admission                                     recommended. Initial
  rate                                                       diagnosis admissions
                                                             included for patients
                                                             age 6 and older.

  Gastroenteritis         Acceptable (-)     Not recommended SES risk adjustment
  admission rate                                             recommended. Data does
                                                             not capture admission
                                                             to short stay or
                                                             extended ED stays.

  Perforated appendix     Acceptable (-)     Acceptable (-)  SES risk adjustment
  admission rate                                             recommended.

  Urinary tract infection Acceptable (-)     Not recommended SES risk adjustment
  admission rate                                             recommended. Data does
                                                             not capture admission
                                                             to short stay or
                                                             extended ED stays.
                                                             Hospitals differ on
                                                             evaluation for chronic
                                                             urinary tract disorders
                                                             (exclusion criteria),
                                                             leading to bias.
  ----------------------- ------------------ --------------- -----------------------

[]{#__RefHeading___Toc128231358 .anchor}Table 7. Indicators no longer
being considered for inclusion

  ----------------------- -----------------------------------------------
  **Indicator name**      **Reason**

  Postoperative           Not recommended by panel and no strong evidence
  physiologic and         base for useful application to pediatric
  metabolic derangement   population.

  Dehydration admission   Admissions due to dehydration combined with
  rate                    gastroenteritis admission rate indicator. Other
                          causes not recommended by panel.

  Bacterial pneumonia     Not recommended by panel and no strong evidence
                          base for useful application to pediatric
                          population.

  Craniotomy mortality    Requires further specialized development to
                          define risk groups as suggested by clinical
                          panel. Will be considered as a new indicator in
                          Phase II.
  ----------------------- -----------------------------------------------

## Overall Results from Clinician Panel Review

Four clinician panels were convened to evaluate the face validity of the
AHRQ QIs adapted and applied solely to a pediatric population. This
section covers general themes highlighted by panelists during the course
of review, which are in many cases applicable to several or all
indicators. Results of the pediatric clinician panels specific to each
indicator are outlined in the section, "Detailed Results By Indicator".

Most panelists were enthusiastic about pediatric quality indicators and
articulated important considerations particularly pertinent to pediatric
application. Panelists expressed that such indicators, if used
appropriately, could improve the quality of patient care by providing an
initial screen for quality concerns. In addition, panelists suggested or
reinforced some overarching themes important to developing and using
pediatric quality indicators.

*Importance of assessing health care quality in high risk groups*

For several indicators (e.g. Postoperative sepsis, Decubitus ulcer)
panelists noted that the indicators are of minimal value when excluding
the high risk populations, as done in the AHRQ QIs. In pediatrics,
unlike adult settings, uncomplicated patients are much less likely to
develop the types of outcomes measured by the QIs. Interventions are
best aimed at populations that are more likely to develop a
complication, and these interventions may in turn lower the overall rate
of the indicator. Further, focusing only on low risk populations reduces
the complication rate to a level where meaningful comparisons over time
and institutions may be difficult. Such low numbers may remain a problem
for community hospitals that do not treat high risk patients.

*Concerns about bias and preferences for addressing differing risk
groups*

Expanding an indicator to include high risk populations may have the
consequence of increasing potential bias. Higher risk children tend to
be concentrated in children's hospitals, potentially biasing any
comparisons between children's hospitals and community hospitals.
Because of differing distributions of complicated cases, indicators that
include lower and higher risk patients require adjustment for
comorbidities, reasons for admission, age and other measures of severity
of illness. Panelists noted that risk adjustment using administrative
data will be limited in effectiveness, so comparisons should generally
focus on similar types of hospitals. For instance, tertiary care
children's hospitals should be compared only to other children's
hospitals.

Stratification of indicators was also preferred by panelists.
Stratification allows an institution to examine the rate in populations
of differing risk, to better target interventions, in addition to
providing a means for less biased inter-institutional comparisons.
Stratification however does reduce the denominator for each comparison;
in some cases, this reduction may make meaningful comparisons
impractical.

One area of concern, particularly for panelists examining potentially
avoidable hospitalization indicators, was adjustment for social factors.
Examples of social factors that may influence outcomes, but remain
beyond provider control, include cultural traditions that inhibit early
presentation to a health care provider, or fear of repercussions of
presenting, such as deportation for illegal residents. Other factors
raised as potentially associated with different socio-economic settings
include higher rates of poor health behaviors leading to poorer overall
health and poorer outcome, and poor adherence to medical care. Risk
adjustment for these types of factors is not straightforward. For
instance, in one area cultural factors associated with a certain ethnic
group may adversely affect outcomes, while that same ethnic group in
another area of the country may not exhibit the same cultural factors.
In addition, broad ethnic categorizations, such as Asian, Black or
Hispanic, are unlikely to be refined enough to capture specific groups
with differential ability to derive benefits consistently from high
quality care. Other factors, such as illegal immigrant status, are not
available in any state database at this time. Some panelists grappled
with the issue that risk adjustment may not be desirable for indicators
that examine area level health care, since interventions to target high
risk groups may still be effective in reducing poor outcomes, by
reducing poor health behaviors or providing culturally sensitive care to
improve compliance. Further supporting the argument against risk
adjustment is that social factors tend to be correlated with poor access
to quality care, and it may be important to focus policy-makers'
attention on these high-risk communities. Risk adjustment may in fact
adjust away some of the poor access to care the indicator is intended to
measure. Despite these concerns, panelists felt that the potential
impact of the health care community on these overarching social factors
is small, and that risk adjustment is essential for fair comparisons
between areas. Since detailed risk adjustment by cultural group is not
possible, socioeconomic status adjustment was recommended as a minimal
risk adjustment approach for social factors. It should be noted that
such adjustment may result in less ability to detect disparities between
socioeconomic groups. Both raw and risk adjusted rates are important
when investigating area level rates.

*Additional data elements to improve indicator definitions*

Panelists frequently requested modifications to indicators that would
require additional data elements to address suggestions more fully than
is feasible with the current national data set. Currently, present on
admission data elements are available in two states only. These data
allow the differentiation between complications and comorbidities.
Specific codes for some important comorbidities, such as coagulopathies,
include conditions that could be acquired during the hospitalization and
reflect poor quality of care (or simply be recorded only due to poor
outcomes). In addition, present on admission data would allow the
expansion of surveillance for potential complications that cannot be
otherwise distinguished from comorbidities. For example, acute renal
failure and diabetic complications measured by the QI "Postoperative
Metabolic and Physiologic Derangements" are rare in children; however,
panelists expressed interest in expanding the complication set to
include less severe, but clinically important electrolyte imbalances.
Currently these complications are impossible to distinguish from
imbalances which may be present on admission. Present on admission data
is also important for adult indicators, and in recent years have
attracted more attention from the quality improvement field and
researchers. Other data elements of interest included expansion of the
base dataset to outpatient surgeries to track complications of
outpatient surgery; readmission data to track readmissions for
complications or for chronic diseases; clinical and pharmacy data to
improve risk adjustment and specificity of the indicators; and linkages
to vital or maternal records to improve risk adjustment and specificity
for neonatal indicators.

*Purpose of indicators*

The intended use of the indicators affected the opinion of the panelists
regarding their overall usefulness. As with the AHRQ QIs, panelists were
more interested in establishing broader definitions when the indicators
would only be used for internal quality improvement instead of
comparative reporting. As discussed above, the importance of including
high risk populations in children heightened the concern about
appropriate use of the indicators. We asked panelists for two overall
usefulness ratings, one for quality improvement and one for comparative
reporting. In general, panelists were more conservative in the
recommendations for comparative reporting, with panelists not
recommending six indicators for comparative reporting that were
recommended for internal quality improvement.

## Overall Results from Peer Review

## 

We received fifteen responses from peer reviewers. Peer reviewers
offered favorable comments with constructive suggestions for content and
presentation enhancements. Peer reviewers offered both general
recommendations and feedback targeted to specific indicators. This
section describes three major themes of the peer review responses.

*Expanded data*

Like our panelists, our peer reviewers also advocated for indicators
based on expanded data sets. Citing the limitations of the discharge
data based on administrative data, peer reviewers called for the option
to use more clinically rich data. Some reviewers argued that many
hospital systems do have access to additional data fields, and would
benefit from tools that would help them use such additional data for
quality improvement purposes. Some of the additional data for which peer
reviewers suggested included: 1) outpatient surgical data which would
better track complications following common operations in children, such
as tonsillectomy and adenoidectomy; 2) condition present on admission
data, which could improve specificity of the indicators and allow the
expansion of complications monitored by the indicators, such as the
addition of physiologic derangements; 3) readmission data, which would
allow for tracking complications occurring after discharge; and 4)
laboratory data and pharmacy data which could improve sensitivity and
specificity of the indicators, and expand the possible indicator set to
include process based measures and expand risk adjustment options.

*Data standards*

Peer reviewers noted that data quality and detail vary from institution
to institution and state to state. For instance, some states require
more diagnosis fields than others. With the truncation of diagnosis
codes, some secondary diagnosis codes may not be included,
systematically biasing the data. One peer reviewer noted that E codes
are not consistently used in pediatric patients, creating bias for
indicators based on E codes. Like E-codes, another peer reviewer noted
that procedure dates are also not standard in data sets. Finally, some
reviewers noted that systems that are in place for adult coding, through
the Medicare audit program, do not affect coding for children. As a
result, coding for children may be more problematic. They advocated for
standardized quality control approaches for pediatric hospital data.

The desire to ensure data standards can sometimes be at odds with the
desire to use additional data available to some but not all hospitals.
The balance between improving the indicator set through use of better
and additional data, and the potential bias created when using data only
consistently available at some institutions should be considered
carefully.

*Validity testing*

Peer reviewers noted that while the current development work was
rigorous and thoughtful, establishing the validity of these indicators
will require further testing. Testing should include the examination of
the sensitivity and specificity of the individual codes used in each
indicator, as well as the sensitivity and specificity of the indicators
to identify potential quality concerns. Peer reviewers suggested using
chart review methods.

*Reinforcement of panel commentary*

Peer reviewers in general agreed with panelist thoughts about
modifications and overall usefulness of the indicators. For instance,
several peer reviewers expressed support for the removal of Bacterial
Pneumonia and Postoperative Metabolic and Physiologic Derangement from
the candidate indicator set. Peer reviewers also agreed with panelists
regarding the expansion of some indicators to include high risk
children. Similarly, they reiterated the desirability of stratification
in some instances, while noting the potential issue of rates being too
low for some strata or subgroups (e.g., stratification by procedure
class of Accidental Puncture and Laceration indicator). The need for
risk adjustment was also identified as crucial by both peer reviewers.
Rarely, peer reviewers suggested views that contradicted panelist input.
These cases were carefully considered and appropriate adjustments to
indicator definitions were made. In general, when panelists specifically
discussed or recommended a change, which later a peer reviewer disagreed
with, the change was investigated empirically and clinically evaluated.
If no further evidence substantiated a change, the panel's
recommendation remained. If further evidence highlighted potential
problems with indicator definitions recommended by the panelists,
appropriate adjustments were made.

*Additional suggestions for existing and future indicators*

In addition to commenting on the panelists' responses to the indicators,
the peer reviewers offered new recommendations. Some suggested
additional indicators for consideration, such as central line
thromboses, craniotomy volume, or admission rates for pneumonia that may
be preventable through vaccination (i.e. Prevnar). They also suggested
indicator-specific modifications, such as expanding the definition of
"immunocompromised patients," or adding additional exclusion criteria.
Some suggestions require additional data for implementation, but others,
that are feasible with current data constraints, were evaluated and
implemented (e.g. expansion of conditions considered
"immunocompromised". Finally, based on their own experiences as quality
experts and clinicians, peer reviewers offered advice regarding the
implementation of these indicators in the real world setting. For
instance, they noted that since transfusion reactions are rare, that
indicator is more useful as a case finding tool, and highlighted the
need for clear communication regarding the distinction between area
level and hospital level indicators.

## Detailed Results by Indicator 

This section provides detailed results for each indicator reviewed by
clinical panels in Phase I. Each indicator section is organized as
follows:

1.  Table summarizing the indicator definition followed by a table
    summarizing associated national rates calculated from the 2003 KID
    data set, including, for recommended indicators, a comparison of
    children's versus community hospitals, whenever applicable (i.e.,
    for hospital level indicators, not for area level indicators).
    Except for population based denominators, all numerators are strict
    subsets of the denominators;

2.  Paragraph describing the final recommendation for indicator
    implementation;

3.  Series of three tables summarizing the revisions from the original
    AHRQ QIs as indicators underwent research team review, panel input
    and final research team deliberation;

4.  Succinct clinical rationale providing a description akin to what
    might be used in the National Quality Measures Clearinghouse;

5.  Text presenting a summary of findings from the original panels
    reviewing the AHRQ QIs (for Patient Safety Indicators only), as
    presented to the pediatric panels for their review;

6.  Text presenting a summary of the pediatric panel review discussion,
    followed by final ratings where applicable;

7.  Short summary of empirical analyses conducted to refine indicator
    definitions, whenever applicable;

8.  Paragraph providing general additional evidence (i.e., not specific
    to pediatric population) from non-literature sources, if available;

9.  Results of the literature review providing pertinent pediatric
    evidence (i.e., for or against the indicator or its underlying
    concept) found in the published literature.

+----------------------------------+-----------------------------------+
| ### ACCIDEN                      |                                   |
| TAL PUNCTURE OR LACERATION (PSI) |                                   |
+----------------------------------+-----------------------------------+
| **Indicator definition:**        |                                   |
|                                  |                                   |
| Cases of technical difficulty    |                                   |
| (e.g., accidental cut or         |                                   |
| laceration during procedure) per |                                   |
| 1,000 eligible discharges        |                                   |
| (population at risk). See        |                                   |
| Pediatric Quality Indicator      |                                   |
| Technical Specifications.        |                                   |
+----------------------------------+-----------------------------------+
| **Definition of technical        | **Definition of population at     |
| difficulty (e.g. accidental cut  | risk:**                           |
| or laceration)**:                |                                   |
|                                  | Patients eligible to be included  |
|                                  | in this indicator:                |
+----------------------------------+-----------------------------------+
| ***Secondary diagnosis code      | ***a. All medical and surgical    |
| for:***                          | discharges (defined by DRGs), age |
|                                  | 0-17 years, except exclusions     |
| Accidental cut, puncture,        | (see below).***                   |
| perforation or hemorrhage during |                                   |
| medical care:                    | ***b. Exclude patients with       |
|                                  | specified principal diagnosis of  |
| -   Surgical operation           | accidental cut, puncture or       |
|     \[E870.0\]                   | laceration.***                    |
|                                  |                                   |
| -   Infusion or transfusion      | ***c. Stratify rates by low birth |
|     \[E870.1\]                   | weight neonate (under 2000g) and  |
|                                  | other patients.***                |
| -   Kidney dialysis or other     |                                   |
|     perfusion \[E870.2\]         | ***d. Risk adjust rates by        |
|                                  | procedure type recorded in        |
| -   Injection or vaccination     | patient record:***                |
|     \[E870.3\]                   |                                   |
|                                  | ***i. no therapeutic***           |
| -   Endoscopic examination       |                                   |
|     \[E870.4\]                   | ***ii. minor therapeutic***       |
|                                  |                                   |
| -   Aspiration of fluid or       | ***iii. one major therapeutic     |
|     tissue, puncture, and        | without diagnostic***             |
|     catheterization \[E870.5\]   |                                   |
|                                  | ***iv. one major therapeutic with |
| -   Heart catheterization        | minor diagnostic***               |
|     \[E870.6\]                   |                                   |
|                                  | ***v. one major therapeutic with  |
| -   Administration of enema      | major diagnostic***               |
|     \[E870.7\]                   |                                   |
|                                  | ***vi. two major therapeutic***   |
| -   Other specified medical care |                                   |
|     \[E870.8\]                   | ***vii. three or more major       |
|                                  | therapeutic***                    |
| -   Unspecified medical care     |                                   |
|     \[E870.9\]                   | ***e. Stratify rates by clinical  |
|                                  | category:***                      |
| Accidental puncture or           |                                   |
| laceration during a procedure    | ***i. Eye, ear, nose, mouth,      |
| \[998.2\]                        | throat, skin, breast, and other   |
|                                  | low-risk procedures***            |
|                                  |                                   |
|                                  | ***ii. Thoracic, cardiovascular,  |
|                                  | and specified neoplastic          |
|                                  | procedures***                     |
|                                  |                                   |
|                                  | ***iii. Kidney, and male/female   |
|                                  | reproductive procedures***        |
|                                  |                                   |
|                                  | ***iv. Infectious, immunological, |
|                                  | hematological, and ungroupable    |
|                                  | procedures***                     |
|                                  |                                   |
|                                  | ***v. Trauma, orthopedic, and     |
|                                  | neurologic procedures***          |
|                                  |                                   |
|                                  | ***vi. Gastrointestinal,          |
|                                  | hepatobiliary, and endocrine      |
|                                  | procedures***                     |
|                                  |                                   |
|                                  | ***f. Exclude normal newborns     |
|                                  | \[DRG 391\].***                   |
|                                  |                                   |
|                                  | ***g. Exclude newborns with a     |
|                                  | birth weight less than 500g.***   |
|                                  |                                   |
|                                  | ***h. Exclude obstetric patients  |
|                                  | (MDC 14).***                      |
+----------------------------------+-----------------------------------+

+-----------------------------------------------------------------------+
| **Rates based on year 2003 KIDs' Inpatient Database (per 1000):**     |
|                                                                       |
|   ------------------- -                                               |
| ---------------- -------------------- ------------ ------------------ |
|   **O                                                                 |
| VERALL**         0.801                                                |
|                                                                       |
|   **A                                                                 |
| ge stratified                                                         |
|   rat                                                                 |
| es:**                                                                 |
|                                                                       |
|   **N                                                                 |
| eonate, \<       0.819                                                |
|   200                                                                 |
| 0g**                                                                  |
|                                                                       |
|   **N                                                                 |
| eonate, ≥        0.495                                                |
|   200                                                                 |
| 0g**                                                                  |
|                                                                       |
|   **2                                                                 |
| 9 days -- 364    1.008                                                |
|   day                                                                 |
| s**                                                                   |
|                                                                       |
|   **1                                                                 |
|  -- 2 years**    0.642                                                |
|                                                                       |
|   **3                                                                 |
|  -- 5 years**    0.908                                                |
|                                                                       |
|   **6                                                                 |
|  -- 12 years**   0.990                                                |
|                                                                       |
|   **1                                                                 |
| 3 -- 17 years**  1.113                                                |
|                                                                       |
|                                                                       |
|                                                                       |
|                                                                       |
|   **C                                                                 |
| linical                                                               |
|   str                                                                 |
| atification**                                                         |
|                                                                       |
|   **S                                                                 |
| trata 1. Eye,                                           0.259         |
|   ear                                                                 |
| , nose, mouth,                                                        |
|   thr                                                                 |
| oat, skin,                                                            |
|   bre                                                                 |
| ast, and other                                                        |
|   low                                                                 |
| -risk                                                                 |
|   pro                                                                 |
| cedures**                                                             |
|                                                                       |
|   **S                                                                 |
| trata 2.                                                0.734         |
|   Tho                                                                 |
| racic,                                                                |
|   car                                                                 |
| diovascular, and                                                      |
|   spe                                                                 |
| cified                                                                |
|   neo                                                                 |
| plastic                                                               |
|   pro                                                                 |
| cedures**                                                             |
|                                                                       |
|   **S                                                                 |
| trata 3. Kidney,                                        2.261         |
|   and                                                                 |
|  male/female                                                          |
|   rep                                                                 |
| roductive                                                             |
|   pro                                                                 |
| cedures**                                                             |
|                                                                       |
|   **S                                                                 |
| trata 4.                                                0.418         |
|   Inf                                                                 |
| ectious,                                                              |
|   imm                                                                 |
| unological,                                                           |
|   hem                                                                 |
| atological, and                                                       |
|   ung                                                                 |
| roupable                                                              |
|   pro                                                                 |
| cedures**                                                             |
|                                                                       |
|   **S                                                                 |
| trata 5. Trauma,                                        1.143         |
|   ort                                                                 |
| hopedic, and                                                          |
|   neu                                                                 |
| rologic                                                               |
|   pro                                                                 |
| cedures**                                                             |
|                                                                       |
|   **S                                                                 |
| trata 6.                                                1.768         |
|   Gas                                                                 |
| trointestinal,                                                        |
|   hep                                                                 |
| atobiliary, and                                                       |
|   end                                                                 |
| ocrine                                                                |
|   pro                                                                 |
| cedures**                                                             |
|                                                                       |
|                                                                       |
|                                                                       |
|                                                                       |
|   **H                                                                 |
| ospital type**                                                        |
|                                                                       |
|   **C                                                                 |
| hildren's**                        **Non-Children's**                 |
|                                                                       |
|   **OVERAL                                                            |
| L**         1.578             **OVERALL**                       0.440 |
|                                                                       |
|   **A                                                                 |
| ge stratified                      **Age stratified                   |
|   rat                                                                 |
| es:**                              rates:**                           |
|                                                                       |
|   **Neonat                                                            |
| e, \<       1.717             **Neonate, \<                     0.502 |
|   200                                                                 |
| 0g**                               2000g**                            |
|                                                                       |
|   **Neonat                                                            |
| e, ≥        1.257             **Neonate, ≥ 2000g**              0.339 |
|   200                                                                 |
| 0g**                                                                  |
|                                                                       |
|   **29 day                                                            |
| s -- 364    2.058             **29 days -- 364                  0.301 |
|   day                                                                 |
| s**                                days**                             |
|                                                                       |
|   **1 -- 2                                                            |
|  years**    1.328             **1 -- 2 years**                  0.224 |
|                                                                       |
|   **3 -- 5                                                            |
|  years**    1.628             **3 -- 5 years**                  0.386 |
|                                                                       |
|   **6 -- 1                                                            |
| 2 years**   1.470             **6 -- 12 years**                 0.646 |
|                                                                       |
|   **13 --                                                             |
| 17 years**  1.717             **13 -- 17 years**                0.824 |
|                                                                       |
|   **C                                                                 |
| linical                            **Clinical strata:**               |
|   str                                                                 |
| ata:**                                                                |
|                                                                       |
|   **Strata                                                            |
|  1**        0.480             **Strata 1**                      0.145 |
|                                                                       |
|   **Strata                                                            |
|  2**        1.724             **Strata 2**                      0.128 |
|                                                                       |
|   **Strata                                                            |
|  3**        3.138             **Strata 3**                      1.746 |
|                                                                       |
|   **Strata                                                            |
|  4**        0.599             **Strata 4**                      0.235 |
|                                                                       |
|   **Strata                                                            |
|  5**        1.666             **Strata 5**                      0.594 |
|                                                                       |
|   **Strata                                                            |
|  6**        2.917             **Strata 6**                      1.163 |
|   ------------------- -                                               |
| ---------------- -------------------- ------------ ------------------ |
+-----------------------------------------------------------------------+

***Status summary.*** Based on the current evidence base, from the
pediatric literature review, pediatric panel review, and empirical
analyses, this indicator will be included in the pediatric quality
indicator set. Panelists rated this indicator favorably, with
indeterimimate agreement for internal quality improvement, but did not
recommend the indicator for comparative reporting purposes.

***Changes from AHRQ QI Implemented Prior to Pediatric Panel Review***

  ---------------------- ------------------------ ------------------------
  **AHRQ QI definition** **Pediatric indicator    **Reason implemented**
                         definition**             

  All ages               Age 0 -- 17              Pediatric age range

  Includes all patients. Exclude normal newborns  Normal newborns do not
                         \[DRG 391\].             usually undergo
                                                  procedures that put them
                                                  at risk for these
                                                  complications

  Includes all patients  Exclude newborns with a  Excluded from all
                         birth weight less than   indicators due to very
                         500g.                    high risk nature and
                                                  bias related to delivery
                                                  practices (i.e.
                                                  attempting delivery vs.
                                                  allowing fetal death).

  No stratification.     Stratify rates by low    Small infants may be at
                         birth weight neonate     higher risk for this
                         (under 2000g) and other  procedure than larger
                         patients.                patients due to smaller
                                                  anatomy and fragile
                                                  structures.
  ---------------------- ------------------------ ------------------------

***Changes Implemented to Pediatric Indicator as a Result of Pediatric
Panel Review***

  ---------------------- ------------------------ ------------------------
  **Pre-panel            **Post-panel indicator   **Reason implemented**
  definition**           definition**             

  All procedures         Stratification by        Risks vary by procedure
  analyzed together.     procedure type, based on type. Stratification by
                         clinical MDC groups.     clinically coherent
                                                  categories improves the
                                                  usefulness of the
                                                  indicator.

  No indicator specific  Risk adjustment specific Risks vary by number of
  risk adjustment.       to this indicator based  surgical encounters, the
                         on procedure type (i.e.  intensity of the
                         diagnostic, therapeutic) procedure and the
                         and number of            purpose of the
                         procedures.              procedure.
  ---------------------- ------------------------ ------------------------

Changes considered, but not implemented

  ----------------------- ----------------------- -----------------------
  **AHRQ QI definition**  **Pediatric indicator   **Reason not
                          definition**            implemented**

  None                                            
  ----------------------- ----------------------- -----------------------

***Clinical rationale***

This indicator is intended to track injuries occurring during a
procedure, specifically accidental cut, puncture, perforation, or
laceration. These procedures may be prevented through proper technique
during procedures.

***Summary of AHRQ QI clinician panel reviews***

This indicator was reviewed during our development of the Patient Safety
Indicators, which included a clinical panel review. For this indicator
the panel consisted of eight physicians: an internist and
gastroenterologist, a general surgeon, a cardiologist and critical care
physician, two interventional radiologists, two specialized nurses, and
an anesthesiologist. The panel reviewed several other indicators. In the
course of review the panel suggested the following:

-   The original indicator reviewed by this panel was entitled
    "Technical difficulty with care." During the course of review, the
    panel suggested removing complications such as failure of sterile
    precautions, cataract fragments in the eye following cataract
    surgery, emphysema arising from a procedure and air embolism, due to
    questionable clinical significance and variability in reporting.

-   Panelists noted that for the remaining codes (those in the indicator
    presented here) reporting may be variable, although they thought
    only severe cases would be reported in most cases.

-   One panelist suggested that limiting the indicator to re-operations
    may be one way to improve the indicator. Further investigation is
    required to determine whether or not this is feasible or desirable,
    given the small number that would remain in the numerator.

***Results of pediatric clinician panel review***

As part of the current pediatric indicator development process, this
indicator was reviewed by a panel of ten pediatric clinicians, including
one neonatologist, one infectious disease specialist, one ambulatory
care pediatrician, one pediatric hospitalist, one pediatric
cardiovascular surgeon, one pediatric oncologist, two pediatric
surgeons, one pediatric interventional radiologist, and one pediatric
critical care physician. The panel reviewed several other indicators. In
the course of review the panel suggested the following:

-   Panelists noted that the risk of accidental puncture or laceration
    varies greatly by the type of procedure. Panelists suggested that
    this indicator be stratified by procedure class, namely 1)
    endoscopy, 2) catheter-based procedures, 3) venous access, and 4)
    major surgeries. Some patients will have more than one procedure
    type. Panelists suggested that these patients may be placed in yet
    another category for multiple procedures, or that a hierarchy of
    procedures could be developed. Extensive redefinition work is
    required to implement this change. For this reason this indicator
    was re-rated by this panel only after significant modification to
    the definition and a second round of rating (see below).

The same panel participated in a second round of rating, which included
preliminary rating, followed by a conference call, and a final rating.
The panel was identical except for the attrition of three panelists
(pediatric cardiovascular surgeon, pediatric oncologist, pediatric
hospitalist). The panel re-reviewed three other indicators. In the
course of review the panel further suggested the following, in addition
to the comments from the previous review:

-   The panelists were presented with a stratification scheme based on
    the number and type (i.e. diagnostic or therapeutic) of procedures a
    patient underwent during a hospitalization. This is the same scheme
    that is now used as risk adjustment in this indicator. As a
    stratification scheme the panelists felt that it was too complex.
    They expressed concern about potential low sample size in some of
    the strata at individual hospitals. In addition, they felt that the
    usefulness of the scheme was questionable, since it would be
    difficult to understand how and in which service to intervene if
    rates were high. They suggested an alternative stratification scheme
    that would group together procedures in a more clinically coherent
    manner, such as all endoscopy procedures, or cardiac
    catheterizations. As with the original stratification scheme, the
    panel expressed concern over low sample sizes in each strata and
    suggested empirically investigating more clinically coherent strata.

**^Post-conference\ call\ panel\ ratings^**

  --------------------------- ------------- --------------------------------
  **Question**                **Median**    **Agreement status**

  Overall rating -- internal  7             Indeterminate agreement
  QI                                        

  Overall rating --           6.5           Indeterminate agreement
  comparative purposes                      

  Not present on admission    7.5           Agreement

  Preventability              7             Indeterminate agreement

  Due to medical error        6             Agreement

  Charting by physicians      6             Indeterminate agreement

  Lack of bias                5             Indeterminate agreement

  Final recommendation        Internal QI:  
                              Acceptable    
                              (-)           
                              Comparative   
                              purposes: Not 
                              recommended   
  --------------------------- ------------- --------------------------------

***Empirical analyses to inform indicator definition***

The following empirical analyses were completed after the initial panel
review using the 2003 KIDs' Inpatient Database (KID).

We investigated several approaches to stratification through a series of
empirical analyses. These approaches included applying the HCUP
Procedure Classes developed by AHRQ. We found that both the number of
major therapeutic procedures, and the presence of a diagnostic procedure
increased the risk of this complication. Based on these analyses we
developed a seven strata system, which we tested empirically. Each
strata represented a stepwise increase in risk. See table below to see
the rate for each strata based on the definition at the time of the
analysis. This stratification system was presented to panelists, who
suggested that the system was clinically less useful than one based on
more clinical concepts (e.g. organ system, procedure operator). We
investigated several options, by identifying which DRGs appeared in the
numerator and denominator in each of our original strata, as well as by
MDC. See table below for rates by MDC based on the definition at the
time of analysis. We found no clear patterns that enabled maping of our
original seven strata, which empirically performed well, to more
clinical concepts. Based on these analyses and panel feedback we adopted
the empirically derived classification system as risk adjustment and the
MDC based system as stratification.

**Rates for strata based on definition at time of analysis**

  ---------------------------------------------------------- ---------------
  **Stratum based on empirical results**                     **Rate/1000**

  No therapeutic                                             0.047

  Minor therapeutic                                          0.362

  1 major therapeutic, with no diagnostic                    1.155

  1 major therapeutic, with minor diagnoistic                2.317

  1 major therapeutic, with major diagnoistic                4.784

  2 major therapeutic                                        7.031

  3 or more major therapeutic                                14.25
  ---------------------------------------------------------- ---------------

**Rates for MDCs, based on definition at time of analysis**

  ----------------------------------------------------- -----------------
  **MDC**                                               **Rate per 1000**

  Ungroupable or Pre-MDC                                6.52

  MDC 1. Diseases and Disorders of the Nervous Systems  2.79

  MDC 2. Diseases and Disorders of the Eye              0.99

  MDC 3. Diseases and Disorders of the Ear, Nose, Mouth 1.12
  and Throat                                            

  MDC 4. Diseases and Disorders of the Respiratory      5.04
  System                                                

  MDC 5. Diseases and Disorders of the Circulatory      8.66
  System                                                

  MDC 6. Diseases and Disorders of the Digestive System 5.33

  MDC 7. Diseases and Disorders of the Hepatobiliary    7.48
  System and Pancreas                                   

  MDC 8. Diseases and Disorders of the Musculoskeletal  2.79
  System and Connective Tissue                          

  MDC 9. Diseases and Disroders of the Skin,            0.55
  Subcutaneous Tissue and Breast                        

  MDC 10. Endocrine, Nutritional and Metabolic Diseases 12.55
  and Disorders                                         

  MDC 11. Diseases and Disorders of the Kidney and      6.55
  Urinary Tract                                         

  MDC 12. Diseases and Disorders of the Male            3.97
  Reproductive System                                   

  MDC 13. Diseases and Disorders of the Female          6.55
  Reproductive System                                   

  MDC 16. Diseases and Disorders of the Blood and Blood 6.35
  Forming Organs and Immunological Disorders            

  MDC 17. Myeloproliferative Diseases and Disorders,    9.18
  and Poorly Differentiated Neoplasms                   

  MDC 18. Infectious and Parasitic Disease (Systemic or 4.96
  unspecified sites)                                    

  MDC 19. Mental Diseases and Disorders                 0.00

  MDC 21. Injuries, Poisonings and Toxic Effects of     2.96
  Drugs                                                 

  MDC 22. Burns                                         0.67

  MDC 23. Factors Influencing Health Status and Other   0.00
  Contacts with Health Services                         

  MDC 24. Multiple Significant Trauma                   1.98

  MDC 25. Human Immunodeficiency Virus Infections       0.00
  ----------------------------------------------------- -----------------

***Literature based evidence specific to pediatric population***

Surgeries in pediatric patients, because of their smaller anatomy, can
be technically more complex and can carry a high risk of accidental
puncture or laceration (e.g., 2.22 per 1,000 discharges at 0-17 years,
1.84 at 18-44 years, 2.82 at 45-64 years, and 3.47 at 65 or more
years).^10^ This indicator was investigated by two groups, although the
definition differed slightly from the definition proposed above. Miller
and colleagues analyzed HCUP data in 2000, using a publicly released
version of this indicator applied to a pediatric population, and found a
significant incidence of accidental puncture or laceration in pediatric
patients (1.0 per 1,000 in 2000 among 0-18 year old children).^11^
Additionally, Miller & Zhan found that this error resulted in increased
mean length of stay (by 7.7 days) and charges per stay (\$41,204 on
average) in affected patients, with 2.7 times higher odds of in-hospital
mortality (after adjusting for age, gender, expected payer, up to 30
comorbidities, and multiple hospital characteristics, including
ownership, teaching status, nursing expertise, urban location, bed size,
pediatric volume, coding intensity, ICU bed percentage, and surgical
discharge percentage).^11^ Sedman et al found observed rates varying
from 1.7 per 1,000 in 1999 to 1.9 per 1,000 in 2002 in the NACHRI
database (i.e., a slight upward trend over time), when applying the
publicly released AHRQ QI definition to a pediatric population.^12^

+----------------------------------+-----------------------------------+
| ### DECUBITUS ULCER (PSI)        |                                   |
+----------------------------------+-----------------------------------+
| **Indicator definition:**        |                                   |
|                                  |                                   |
| Number of patients with          |                                   |
| decubitus ulcer (see definition  |                                   |
| and exclusions below) per 1,000  |                                   |
| eligible admissions (population  |                                   |
| at risk). See Pediatric Quality  |                                   |
| Indicator Technical              |                                   |
| Specifications.                  |                                   |
+----------------------------------+-----------------------------------+
| **Definition of decubitus        | **Definition of population at     |
| ulcer:**                         | risk:**                           |
|                                  |                                   |
|                                  | Patients eligible to be included  |
|                                  | in this indicator:                |
+----------------------------------+-----------------------------------+
| ***Secondary diagnosis code      | ***a. All medical and surgical    |
| for:***                          | patients (defined by DRG), age    |
|                                  | 0-17 years, except exclusions     |
| Decubitus ulcer:                 | (see below).***                   |
|                                  |                                   |
| -   Unspecified site \[707.00\]  | ***b. Include only patients with  |
|                                  | a length of stay of 5 or more     |
| -   Elbow \[707.01\]             | days.***                          |
|                                  |                                   |
| -   Upper back \[707.02\]        | ***c. Exclude patients in MDC 9   |
|                                  | (Diseases and disorders of the    |
| -   Lower back \[707.03\]        | skin, subcutaneous tissue and     |
|                                  | breast).***                       |
| -   Hip \[707.04\]               |                                   |
|                                  | ***d. Exclude all neonates (age   |
| -   Buttock \[707.05\]           | \< 28 days).***                   |
|                                  |                                   |
| -   Ankle \[707.06\]             | ***e. Stratify by high risk       |
|                                  | (hemi-, para-, and quadriplegia,  |
| -   Heel \[707.07\]              | spinia bifida and anoxic brain    |
|                                  | damage (dx codes 348.1, 768.5),   |
| -   Other site \[707.09\]        | mechanical ventilation \>96       |
|                                  | hrs.***                           |
|                                  |                                   |
|                                  | ***f. Exclude newborns with a     |
|                                  | birth weight less than 500g.***   |
|                                  |                                   |
|                                  | ***g. Exclude patients            |
|                                  | transferring in from long term    |
|                                  | care facility or an acute care    |
|                                  | facility.***                      |
|                                  |                                   |
|                                  | ***h. Exclude obstetric patients  |
|                                  | (MDC 14)***                       |
|                                  |                                   |
|                                  | ***i. Exclude patients with a     |
|                                  | principal diagnosis of decubitus  |
|                                  | ulcer.***                         |
|                                  |                                   |
|                                  | ***j. Patients with an ICD-9-CM   |
|                                  | procedure code for debridement or |
|                                  | pedicle graft before or on the    |
|                                  | same day as the major operating   |
|                                  | room procedure (surgical cases    |
|                                  | only).***                         |
+----------------------------------+-----------------------------------+

+-----------------------------------------------------------------------+
| **Rates based on year 2003 KIDs' Inpatient Database** (per 1000):     |
|                                                                       |
|   ------------------ -                                                |
| ---------------- -------------------- ------------ ------------------ |
|   **                                                                  |
| OVERALL**        3.16                                                 |
|                                                                       |
|   **                                                                  |
| Age stratified                                                        |
|   ra                                                                  |
| tes:**                                                                |
|                                                                       |
|   **                                                                  |
| Neonate, \<                                                           |
|   20                                                                  |
| 00g**                                                                 |
|                                                                       |
|   **                                                                  |
| Neonate, ≥                                                            |
|   20                                                                  |
| 00g**                                                                 |
|                                                                       |
|   **                                                                  |
| 29 days -- 364   0.86                                                 |
|   da                                                                  |
| ys**                                                                  |
|                                                                       |
|   **                                                                  |
| 1 -- 2 years**   1.93                                                 |
|                                                                       |
|   **                                                                  |
| 3 -- 5 years**   1.86                                                 |
|                                                                       |
|   **                                                                  |
| 6 -- 12 years**  3.58                                                 |
|                                                                       |
|   **                                                                  |
| 13 -- 17 years** 4.89                                                 |
|                                                                       |
|                                                                       |
|                                                                       |
|                                                                       |
|   **                                                                  |
| Clinical                                                              |
|   st                                                                  |
| ratification**                                                        |
|                                                                       |
|   **                                                                  |
| High risk:                                              23.08         |
|   Qu                                                                  |
| adri, hemi-,                                                          |
|   pa                                                                  |
| raplegia, spina                                                       |
|   bi                                                                  |
| fida, anoxic                                                          |
|   br                                                                  |
| ain damage**                                                          |
|                                                                       |
|   **                                                                  |
| Low risk: All                                           1.43          |
|   ot                                                                  |
| her patients**                                                        |
|                                                                       |
|                                                                       |
|                                                                       |
|                                                                       |
|   **                                                                  |
| Hospital type**                                                       |
|                                                                       |
|   **                                                                  |
| Children's**                       **Non-Children's**                 |
|                                                                       |
|   **OVER                                                              |
| ALL**        4.33              **OVERALL**                       1.79 |
|                                                                       |
|   **                                                                  |
| Age stratified                     **Age stratified                   |
|   ra                                                                  |
| tes:**                             rates:**                           |
|                                                                       |
|   **                                                                  |
| Neonate, \<                        **Neonate, \<                      |
|   20                                                                  |
| 00g**                              2000g**                            |
|                                                                       |
|   **                                                                  |
| Neonate, ≥                         **Neonate, ≥ 2000g**               |
|   20                                                                  |
| 00g**                                                                 |
|                                                                       |
|   **29 d                                                              |
| ays -- 364   1.38              **29 days -- 364                  0.26 |
|   da                                                                  |
| ys**                               days**                             |
|                                                                       |
|   **1 --                                                              |
|  2 years**   2.14              **1 -- 2 years**                  1.40 |
|                                                                       |
|   **3 --                                                              |
|  5 years**   2.29              **3 -- 5 years**                  1.05 |
|                                                                       |
|   **6 --                                                              |
|  12 years**  4.92              **6 -- 12 years**                 1.80 |
|                                                                       |
|   **13 -                                                              |
| - 17 years** 7.99              **13 -- 17 years**                2.66 |
|                                                                       |
|   **                                                                  |
| Clinical                           **Clinical strata:**               |
|   st                                                                  |
| rata:**                                                               |
|                                                                       |
|   **High                                                              |
| risk**      21.76             **High risk**                     24.53 |
|                                                                       |
|   **Low                                                               |
| risk**       2.06              **Low risk**                      0.76 |
|   ------------------ -                                                |
| ---------------- -------------------- ------------ ------------------ |
+-----------------------------------------------------------------------+

***Status summary.*** Based on the current evidence base, from the
pediatric literature review, pediatric panel review, and empirical
analyses, this indicator will be included in the pediatric quality
indicator set. Panelists rated this indicator favorably and with
indeterminate agreement for internal quality improvement and comparative
reporting purposes.

***Changes from AHRQ QI Implemented Prior to Pediatric Panel Review***

  ---------------------- ------------------------ ------------------------
  **AHRQ QI definition** **Pediatric indicator    **Reason implemented**
                         definition**             

  Age 0 -- 85            Age 0 -- 17              Pediatric age range

  Premature neonates     Exclude normal neonates. Normal neonates not at
  included.                                       risk for developing
                                                  condition.

  Premature neonates     Exclude newborns with a  Excluded from all
  included.              birth weight less than   indicators due to very
                         500g.                    high risk nature and
                                                  bias related to delivery
                                                  practices (i.e.
                                                  attempting delivery vs.
                                                  allowing fetal death).

  No stratification      Stratify by low birth    Premature neonates are
                         weight neonates (2000 g  at risk for decubiti by
                         and under) and other     different mechanism, due
                         patients.                to fragile skin.

  Exclude patients in    Patients with paralysis  Children rarely develop
  MDC-9 or patients with are included.            decubiti without
  any diagnosis of                                underlying high risk
  hemiplegia,                                     conditions.
  paraplegia, or                                  
  quadriplegia.                                   
  ---------------------- ------------------------ ------------------------

***Changes Implemented to Pediatric Indicator as a Result of Pediatric
Panel Review***

  ---------------------- ------------------------ ------------------------
  **Pre-panel            **Post-panel indicator   **Reason implemented**
  definition**           definition**             

  All patients,          Although not yet         These patients are at
  including high risk    implemented, panelists   high risk for developing
  patients, examined     requested that high risk decubiti and are cared
  together.              patients (paralysis,     for disproportionately
                         spina bifida, anoxic     by tertiary care
                         brain damage, mechanical facilities.
                         ventilation) be examined 
                         separately.              

  Neonates included.     Exclude neonates.        "Skin breakdown," common
                                                  terminology used for
                                                  neonates, is coded to a
                                                  separate and
                                                  non-specific code. This
                                                  complication cannot be
                                                  captured in this
                                                  population.

  Include patients       Exclude patients         Like patients
  transferred from       transferred from another transferred from a long
  another acute care     acute care facility.     term care facility,
  facility.                                       these patients are at
                                                  high risk for having
                                                  decubiti present on
                                                  admission.
  ---------------------- ------------------------ ------------------------

Changes considered, but not implemented

  ----------------------- ----------------------- -----------------------
  **AHRQ QI definition**  **Pediatric indicator   **Reason not
                          definition**            implemented**

  Exclude patients        Include patients        Although more rare for
  admitted from long term admitted from long term children to be admitted
  care facility           care facility           from a long term care
                                                  facility, these
                                                  patients are at higher
                                                  risk for having
                                                  decubiti present on
                                                  admission
  ----------------------- ----------------------- -----------------------

***Clinical rationale***

This indicator is intended to flag cases of in-hospital decubitus ulcers
(pressure sores). Common practice asserts that decubiti can be prevented
by frequent movement, close monitoring of at risk patients, and
specialized beds or bedding.

***Summary of AHRQ QI clinician panel reviews***

This indicator was reviewed during our development of the Patient Safety
Indicators, which included a clinical panel review. For this indicator
the panel consisted of seven physicians: two general surgeons, a
geriatrician, two adult hospitalists, an internist, and a nurse
specialist. The panel reviewed several other indicators. In the course
of review the panel suggested the following:

-   The panel modified several exclusion criteria that were based on the
    original Complications Screening Program indicator. Instead of
    excluding all very elderly patients because they may have
    pre-existing decubiti, panelists argued for the more limited
    exclusion of patients admitted from a long term care facility.
    Panelists also reduced the original length of stay requirement of 10
    days to 4 days.

-   Panelists noted that a few decubiti may not be preventable, and that
    charting will vary with the less severe decubiti.

-   Panelists noted that very ill patients are at higher risk for
    decubiti.

-   Panelists were interested in tracking decubiti in high risk
    patients, such as paralysis patients, and argued that these patients
    should be tracked separately.

***Results of pediatric clinician panel review***

As part of the current pediatric indicator development process, this
indicator was reviewed by a panel of ten pediatric clinicians, including
one neonatologist, one infectious disease specialist, one ambulatory
care pediatrician, one pediatric hospitalist, one pediatric
cardiovascular surgeon, one pediatric oncologist, two pediatric
surgeons, one pediatric interventional radiologist, and one pediatric
critical care physician. The panel reviewed several other indicators. In
the course of review the panel suggested the following:

-   The panel felt that the indicator was most useful when tracking high
    risk populations, including patients with hemiplegia, paraplegia,
    quadriplegia (e.g., due to cerebral palsy), spina bifida, muscular
    dystrophy or glycogen storage diseases and neurodevastation due to
    trauma. These patients are at high risk for developing ulcers due to
    neurologic impairments, and as a result may have ulcers present on
    admission. Despite the inability to easily distinguish ulcers
    present on admission, panelists felt they desired to have separates
    rates available for high risk and lower risk patients.

-   Panelists noted that "skin breakdown" or "decubiti" in newborns
    rarely stem from gravity related causes, but rather from friction
    from equipment and other processes. These sores are rarely
    identified as decubiti, but rather as skin breakdown and panelists
    felt these are likely to be coded differently. Coding consultation
    will help inform the inclusion of newborns in this indicator.

-   Given the need to accurately identify premature infants and light
    for gestational age infants as requested, additional definitional
    work is required and this indicator was not re-rated by panelists.
    It will be re-examined in the second round of pediatric indicator
    development.

The same panel participated in a second round of rating, which included
preliminary rating, followed by a conference call, and a final rating.
The panel was identical except for the attrition of three panelists
(pediatric cardiovascular surgeon, pediatric oncologist, pediatric
hospitalist). The panel re-reviewed three other indicators. In the
course of review the panel further suggested the following, in addition
to the comments from the previous review:

-   The panel agreed that the inclusion and stratification of high risk
    patients is useful. They suggested that if possible ICU patients be
    included in the high risk patient groups, since these patients may
    be at high risk of developing the complication in hospital. The
    original stratification was proposed to stratify those that are at
    high risk of having a decubitus present on admission. Patients in
    the ICU, unless chronically ill, however, are unlikely to have been
    admitted with decubiti.

**^Post-conference\ call\ panel\ ratings^**

  --------------------------- ------------- --------------------------------
  **Question**                **Median**    **Agreement status**

  Overall rating -- internal  7             Indeterminate agreement
  QI                                        

  Overall rating --           7             Indeterminate agreement
  comparative purposes                      

  Not present on admission    8             Indeterminate agreement

  Preventability              7             Indeterminate agreement

  Due to medical error        6             Indeterminate agreement

  Charting by physicians      7             Indeterminate agreement

  Lack of bias                4             Indeterminate agreement

  Final recommendation        Internal QI:  
                              Acceptable    
                              (-)           
                              Comparative   
                              purposes:     
                              Acceptable    
                              (-)           
  --------------------------- ------------- --------------------------------

***Empirical analyses to inform indicator definition***

Prior to panel review we examined the percentage of complications in
this indicator related to high risk conditions, including paralysis,
spina bifida, muscular dystrophy and glycogen storage diseases. Patients
with high risk conditions constituted 30% of the numerator. No patients
with glycogen storage disease were in the numerator.

The following empirical analyses were completed after the initial panel
review using the 2003 KIDs' Inpatient Database (KID).

First, we examined the effect of adding an exclusion for patients
transferred from an acute care facility, as suggested by the panelists.
The change decreased the overall rate by 12%.

Second, we examined the risk of this complication based on several
groups theorized to be higher risk. Based on the working definition at
the time we found that three disorders were associated with higher risk:
paralysis (RR = 12.3), spina bifida (RR = 22.8), and anoxic brain damage
(RR =6.3). Patients with muscular dystrophy were not at elevated risk.

Finally we examined patients with a procedure code for mechanical
ventilation. Current codes designate patients based on the duration of
ventilation. Patients with continuous mechanical ventilation for less
than 96 hours (96.71) were not at significantly higher risk for
decubitus ulcer = (RR 1.34). Few patients had a code denoting
"unspecified duration (code 96.70) (n=171) and none of those patients
also had a code for decubitus ulcer. In contrast, patients with
continuous ventilation for 96 hours or more had a significantly elevated
risk (RR = 6.68).

***Literature based evidence specific to pediatric population***

While children, on the whole, are more active and less chronically ill
than their adult counterparts, decubitus ulcers are of great concern to
those caring for critically ill infants and children. These skin
injuries represent a significant iatrogenic problem in pediatric health
care. It is known that interventions such as frequent turning,
repositioning, softer bedding surfaces (e.g. egg crate foam), and
elevating heels off bed surfaces can be used to lessen pressure-related
injuries.^13,\ 14^ Also, studies have attempted to modify existing adult
risk assessment tools to the pediatric population to help medical
personnel assess the risk for decubitus ulcers in their patients.^14-16^

Other groups have analyzed rates of this indicator using the publicly
available indicator definition applied to a pediatric population; this
definition differs slightly from the definition proposed above. This
indicator was applied to pediatric hospital populations (e.g., 7.67 per
1,000 discharges at 0-17 years, 4.95 at 18-44 years, 9.84 at 45-64
years, and 25.17 at 65 or more years).^10^ Other groups have analyzed
rates of this indicator using the publicly available indicator
definition applied to a pediatric population; this definition differs
slightly from the definition proposed above. Miller and colleagues
analyzed HCUP data from 2000 and found a significant incidence of
decubitus ulcers in pediatric patients 0-18 years of age (2.4 per
1,000).^11^ Sedman et al found observed rates varying from 4.1 per 1,000
in 1999 to 4.3 per 1,000 in 2001 in the NACHRI database (i.e., a slight
upward trend over time).^12^ Additionally, Miller & Zhan found that this
complication resulted in increased mean length of stay (by 18 days) and
\$85,344 in increased charges in affected patients, with 3.5 times
higher odds of in-hospital mortality (after adjusting for age, gender,
expected payer, up to 30 comorbidities, and multiple hospital
characteristics, including ownership, teaching status, nursing
expertise, urban location, bed size, pediatric volume, coding intensity,
ICU bed percentage, and surgical discharge percentage).^11^

+----------------------------------+-----------------------------------+
| ### FOREIGN BOD                  |                                   |
| Y LEFT IN DURING PROCEDURE (PSI) |                                   |
+----------------------------------+-----------------------------------+
| **Indicator definition:**        |                                   |
|                                  |                                   |
| Number of patients with a        |                                   |
| foreign body unintentionally     |                                   |
| left in during a procedure (see  |                                   |
| definition and exclusions below) |                                   |
| per 1,000 eligible admissions    |                                   |
| (population at risk). See The    |                                   |
| Pediatric Quality Indicator      |                                   |
| Technical Specifications.        |                                   |
+----------------------------------+-----------------------------------+
| **Definition of foreign body     | **Definition of population at     |
| left in during procedure**:      | risk:**                           |
|                                  |                                   |
|                                  | Patients eligible to be included  |
|                                  | in this indicator:                |
+----------------------------------+-----------------------------------+
| ***Secondary diagnosis code      | ***a. All medical and surgical    |
| for:***                          | patients (defined by DRG), age    |
|                                  | 0-17 years, except exclusions     |
| -   Foreign body accidentally    | (see below).***                   |
|     left during a procedure      |                                   |
|     \[998.4\]                    | ***b. Exclude patients with       |
|                                  | principal diagnosis code for      |
| -   Acute reactions to foreign   | foreign body left in during       |
|     substance accidentally left  | procedure.***                     |
|     during a procedure \[998.7\] |                                   |
|                                  | ***c. Exclude normal newborns     |
| Foreign body left in during:     | \[DRG 391\].***                   |
|                                  |                                   |
| -   Surgical operation           | ***d. Exclude newborns with a     |
|     \[E871.0\]                   | birth weight less than 500g.***   |
|                                  |                                   |
| -   Infusion or transfusion      | ***e. Exclude obstetric patients  |
|     \[E871.1\]                   | (MDC 14).***                      |
|                                  |                                   |
| -   Kidney dialysis or other     |                                   |
|     perfusion \[E871.2\]         |                                   |
|                                  |                                   |
| -   Injection or vaccination     |                                   |
|     \[E871.3\]                   |                                   |
|                                  |                                   |
| -   Endoscopic examination       |                                   |
|     \[E871.4\]                   |                                   |
|                                  |                                   |
| -   Aspiration of fluid or       |                                   |
|     tissue, puncture, and        |                                   |
|     catheterization \[E871.5\]   |                                   |
|                                  |                                   |
| -   Heart catheterization        |                                   |
|     \[E871.6\]                   |                                   |
|                                  |                                   |
| -   Removal of catheter or       |                                   |
|     packing \[E871.7\]           |                                   |
|                                  |                                   |
| -   Other specified procedures   |                                   |
|     \[E871.8\]                   |                                   |
|                                  |                                   |
| -   Unspecified procedure        |                                   |
|     \[E871.9\]                   |                                   |
+----------------------------------+-----------------------------------+

+-----------------------------------------------------------------------+
| **Rates based on year 2003 KIDs' Inpatient Database** (per 1000):     |
|                                                                       |
|   -----                                                               |
| ----------- ----------------- -------------------- ------------------ |
|   **OVERALL**      0.031                                              |
|                                                                       |
|   **Age stratified                                                    |
|   rates:**                                                            |
|                                                                       |
|   **Neonate, \<    0.036                                              |
|   2000g**                                                             |
|                                                                       |
|   **Neonate, ≥     0.003                                              |
|   2000g**                                                             |
|                                                                       |
|   **29 days -- 364 0.035                                              |
|   days**                                                              |
|                                                                       |
|   **1 -- 2 years** 0.029                                              |
|                                                                       |
|   **3 -- 5 years** 0.036                                              |
|                                                                       |
|   **6 -- 12        0.051                                              |
|   years**                                                             |
|                                                                       |
|   **13 -- 17       0.063                                              |
|   years**                                                             |
|                                                                       |
|                                                                       |
|                                                                       |
|   **Hospital                                                          |
|   type**                                                              |
|                                                                       |
|   **Children's**                     **Non-Children's**               |
|                                                                       |
|   **OVERALL**      0.067             **OVERALL**          0.013       |
|                                                                       |
|   **Age stratified                   **Age stratified                 |
|   rates:**                           rates:**                         |
|                                                                       |
|   **Neonate, \<    0.097             **Neonate, \<        0.017       |
|   2000g**                            2000g**                          |
|                                                                       |
|   **Neonate, ≥     0.011             **Neonate, ≥ 2000g** 0.002       |
|   2000g**                                                             |
|                                                                       |
|   **29 days -- 364 0.081             **29 days -- 364     0.000       |
|   days**                             days**                           |
|                                                                       |
|   **1 -- 2 years** 0.070             **1 -- 2 years**     0.006       |
|                                                                       |
|   **3 -- 5 years** 0.091             **3 -- 5 years**     0.000       |
|                                                                       |
|   **6 -- 12        0.072             **6 -- 12 years**    0.017       |
|   years**                                                             |
|                                                                       |
|   **13 -- 17       0.084             **13 -- 17 years**   0.058       |
|   years**                                                             |
|   -----                                                               |
| ----------- ----------------- -------------------- ------------------ |
+-----------------------------------------------------------------------+

***Status summary.*** Based on the current evidence base, from the
pediatric literature review pediatric panel review, and empirical
analyses, this indicator is recommended for inclusion in the pediatric
quality indicator set. Panelists rated the indicator favorably for use
both for internal quality improvement and comparative purposes.

***Changes from AHRQ QI Implemented Prior to Pediatric Panel Review***

  ---------------------- ------------------------ ------------------------
  **AHRQ QI definition** **Pediatric indicator    **Reason implemented**
                         definition**             

  All ages               Age 0 -- 17              Pediatric age range

  Newborns included.     Exclude normal newborns  Normal newborns rarely
                         \[DRG 391\].             undergo procedures that
                                                  place them at risk for
                                                  this complication.

  Premature neonates     Exclude newborns with a  Excluded from all
  included.              birth weight less than   indicators due to very
                         500g.                    high risk nature and
                                                  bias related to delivery
                                                  practices (i.e.
                                                  attempting delivery vs.
                                                  allowing fetal death).
  ---------------------- ------------------------ ------------------------

***Changes Implemented to Pediatric Indicator as a Result of Pediatric
Panel Review***

  ---------------------- ------------------------ ------------------------
  **Pre-panel            **Post-panel indicator   **Reason implemented**
  definition**           definition**             

  No additional changes.                          
  ---------------------- ------------------------ ------------------------

Changes considered, but not implemented

  ----------------------- ----------------------- -----------------------
  **AHRQ QI definition**  **Pediatric indicator   **Reason not
                          definition**            implemented**

  None.                                           
  ----------------------- ----------------------- -----------------------

***Clinical rationale***

This indicator is intended to flag cases of a foreign body accidentally
left in a patient's body during a procedure. It is based on an indicator
originally developed as part of the Complications Screening Program by
Lisa Iezzoni and colleagues. Interventions such as surgical instrument
counting and post-operative imaging have been implemented to reduce the
number of foreign bodies unintentionally left in during a procedure.

***Summary of AHRQ QI clinician panel reviews***

This indicator was reviewed twice during our development of the Patient
Safety Indicators, which included a clinical panel review. For this
indicator, the first panel (multispecialty) consisted of 6 clinicians: a
general surgeon, an internist, two adult hospitalists, and two
specialized nurses. The second (surgery specialist) panel consisted of 9
clinicians: a urologist, a transplant surgeon, two orthopedic surgeons,
a pediatric neurosurgeon, a neurosurgeon, and two colon and rectal
surgeons. Both panels reviewed several other indicators. In the course
of review the panels suggested or noted the following:

-   Suture granulomas requiring treatment are also detected by this
    indicator (because the retained suture is a foreign body). Panelists
    noted that these are substantially different than other foreign
    bodies, but did not feel this invalidated the indicator.

-   Panelists expressed concern that some foreign bodies are left in
    intentionally and may be coded due to lack of clear documentation by
    physicians. Also some foreign bodies do not cause substantial
    morbidity.

-   The patients included in the denominator may not actually undergo a
    procedure. Panelists felt that limiting the denominator to surgical
    patients would too severely reduce the sensitivity of this
    indicator, because foreign bodies may be left in during bedside
    procedures such as central line placement.

***Results of pediatric clinician panel review***

This indicator was also reviewed, during the current development process
by a panel of eleven pediatric clinicians, including one general
pediatrician, one pediatric hospitalist, one pediatric critical care
physician, one neonatologist, one pediatric infectious disease
specialist, one pediatric hematologist/oncologist, one pediatric
cardiothoracic surgeon, one pediatric emergency medicine specialist, on
pediatric interventional radiologist, and two pediatric surgeons. In the
course of review the panels suggested or noted the following:

-   Panelists agreed that many foreign bodies will not be discovered
    until after discharge or may result from outpatient surgery. In
    order to track these complications, an area level indicator will be
    developed for this indicator, which includes principal diagnoses for
    foreign body, and which utilizes a population denominator. The area
    level indicator is intended to capture transfers and readmissions
    for foreign body. It will be available in addition to this
    hospital-based indicator.

**^Post-conference\ call\ panel\ ratings^**

  --------------------------- ------------- --------------------------------
  **Question**                **Median**    **Agreement status**

  Overall rating -- internal  8             Agreement
  QI                                        

  Overall rating --           8             Agreement
  comparative purposes                      

  Not present on admission    8             Agreement

  Preventability              8             Agreement

  Due to medical error        7             Indeterminate agreement

  Charting by physicians      7             Agreement

  Lack of bias                7.5           Indeterminate agreement

  Final recommendation        Internal QI:  
                              Acceptable    
                              (+)           
                              Comparative   
                              purposes:     
                              Acceptable    
                              (+)           
  --------------------------- ------------- --------------------------------

***Literature based evidence specific to pediatric population***

Children, as adults, are at risk for having foreign bodies left in the
surgical field after a procedure. The incidence of this indicator, using
the publicly available definition was investigated in pediatric
populations (e.g., 0.07 per 1,000 discharges at 0-17 years, 0.07 at
18-44 years, 1.10 at 45-64 years, and 0.09 at 65 or more years).^10^
Miller and colleagues analyzed HCUP data from 1997, using a predecessor
of the AHRQ Patient Safety Indicators, and found a rate of 0.02 per
1,000 discharges in 1997.^17^ Other groups have analyzed rates of this
indicator using the publicly available indicator definition applied to a
pediatric population; this definition differs slightly from the
definition proposed above. Miller et al analyzed HCUP data from 2000,
and found a rate of 0.05 per 1,000 discharges in 2000, for foreign body
left in during procedure among 0-18 year old children.^11^ Sedman et al
found observed rates varying from 0.14 per 1,000 in 2000 to 0.10 per
1,000 in 1999 in the NACHRI database (without any consistent trend over
time).^12^ Additionally, Miller & Zhan found that this error resulted in
an increased mean length of stay (by 5.7 days) and an average of
\$31,366 in increased charges in affected patients, with no significant
effect on in-hospital mortality (after adjusting for age, gender,
expected payer, up to 30 comorbidities, and multiple hospital
characteristics, including ownership, teaching status, nursing
expertise, urban location, bed size, pediatric volume, coding intensity,
ICU bed percentage, and surgical discharge percentage).^11^

+----------------------------------+-----------------------------------+
| ### IATROGENIC PNEUMOTH          |                                   |
| ORAX (IN NEONATES AT RISK) (PSI) |                                   |
+----------------------------------+-----------------------------------+
| **Indicator definition:**        |                                   |
|                                  |                                   |
| Number of patients with an       |                                   |
| iatrogenic pneumothorax (see     |                                   |
| definition and exclusions below) |                                   |
| per 1,000 eligible admissions    |                                   |
| (population at risk). See The    |                                   |
| Pediatric Quality Indicator      |                                   |
| Technical Specifications.        |                                   |
+----------------------------------+-----------------------------------+
| **Definition of iatrogenic       | **Definition of population at     |
| pneumothorax**:                  | risk:**                           |
|                                  |                                   |
|                                  | Patients eligible to be included  |
|                                  | in this indicator:                |
+----------------------------------+-----------------------------------+
| ***Secondary diagnosis code      | ***a. All neonates (defined by    |
| for:***                          | DRG), with a birthweight 2500 g   |
|                                  | or less, except exclusions (see   |
| -   Iatrogenic pneumothorax      | below).***                        |
|     \[512.1\]                    |                                   |
|                                  | ***b. Exclude patients with       |
|                                  | principal diagnosis of iatrogenic |
|                                  | pneumothorax.***                  |
|                                  |                                   |
|                                  | ***c. Exclude patients with any   |
|                                  | diagnosis of chest trauma.***     |
|                                  |                                   |
|                                  | ***d. Exclude patients with any   |
|                                  | code indicating thoracic surgery  |
|                                  | or lung or pleural biopsy or      |
|                                  | assigned to cardiac surgery       |
|                                  | DRGs.***                          |
|                                  |                                   |
|                                  | ***e. Exclude normal newborns.*** |
|                                  |                                   |
|                                  | ***f. Stratify rates by           |
|                                  | birthweight (500 g                |
|                                  | increments).***                   |
|                                  |                                   |
|                                  | ***g. Exclude newborns with a     |
|                                  | birth weight less than 500g.***   |
|                                  |                                   |
|                                  | ***h. Exclude patients with any   |
|                                  | procedure code for diaphragmatic  |
|                                  | surgery.***                       |
|                                  |                                   |
|                                  | ***i. Exclude patients with any   |
|                                  | diagnosis of pleural effusion.*** |
+----------------------------------+-----------------------------------+
| **Rates based on year 2003 KIDs' |                                   |
| Inpatient Database** (per 1000): |                                   |
|                                  |                                   |
|   ---------------- ------- ----- |                                   |
| -- -------------------- -------- |                                   |
|   **OVERALL**                    |                                   |
|    0.372                         |                                   |
|                                  |                                   |
|   **Weight                       |                                   |
|                                  |                                   |
|   stratified                     |                                   |
|                                  |                                   |
|   rates:**                       |                                   |
|                                  |                                   |
|                                  |                                   |
|   **500 -- 999 g**               |                                   |
|    2.084                         |                                   |
|                                  |                                   |
|   **1000 -- 1499                 |                                   |
|    0.447                         |                                   |
|   g**                            |                                   |
|                                  |                                   |
|                                  |                                   |
|   **1500 -- 1999                 |                                   |
|    0.234                         |                                   |
|   g**                            |                                   |
|                                  |                                   |
|                                  |                                   |
|   **2000 -- 2499                 |                                   |
|    0.081                         |                                   |
|   g**                            |                                   |
|                                  |                                   |
|                                  |                                   |
|                                  |                                   |
|                                  |                                   |
|                                  |                                   |
|   **Hospital                     |                                   |
|                                  |                                   |
|   type**                         |                                   |
|                                  |                                   |
|                                  |                                   |
|   **Children's**                 |                                   |
|            **Non-Children's**    |                                   |
|                                  |                                   |
|   **OVERALL**      0.675         |                                   |
|       **OVERALL**          0.296 |                                   |
|                                  |                                   |
|   **Age stratified               |                                   |
|            **Age stratified      |                                   |
|   rates:**                       |                                   |
|            rates:**              |                                   |
|                                  |                                   |
|   **500 -- 999 g** 3.321         |                                   |
|       **500 -- 999 g**     1.457 |                                   |
|                                  |                                   |
|   **1000 -- 1499   0.505         |                                   |
|       **1000 -- 1499 g**   0.464 |                                   |
|   g**                            |                                   |
|                                  |                                   |
|                                  |                                   |
|   **1500 -- 1999   0.230         |                                   |
|       **1500 -- 1999 g**   0.254 |                                   |
|   g**                            |                                   |
|                                  |                                   |
|                                  |                                   |
|   **2000 -- 2499   0.080         |                                   |
|       **2000 -- 2499 g**   0.087 |                                   |
|   g**                            |                                   |
|                                  |                                   |
|   ---------------- ------- ----- |                                   |
| -- -------------------- -------- |                                   |
+----------------------------------+-----------------------------------+

***Status summary.*** Based on the current evidence base, from the
pediatric literature review pediatric panel review, and empirical
analyses, this indicator is recommended for inclusion in the pediatric
quality indicator set. Panelists rated the indicator favorably and with
agreement for use in internal quality improvement and favorably with
indeterminate agreement for comparative purposes.

***Changes from AHRQ QI Implemented Prior to Pediatric Panel Review***

***NOTE: The pre-panel definition combined neonatal and non-neonatal
patients***

  ---------------------- ------------------------ ------------------------
  **AHRQ QI definition** **Pediatric indicator    **Reason implemented**
                         definition**             

  All ages               Age 0 -- 17              Pediatric age range

  Premature neonates     Exclude newborns with a  Excluded from all
  included.              birth weight less than   indicators due to very
                         500g.                    high risk nature and
                                                  bias related to delivery
                                                  practices (i.e.
                                                  attempting delivery vs.
                                                  allowing fetal death).

  No stratification.     Stratify rates by low    Risk for pneumothorax
                         birth weight neonate     increases dramatically
                         (500g increments) and    with lower birth weight.
                         other patients.          
  ---------------------- ------------------------ ------------------------

***Changes Implemented to Pediatric Indicator as a Result of Pediatric
Panel Review***

  ---------------------- ------------------------ ------------------------
  **Pre-panel            **Post-panel indicator   **Reason implemented**
  definition**           definition**             

  One indicator included Two indicators were      It is not possible to
  both barotrauma and    created -- one for high  separate barotrauma from
  procedural caused      risk neonates            procedural caused
  pneumothoraces.        (birthweight less than   pneumothoraces. Since
                         2500 g) and one for      premature infants are at
                         other patients.          higher risk for
                                                  barotrauma, panelists
                                                  suggested they be
                                                  examined in a separate
                                                  indicator.

  All trauma patients    Chest trauma patients    Only chest trauma
  excluded.              excluded.                patients are at elevated
                                                  risk for traumatic
                                                  pneumothoraces.

  Include patients with  Exclude discharges with  Pneumothorax is an
  any procedure code of  any procedure code of    expected complication
  diaphragm surgery      diaphragm surgery        for these patients.

  Include patients with  Exclude patients with    These patients almost
  pleural effusion.      pleural effusion.        always receive chest
                                                  tubes to drain the
                                                  effusion and
                                                  pneumothorax is expected
                                                  following removal.
                                                  Although such an
                                                  expected complication is
                                                  not technically a
                                                  codable complication, it
                                                  is "cleaner" to remove
                                                  these patients.
  ---------------------- ------------------------ ------------------------

Changes considered, but not implemented

  ----------------------- ----------------------- -----------------------
  **AHRQ QI definition**  **Pediatric indicator   **Reason not
                          definition**            implemented**

  None.                                           
  ----------------------- ----------------------- -----------------------

***Clinical rationale***

This indicator is intended to flag cases of pneumothorax caused by
medical care in high risk neonates. Premature neonates are at higher
risk of developing barotrauma due to ventilation. Close monitoring of
ventilation and pressures decreases the risk of pneumothorax. These
patients may also sustain pneumothoraces secondary to procedures. Good
technique may reduce the rate of these pneumothoraces.

***Summary of AHRQ QI clinician panel reviews***

This indicator was reviewed during our development of the Patient Safety
Indicators, which included a clinical panel review. For this indicator
the panel consisted of 8 physicians: an internist and
gastroenterologist, a general surgeon, a cardiologist and critical care
physician, two interventional radiologists, two specialized nurses, and
an anesthesiologist. The panel reviewed several other indicators. In the
course of review the panel suggested the following:

-   The exclusion (currently implemented) of patients undergoing a
    procedure that involves entering the lung parenchyma or opening the
    pleural space (because incidental pneumothoraces are anticipated
    after these procedures).

-   Restriction to patients receiving a central line, Swan-Ganz catheter
    or thoracentesis (because these are the patients for whom iatrogenic
    pneumothoraces are most likely to be preventable). However,
    empirical analyses revealed that these procedures were not reliably
    identified using administrative data, and this recommendation could
    not be implemented.

-   Identification of central line placement approach, since
    pneumothoraces may be reduced by using specific approaches (e.g.,
    internal jugular instead of subclavian), while increasing other
    potentially serious complications. Because the placement approach is
    not designated in ICD-9-CM, this recommendation could not be
    implemented.

-   The exclusion or stratification of pneumothoraces with barotrauma.
    Because it is not possible to identify the cause of pneumothoraces
    using administrative data, this recommendation could not be
    implemented.

***Results of pediatric clinician panel review***

This indicator was also reviewed, during the current development process
by a panel of eleven pediatric clinicians, including one general
pediatrician, one pediatric hospitalist, one pediatric critical care
physician, one neonatologist, one pediatric infectious disease
specialist, one pediatric hematologist/oncologist, one pediatric
cardiothoracic surgeon, one pediatric emergency medicine specialist, one
pediatric interventional radiologist, and two pediatric surgeons. In the
course of review the panels suggested or noted the following:

-   At the onset of the review, this indicator included both neonates
    and other pediatric patients. Panelists, like the previous panel,
    argued for the stratification by cause of pneumothoraces (i.e.
    barotrauma vs. procedure related). Again, cause is not discernible
    using the data. In order to better analyze the data, the panel split
    the indicator into two separate indicators: 1.) iatrogenic
    pneumothorax (neonates), and 2.) iatrogenic pneumothorax
    (non-neonates). The first indicator, presented here, examines
    iatrogenic pneumothorax in neonates under 2500g, as a group that is
    at particularly elevated risk for pneumothorax due to barotrauma, in
    addition to line-related pneumothorax. This indicator is limited to
    neonates with a recorded birthweight of less than 2500 g. This
    indicator is stratified by birthweight groups in 500 gram
    increments.

-   Panelists argued for the narrowing of the previous exclusion of all
    trauma patients to include only chest trauma, as panelists expressed
    that only chest trauma patients are truly at higher risk for
    pneumothorax. This exclusion is unlikely to affect the neonatal
    version of this indicator.

-   An exclusion for patients undergoing diaphragmatic surgery was
    added, as these patients may incur a pneumothorax as an expected
    complication.

**^Post-conference\ call\ panel\ ratings^**

  --------------------------- ------------- --------------------------------
  **Question**                **Median**    **Agreement status**

  Overall rating -- internal  8             Agreement
  QI                                        

  Overall rating --           7             Indeterminate agreement
  comparative purposes                      

  Not present on admission    8             Agreement

  Preventability              5             Agreement

  Due to medical error        3             Indeterminate agreement

  Charting by physicians      8             Agreement

  Lack of bias                4             Indeterminate agreement

  Final recommendation        Internal QI:  
                              Acceptable    
                              (+)           
                              Comparative   
                              purposes:     
                              Acceptable    
                              (-)           
  --------------------------- ------------- --------------------------------

***Literature based evidence specific to pediatric population***

In children procedures like central line placement, thoracentesis, or
Swan-Ganz catheter placement can be technically more complex than in
older patients, due to their smaller anatomy (though they are more
likely to be performed in a monitored setting). Also, in comparison to
adults, iatrogenic pneumothoraces in neonates are primarily due to
barotrauma, with the very smallest infants being at greatest risk (as
shown by our preliminary empirical analyses). In an older pediatric
population, while barotrauma can occur, the risks for iatrogenic
pneumothoraces are more clinically similar to an adult population (e.g.
at risk while receiving a central line, catheter, or undergoing
thoracentesis procedures).

Important interventions are available which have been shown to decrease
the incidence of barotrauma and pneumothoraces in the low birth weight
neonate population. For example, timely administration of antenatal
steroids, use of prophylactic surfactant, and appropriate resuscitation
and ventilation of the smallest infants (\<30 weeks gestational age)
have all been shown to reduce the risk of iatrogenic pneumothoraces in
these patients.^18-21^ While low birth weight infants are also at risk
for pneumothoraces when undergoing medical procedures, the prevention of
pneumothoraces in this population is more focused on preventing injury
to an immature lung during ventilation.

Using 1997 HCUP data, the National Healthcare Quality Report, cited
rates of iatrogenic pneumothoraces in the pediatric population (\< 19
years). These analyses showed that this patient safety event occurred
frequently and at rates comparable to those in adults (e.g., 0.48 per
1,000 discharges at 0-17 years, 0.42 at 18-44 years, 0.43 at 45-64
years, and 0.74 at 65 or more years).^10^ Other groups have analyzed
rates of this indicator using the publicly available indicator
definition applied to a pediatric population; this definition differs
slightly from the definition proposed above. In 2000, Miller et al found
iatrogenic pneumothoraces occurred at a rate of 0.3 per 1,000 discharges
among 0-18 year old children.^11^ Also, iatrogenic pneumothorax was
found to result in, on average, 11.6 days increased length of stay,
\$61,991 increased charges, and 7.5 times higher odds of in-hospital
mortality (after adjusting for age, gender, expected payer, up to 30
comorbidities, and multiple hospital characteristics, including
ownership, teaching status, nursing expertise, urban location, bed size,
pediatric volume, coding intensity, ICU bed percentage, and surgical
discharge percentage).^11^ An analysis of NACHRI data from 1999 to 2002
showed a range of rates (risk adjusted) from 0.74 per 1,000 discharges
in 2002 to 0.82 per 1,000 discharges in 1999 (i.e., a slight downward
trend over time).^12^

+----------------------------------+-----------------------------------+
| ### IATROGENIC PN                |                                   |
| EUMOTHORAX IN NON-NEONATES (PSI) |                                   |
+----------------------------------+-----------------------------------+
| **Indicator definition:**        |                                   |
|                                  |                                   |
| Number of patients with an       |                                   |
| iatrogenic pneumothorax (see     |                                   |
| definition and exclusions below) |                                   |
| per 1,000 eligible admissions    |                                   |
| (population at risk). See The    |                                   |
| Pediatric Quality Indicator      |                                   |
| Technical Specifications.        |                                   |
+----------------------------------+-----------------------------------+
| **Definition of iatrogenic       | **Definition of population at     |
| pneumothorax**:                  | risk:**                           |
|                                  |                                   |
|                                  | Patients eligible to be included  |
|                                  | in this indicator:                |
+----------------------------------+-----------------------------------+
| ***Secondary diagnosis code      | ***a. All medical and surgical    |
| for:***                          | patients (defined by DRG), age    |
|                                  | 0-17 years, except exclusions     |
| -   Iatrogenic pneumothorax      | (see below).***                   |
|     \[512.1\]                    |                                   |
|                                  | ***b. Exclude patients with       |
|                                  | principal diagnosis of iatrogenic |
|                                  | pneumothorax.***                  |
|                                  |                                   |
|                                  | ***c. Exclude patients with any   |
|                                  | diagnosis of chest trauma.***     |
|                                  |                                   |
|                                  | ***d. Exclude patients with any   |
|                                  | code indicating thoracic surgery  |
|                                  | or lung or pleural biopsy or      |
|                                  | assigned to cardiac surgery       |
|                                  | DRGs.***                          |
|                                  |                                   |
|                                  | ***e. Exclude normal newborns and |
|                                  | newborns with a birthweight of    |
|                                  | 2500 g or less.***                |
|                                  |                                   |
|                                  | ***h. Exclude all obstetric       |
|                                  | discharges (MDC 14).***           |
|                                  |                                   |
|                                  | ***i. Exclude patients with any   |
|                                  | procedure code for diaphragmatic  |
|                                  | surgery.***                       |
|                                  |                                   |
|                                  | ***j. Exclude patients with any   |
|                                  | diagnosis of pleural effusion.*** |
+----------------------------------+-----------------------------------+

+-----------------------------------------------------------------------+
| **Rates based on year 2003 KIDs' Inpatient Database** (per 1000):     |
|                                                                       |
|   -----                                                               |
| ----------- ----------------- -------------------- ------------------ |
|   **OVERALL**      0.213                                              |
|                                                                       |
|   **Age stratified                                                    |
|   rates:**                                                            |
|                                                                       |
|   **Neonate**      0.105                                              |
|                                                                       |
|                                                                       |
|                                                                       |
|   **29 days -- 364 0.246                                              |
|   days**                                                              |
|                                                                       |
|   **1 -- 2 years** 0.186                                              |
|                                                                       |
|   **3 -- 5 years** 0.222                                              |
|                                                                       |
|   **6 -- 12        0.202                                              |
|   years**                                                             |
|                                                                       |
|   **13 -- 17       0.417                                              |
|   years**                                                             |
|                                                                       |
|                                                                       |
|                                                                       |
|   **Hospital                                                          |
|   type**                                                              |
|                                                                       |
|   **Children's**                     **Non-Children's**               |
|                                                                       |
|   **OVERALL**      0.439             **OVERALL**          0.108       |
|                                                                       |
|   **Age stratified                   **Age stratified                 |
|   rates:**                           rates:**                         |
|                                                                       |
|   **Neonate**      0.234             **Neonate**          0.080       |
|                                                                       |
|                                                                       |
|                                                                       |
|   **29 days -- 364 0.452             **29 days -- 364     0.117       |
|   days**                             days**                           |
|                                                                       |
|   **1 -- 2 years** 0.355             **1 -- 2 years**     0.090       |
|                                                                       |
|   **3 -- 5 years** 0.354             **3 -- 5 years**     0.137       |
|                                                                       |
|   **6 -- 12        0.363             **6 -- 12 years**    0.092       |
|   years**                                                             |
|                                                                       |
|   **13 -- 17       0.860             **13 -- 17 years**   0.182       |
|   years**                                                             |
|   -----                                                               |
| ----------- ----------------- -------------------- ------------------ |
+-----------------------------------------------------------------------+

***Status summary.*** Based on the current evidence base, from the
pediatric literature review pediatric panel review, and empirical
analyses, this indicator is recommended for inclusion in the pediatric
quality indicator set. Panelists rated the indicator favorably with
agreement for use in internal quality improvement and favorably with
indeterminate agreement for comparative purposes.

***Changes from AHRQ QI Implemented Prior to Pediatric Panel Review***

***NOTE: The pre-panel definition combined neonatal and non-neonatal
patients***

  ---------------------- ------------------------ ------------------------
  **AHRQ QI definition** **Pediatric indicator    **Reason implemented**
                         definition**             

  All ages               Age 0 -- 17              Pediatric age range

  Premature neonates     Exclude newborns with a  Excluded from all
  included.              birth weight less than   indicators due to very
                         500g.                    high risk nature and
                                                  bias related to delivery
                                                  practices (i.e.
                                                  attempting delivery vs.
                                                  allowing fetal death).

  No stratification.     Stratify rates by low    Risk for pneumothorax
                         birth weight neonate     increases dramatically
                         (500g increments) and    with lower birth weight.
                         other patients.          
  ---------------------- ------------------------ ------------------------

***Changes Implemented to Pediatric Indicator as a Result of Pediatric
Panel Review***

  ---------------------- ------------------------ ------------------------
  **Pre-panel            **Post-panel indicator   **Reason implemented**
  definition**           definition**             

  One indicator included Two indicators were      It is not possible to
  both barotrauma and    created -- one for high  separate barotrauma from
  procedural caused      risk neonates            procedural caused
  pneumothoraces.        (birthweight less than   pneumothoraces. Since
                         2500 g) and one for      premature infants are at
                         other patients.          higher risk for
                                                  barotrauma, panelists
                                                  suggested they be
                                                  examined in a separate
                                                  indicator.

  All trauma patients    Chest trauma patients    Only chest trauma
  excluded.              excluded.                patients are at elevated
                                                  risk for traumatic
                                                  pneumothoraces.

  Include patients with  Exclude discharges with  Pneumothorax is an
  any procedure code of  any procedure code of    expected complication
  diaphragm surgery      diaphragm surgery        for these patients.

  Include patients with  Exclude patients with    These patients almost
  pleural effusion.      pleural effusion.        always receive chest
                                                  tubes to drain the
                                                  effusion and
                                                  pneumothorax is expected
                                                  following removal.
                                                  Although such an
                                                  expected complication is
                                                  not technically a
                                                  codable complication, it
                                                  is "cleaner" to remove
                                                  these patients.
  ---------------------- ------------------------ ------------------------

Changes considered, but not implemented

  ----------------------- ----------------------- -----------------------
  **AHRQ QI definition**  **Pediatric indicator   **Reason not
                          definition**            implemented**

  None.                                           
  ----------------------- ----------------------- -----------------------

***Clinical rationale***

This indicator is intended to flag cases of pneumothorax caused by
medical care, which is sustained following a procedure or due to
barotrauma. Good technique when performing vascular access or
thorocentesis may reduce the risk of this complication. For patients on
ventilators, monitoring of pressures may also reduce the risk of this
complication.

***Summary of AHRQ QI clinician panel reviews***

This indicator was reviewed during our development of the Patient Safety
Indicators, which included a clinical panel review. For this indicator
the panel consisted of 8 physicians: an internist and
gastroenterologist, a general surgeon, a cardiologist and critical care
physician, two interventional radiologists, two specialized nurses, and
an anesthesiologist. The panel reviewed several other indicators. In the
course of review the panel suggested the following:

-   The exclusion (currently implemented) of patients undergoing a
    procedure that involves entering the lung parenchyma or opening the
    pleural space (because incidental pneumothoraces are anticipated
    after these procedures).

-   Restriction to patients receiving a central line, Swan-Ganz catheter
    or thoracentesis (because these are the patients for whom iatrogenic
    pneumothoraces are most likely to be preventable). However,
    empirical analyses revealed that these procedures were not reliably
    identified using administrative data, and this recommendation could
    not be implemented.

-   Identification of central line placement approach, since
    pneumothoraces may be reduced by using specific approaches (e.g.,
    internal jugular instead of subclavian), while increasing other
    potentially serious complications. Because the placement approach is
    not designated in ICD-9-CM, this recommendation could not be
    implemented.

-   The exclusion or stratification of pneumothoraces with barotrauma.
    Because it is not possible to identify the cause of pneumothoraces
    using administrative data, this recommendation could not be
    implemented.

***Results of pediatric clinician panel review***

This indicator was also reviewed, during the current development process
by a panel of eleven pediatric clinicians, including one general
pediatrician, one pediatric hospitalist, one pediatric critical care
physician, one neonatologist, one pediatric infectious disease
specialist, one pediatric hematologist/oncologist, one pediatric
cardiothoracic surgeon, one pediatric emergency medicine specialist, one
pediatric interventional radiologist, and two pediatric surgeons. In the
course of review the panels suggested or noted the following:

-   At the onset of the review, this indicator included both neonates
    and other pediatric patients. Panelists, like the previous panel,
    argued for the stratification by cause of pneumothoraces (i.e.
    barotrauma vs. procedure related). Again, cause is not discernible
    using the data. In order to better analyze the data, the panel split
    the indicator into two separate indicators: 1.) iatrogenic
    pneumothorax (neonates), and 2.) iatrogenic pneumothorax
    (non-neonates). The second indicator, which is presented here,
    examines all other pediatric patients, using an exclusion of both
    normal newborns and neonates with a recorded birthweight of less
    than 2500 grams.

-   Panelists argued for the narrowing of the previous exclusion of all
    trauma patients to include only chest trauma, as panelists expressed
    that only chest trauma patients are truly at higher risk for
    pneumothorax.

-   An exclusion for patients undergoing diaphragmatic surgery was
    added, as these patients may incur a pneumothorax as an expected
    complication.

**^Post-conference\ call\ panel\ ratings^**

  --------------------------- ------------- --------------------------------
  **Question**                **Median**    **Agreement status**

  Overall rating -- internal  8             Agreement
  QI                                        

  Overall rating --           7             Indeterminate agreement
  comparative purposes                      

  Not present on admission    7.5           Agreement

  Preventability              7             Indeterminate agreement

  Due to medical error        6             Indeterminate agreement

  Charting by physicians      7.5           Agreement

  Lack of bias                4             Indeterminate agreement

  Final recommendation        Internal QI:  
                              Acceptable    
                              (+)           
                              Comparative   
                              purposes:     
                              Acceptable    
                              (-)           
  --------------------------- ------------- --------------------------------

***Empirical analyses to inform indicator definition***

The following empirical analyses were completed after the initial panel
review using the 2003 KIDs' Inpatient Database (KID).

We examined codes for pleural effusion, since patients with pleural
effusion receive chest tubes and pneumothorax often follows the removal
of chest tubes. While these, when not requiring further treatment, are
not codable complications, we wished to see whether these patients
appeared in this indicator. We found that the rate of iatrogenic
pneumothorax in these patients to be very high (13.46 -- 45.52 per 1000
patients with pleural effusion compared to 0.21 per 1000 patients
overall). We found that a little over 20% of numerator patients had a
diagnosis code of pleural effusion.

***Literature based evidence specific to pediatric population \[Same as
previous iatrogenic pneumathorax indicator\]***

In children procedures like central line placement, thoracentesis, or
Swan-Ganz catheter placement can be technically more complex than in
older patients, due to their smaller anatomy (though they are more
likely to be performed in a monitored setting). Also, in comparison to
adults, iatrogenic pneumothoraces in neonates are primarily due to
barotrauma, with the very smallest infants being at greatest risk (as
shown by our preliminary empirical analyses). In an older pediatric
population, while barotrauma can occur, the risks for iatrogenic
pneumothoraces are more clinically similar to an adult population (e.g.
at risk while receiving a central line, catheter, or undergoing
thoracentesis procedures).

Important interventions are available which have been shown to decrease
the incidence of barotrauma and pneumothoraces in the low birth weight
neonate population. For example, timely administration of antenatal
steroids, use of prophylactic surfactant, and appropriate resuscitation
and ventilation of the smallest infants (\<30 weeks gestational age)
have all been shown to reduce the risk of iatrogenic pneumothoraces in
these patients.^18-21^ While low birth weight infants are also at risk
for pneumothoraces when undergoing medical procedures, the prevention of
pneumothoraces in this population is more focused on preventing injury
to an immature lung during ventilation.

Using 1997 HCUP data, the National Healthcare Quality Report, cited
rates of iatrogenic pneumothoraces in the pediatric population (\< 19
years). These analyses showed that this patient safety event occurred
frequently and at rates comparable to those in adults (e.g., 0.48 per
1,000 discharges at 0-17 years, 0.42 at 18-44 years, 0.43 at 45-64
years, and 0.74 at 65 or more years).^10^ Other groups have analyzed
rates of this indicator using the publicly available indicator
definition applied to a pediatric population; this definition differs
slightly from the definition proposed above. In 2000, Miller et al found
iatrogenic pneumothoraces occurred at a rate of 0.3 per 1,000 discharges
among 0-18 year old children.^11^ Also, iatrogenic pneumothorax was
found to result in, on average, 11.6 days increased length of stay,
\$61,991 increased charges, and 7.5 times higher odds of in-hospital
mortality (after adjusting for age, gender, expected payer, up to 30
comorbidities, and multiple hospital characteristics, including
ownership, teaching status, nursing expertise, urban location, bed size,
pediatric volume, coding intensity, ICU bed percentage, and surgical
discharge percentage).^11^ An analysis of NACHRI data from 1999 to 2002
showed a range of rates (risk adjusted) from 0.74 per 1,000 discharges
in 2002 to 0.82 per 1,000 discharges in 1999 (i.e., a slight downward
trend over time).^12^

+----------------------------------+-----------------------------------+
| ### POSTOPERATIVE PHYSIOLOGIC    |                                   |
|  AND METABOLIC DERANGEMENT (PSI) |                                   |
+----------------------------------+-----------------------------------+
| **Indicator definition:**        |                                   |
|                                  |                                   |
| Number of patients with          |                                   |
| physiologic and metabolic        |                                   |
| derangements (see definition and |                                   |
| exclusions below) per 1000       |                                   |
| eligible admissions (population  |                                   |
| at risk). See The Pediatric      |                                   |
| Quality Indicator Technical      |                                   |
| Specifications.                  |                                   |
+----------------------------------+-----------------------------------+
| **Definition of physiologic and  | **Definition of population at     |
| metabolic derangements**:        | risk:**                           |
|                                  |                                   |
|                                  | Patients eligible to be included  |
|                                  | in this indicator:                |
+----------------------------------+-----------------------------------+
| ***Secondary diagnosis code      | ***a. All elective surgical       |
| for:***                          | patients (defined by DRG and      |
|                                  | admission type), age 0-17 years,  |
| -   Diabetes with ketoacidosis   | except exclusions (see below).*** |
|     (type I and type II)         |                                   |
|     \[250.10 -- 250.13\]         | ***b. Exclude patients with       |
|                                  | principal diagnosis code for      |
| -   Diabetes with                | physiologic and metabolic         |
|     hyperosmolarity (type I and  | derangements and patients where a |
|     type II) \[250.20 --         | procedure for dialysis occurs     |
|     250.23\]                     | before or on the same day as the  |
|                                  | first operating room              |
| -   Diabetes with other coma     | procedure.***                     |
|     (type I and type II)         |                                   |
|     \[250.30 -- 250.33\]         | ***c. Exclude patients with both  |
|                                  | a diagnosis code of ketoacidosis, |
| -   Acute renal failure \[584.5  | hyperosmolarity, or other coma    |
|     -- 584.9\]                   | (subgroups of physiologic and     |
|                                  | metabolic derangements coding)    |
| Codes for acute renal failure    | AND a principal diagnosis of      |
| must be accompanied with a       | diabetes.***                      |
| procedure code for dialysis.     |                                   |
|                                  | ***d. Exclude patients with both  |
|                                  | a secondary diagnosis code for    |
|                                  | acute renal failure (subgroup of  |
|                                  | physiologic and metabolic         |
|                                  | derangements coding) AND a        |
|                                  | principal diagnosis of AMI,       |
|                                  | cardiac arrhythmia, cardiac       |
|                                  | arrest, shock, hemorrhage or      |
|                                  | gastrointestinal hemorrhage.***   |
|                                  |                                   |
|                                  | ***e. Exclude obstetric patients  |
|                                  | (MDC 14)***                       |
|                                  |                                   |
|                                  | ***f. Exclude newborns with a     |
|                                  | birth weight less than 500g.***   |
|                                  |                                   |
|                                  | ***g. Exclude patients with any   |
|                                  | diagnosis for cancer.***          |
+----------------------------------+-----------------------------------+

+-----------------------------------------------------------------------+
| **Rates based on year 2003 KIDs' Inpatient Database (per 1000):**     |
|                                                                       |
|   -----                                                               |
| ----------- ----------------- -------------------- ------------------ |
|   **OVERALL**                                                         |
|                                                                       |
|   **Age stratified                                                    |
|   rates:**                                                            |
|                                                                       |
|   **Neonate, \<                                                       |
|   2000g**                                                             |
|                                                                       |
|   **Neonate, ≥                                                        |
|   2000g**                                                             |
|                                                                       |
|   **29 days -- 364                                                    |
|   days**                                                              |
|                                                                       |
|   **1 -- 2 years**                                                    |
|                                                                       |
|   **3 -- 5 years**                                                    |
|                                                                       |
|   **6 -- 12                                                           |
|   years**                                                             |
|                                                                       |
|   **13 -- 17                                                          |
|   years**                                                             |
|                                                                       |
|                                                                       |
|                                                                       |
|   **Hospital                                                          |
|   type**                                                              |
|                                                                       |
|   **Children's**                     **Non-Children's**               |
|                                                                       |
|   **OVERALL**                        **OVERALL**                      |
|                                                                       |
|   **Age stratified                   **Age stratified                 |
|   rates:**                           rates:**                         |
|                                                                       |
|   **Neonate, \<                      **Neonate, \<                    |
|   2000g**                            2000g**                          |
|                                                                       |
|   **Neonate, ≥                       **Neonate, ≥ 2000g**             |
|   2000g**                                                             |
|                                                                       |
|   **29 days -- 364                   **29 days -- 364                 |
|   days**                             days**                           |
|                                                                       |
|   **1 -- 2 years**                   **1 -- 2 years**                 |
|                                                                       |
|   **3 -- 5 years**                   **3 -- 5 years**                 |
|                                                                       |
|   **6 -- 12                          **6 -- 12 years**                |
|   years**                                                             |
|                                                                       |
|   **13 -- 17                         **13 -- 17 years**               |
|   years**                                                             |
|   -----                                                               |
| ----------- ----------------- -------------------- ------------------ |
+-----------------------------------------------------------------------+

***Status summary.*** Based on the current evidence base from the
pediatric literature review, and pediatric panel review, this indicator
is not recommended for inclusion in the pediatric quality indicator set.
See summary of evidence below for justification.

***Changes from AHRQ QI Implemented Prior to Pediatric Panel Review***

  ---------------------- ------------------------ ------------------------
  **AHRQ QI definition** **Pediatric indicator    **Reason implemented**
                         definition**             

  All ages               Age 0 -- 17              Pediatric age range

  Premature neonates     Exclude newborns with a  Excluded from all
  included.              birth weight less than   indicators due to very
                         500g.                    high risk nature and
                                                  bias related to delivery
                                                  practices (i.e.
                                                  attempting delivery vs.
                                                  allowing fetal death).
  ---------------------- ------------------------ ------------------------

***Changes Implemented to Pediatric Indicator as a Result of Pediatric
Panel Review***

  ---------------------- ------------------------ ------------------------
  **Pre-panel            **Post-panel indicator   **Reason implemented**
  definition**           definition**             

  Cancer patients        Cancer patients          Cancer patients are at
  included.              excluded.                higher risk for these
                                                  complications, as a
                                                  result of tumor lysis
                                                  syndrome or
                                                  chemotherapy. However,
                                                  their preventability is
                                                  in question,.
  ---------------------- ------------------------ ------------------------

Changes considered, but not implemented

  ----------------------- ----------------------- -----------------------
  **AHRQ QI definition**  **Pediatric indicator   **Reason not
                          definition**            implemented**

  Per panel               Other derangements,     Distinguishing between
  recommendation, only    including hypokalemia,  clinically significant
  acute renal failure and hyper or hyponatremia,  complications and minor
  diabetic complications  or hyper or             derangements is
  included.               hypocalcemia            difficult.
                          considered.             Distinguishing between
                                                  derangements present on
                                                  admission and
                                                  complications is also
                                                  difficult.

  Adults have higher      Because of very low     More clinician feedback
  rates of diabetes and   rate, consider limiting and investigation
  these complications     denominator to cardiac  regarding validity of
  than children.          patients only.          suggestion needed
                                                  before implementation.
  ----------------------- ----------------------- -----------------------

***Clinical rationale***

This indicator is intended to flag cases of selected postoperative
metabolic or physiologic complications, specifically acute renal failure
and diabetes related complications. The population at risk is limited to
elective surgical patients, as patients undergoing non-elective surgery
may develop less preventable derangements or may have these derangements
present at admission. High quality care may reduce the rate of this
complication.

***Summary of AHRQ QI clinician panel reviews***

This indicator was reviewed twice during our development of the Patient
Safety Indicators, which included a clinical panel review. For this
indicator the first panel (multispecialty) consisted of 5 clinicians: a
critical care physician, an adult hospitalist, two specialized nurses,
and an anesthesiologist. The second (surgery specialist) panel consisted
of 6 clinicians: a spine surgeon, a pediatric neurosurgeon, a transplant
surgeon, a female urologist, a colon and rectal surgeon, and an
orthopedic surgeon. Both panels reviewed several other indicators. In
the course of review the panels advocated for the following:

-   The multispecialty panel suggested that in addition to the diabetic
    complications, hyponatremia should also be included.

-   Both panels considered and rejected a code for post-operative shock,
    due to the non-specific nature of this condition.

-   Both panels argued for the restriction of this indicator to elective
    surgery patients.

-   Both panels noted that some conditions may be variably coded and of
    varied clinical significance, leading the second panel to reject
    hyponatremia, oliguria and anuria, and restricting acute renal
    failure to cases requiring dialysis.

-   The two panels created two different definitions for this indicator.
    The most conservative definition was selected.

***Results of pediatric clinician panel review***

As part of the current pediatric indicator development process, this
indicator was reviewed by a panel of twelve pediatric clinicians, one
pediatric critical care specialist, one pediatric hospitalist, one
pediatric anesthesiologist, one pediatric hematologist/oncologist, one
pediatric cardiologist, two pediatric surgeons, one pediatric
neurosurgeon, one pediatric urologist, two pediatric cardiovascular
surgeons, and one neonatologist. The panel reviewed several other
indicators. In the course of review the panel suggested the following:

-   Panelists noted that this complication is rare in children. Unlike
    other rare indicators, they suggested that it would be of limited
    use, with the possible exception of renal failure in cardiac surgery
    patients. Preventability of cases may be unclear and quality review
    in hospitals is almost always undertaken. Limiting the indicator to
    cardiac surgery patients was not implemented since further feedback
    would be necessary to implement.

-   Panelists discussed other types of derangements such as hyponatremia
    and hypo/hyperkalemia, but these were rejected since the presence of
    the condition on admission and the severity of the complication
    cannot be discerned using administrative data.

-   Panelists requested an exclusion for oncology patients, as these
    patients may develop derangements as a result of tumor lysis
    syndrome or from chemotherapy.

**^Post-conference\ call\ panel\ ratings^**

  --------------------------- ------------- --------------------------------
  **Question**                **Median**    **Agreement status**

  Overall rating -- internal  6.5           Indeterminate agreement
  QI                                        

  Overall rating -            6.5           Indeterminate agreement
  comparative                               

  Not present on admission    8             Agreement

  Preventability              7             Indeterminate agreement

  Due to medical error        6             Agreement

  Charting by physicians      7             Indeterminate agreement

  Lack of bias                6.5           Indeterminate agreement

  Final recommendation        Not           
                              recommended   
                              for internal  
                              QI or         
                              comparative   
                              purposes      
  --------------------------- ------------- --------------------------------

***Additional evidence not specific to pediatric population***

Recent unpublished work using linked administrative and clinical data
from the VA Healthcare System showed that the current definition of this
indicator has a sensitivity of 39% (i.e., capturing only 39% of the
patients who truly experienced postoperative renal failure) with a
positive predictive value of 54%. The latter finding is not surprising
because the VA clinical definition is limited to acute renal failure,
and does not include diabetic complications. ^22^

***Literature based evidence specific to pediatric population***

While the pediatric population has lower rates of diabetes and renal
failure than adult patients, children are also at risk for metabolic and
physiologic complications after surgeries. The incidence of these
complications was investigated in pediatric populations (e.g., 0.91 per
1,000 discharges at 0-17 years, 0.54 at 18-44 years, 0.86 at 45-64
years, and 1.33 at 65 or more years).^10^ Other groups have analyzed
rates of this indicator using the publicly available indicator
definition applied to a pediatric population; this definition differs
slightly from the definition proposed above. Miller and Zhan analyzed
HCUP data from 2000 and found 6 pediatric patients (0-18 years of age)
per 10,000 discharges with the diagnosis of postoperative physiologic /
metabolic derangement. Additionally, they found that this complication
resulted in an increased mean length of stay (by 16.3 days) and
\$112,532 in increased charges in affected patients, with 45.8 times
higher odds of in-hospital mortality (after adjusting for age, gender,
expected payer, up to 30 comorbidities, and multiple hospital
characteristics, including ownership, teaching status, nursing
expertise, urban location, bed size, pediatric volume, coding intensity,
ICU bed percentage, and surgical discharge percentage).^11,\ 17^

+----------------------------------+-----------------------------------+
| ### POSTOPERATI                  |                                   |
| VE HEMORRHAGE AND HEMATOMA (PSI) |                                   |
+----------------------------------+-----------------------------------+
| **Indicator definition:**        |                                   |
|                                  |                                   |
| Number of patients with          |                                   |
| postoperative hemorrhage or      |                                   |
| hematoma requiring a procedure   |                                   |
| (see definition and exclusions   |                                   |
| below) per 1000 eligible         |                                   |
| admissions (population at risk). |                                   |
| See The Pediatric Quality        |                                   |
| Indicator Technical              |                                   |
| Specifications.                  |                                   |
+----------------------------------+-----------------------------------+
| **Definition of hemorrhage and   | **Definition of population at     |
| hematoma requiring a             | risk:**                           |
| procedure**:                     |                                   |
|                                  | Patients eligible to be included  |
|                                  | in this indicator:                |
+----------------------------------+-----------------------------------+
| ***Secondary diagnosis code      | ***a. All elective surgical       |
| for:***                          | patients (defined by DRG), age    |
|                                  | 0-17 years, except exclusions     |
| -   Hematoma complicating a      | (see below).***                   |
|     procedure \[998.12\]         |                                   |
|                                  | ***b. Exclude patients with a     |
| ***With any procedure code for   | principal diagnosis of hemorrhage |
| drainage of hematoma.***         | or hematoma, patients where the   |
|                                  | only operating room procedure is  |
| ***OR***                         | control of hemorrhage or drainage |
|                                  | of hematoma, or patients where a  |
| ***Secondary diagnosis code      | procedure for control of          |
| for:***                          | hemorrhage or drainage of         |
|                                  | hematoma occurs before the first  |
| -   Hemorrhage complicating a    | operating room procedure.***      |
|     procedure \[998.11\]         |                                   |
|                                  | ***c. Exclude newborns with a     |
| ***With any procedure code for   | birth weight less than 500g.***   |
| control of hemorrhage.***        |                                   |
|                                  | ***d. Exclude obstetric patients  |
|                                  | (MDC 14)***                       |
|                                  |                                   |
|                                  | ***e. Stratified rates will be    |
|                                  | available for patients with any   |
|                                  | diagnosis code indicating for     |
|                                  | specified coagulopathies          |
|                                  | (Congenital clotting factor       |
|                                  | deficiencies, Von Willebrand's    |
|                                  | disease, primary, secondary and   |
|                                  | unspecified thrombocytopenia,     |
|                                  | intrinsic circulating             |
|                                  | anticoagulants, defibrination     |
|                                  | syndrome and acquired coagulation |
|                                  | factor deficiency) or any         |
|                                  | procedure code for Extracorporeal |
|                                  | Membrane Oxygenation (ECMO).***   |
+----------------------------------+-----------------------------------+
| ***See rates on following        |                                   |
| page.***                         |                                   |
+----------------------------------+-----------------------------------+

+-----------------------------------------------------------------------+
| **Rates based on year 2003 KIDs' Inpatient Database** (per 1000):     |
|                                                                       |
|   ------------------ -                                                |
| ---------------- -------------------- ------------ ------------------ |
|   **                                                                  |
| OVERALL**        1.76                                                 |
|                                                                       |
|   **                                                                  |
| Age stratified                                                        |
|   ra                                                                  |
| tes:**                                                                |
|                                                                       |
|   **                                                                  |
| Neonate, \<      0.00                                                 |
|   20                                                                  |
| 00g**                                                                 |
|                                                                       |
|   **                                                                  |
| Neonate, ≥       1.04                                                 |
|   20                                                                  |
| 00g**                                                                 |
|                                                                       |
|   **                                                                  |
| 29 days -- 364   4.12                                                 |
|   da                                                                  |
| ys**                                                                  |
|                                                                       |
|   **                                                                  |
| 1 -- 2 years**   1.27                                                 |
|                                                                       |
|   **                                                                  |
| 3 -- 5 years**   1.32                                                 |
|                                                                       |
|   **                                                                  |
| 6 -- 12 years**  1.41                                                 |
|                                                                       |
|   **                                                                  |
| 13 -- 17 years** 1.27                                                 |
|                                                                       |
|                                                                       |
|                                                                       |
|                                                                       |
|   **                                                                  |
| Clinical                                                              |
|   st                                                                  |
| ratification**                                                        |
|                                                                       |
|   **                                                                  |
| High risk:                                              18.47         |
|   Sp                                                                  |
| ecified                                                               |
|   co                                                                  |
| agulopathies and                                                      |
|   EC                                                                  |
| MO**                                                                  |
|                                                                       |
|   **                                                                  |
| Low risk: All                                           1.50          |
|   ot                                                                  |
| her patients**                                                        |
|                                                                       |
|                                                                       |
|                                                                       |
|                                                                       |
|   **                                                                  |
| Hospital type**                                                       |
|                                                                       |
|   **                                                                  |
| Children's**                       **Non-Children's**                 |
|                                                                       |
|   **OVER                                                              |
| ALL**        2.06              **OVERALL**                       1.28 |
|                                                                       |
|   **                                                                  |
| Age stratified                     **Age stratified                   |
|   ra                                                                  |
| tes:**                             rates:**                           |
|                                                                       |
|   **Neon                                                              |
| ate, \<      0.00              **Neonate, \<                     0.00 |
|   20                                                                  |
| 00g**                              2000g**                            |
|                                                                       |
|   **Neon                                                              |
| ate, ≥       1.88              **Neonate, ≥ 2000g**              0.00 |
|   20                                                                  |
| 00g**                                                                 |
|                                                                       |
|   **29 d                                                              |
| ays -- 364   4.92              **29 days -- 364                  1.57 |
|   da                                                                  |
| ys**                               days**                             |
|                                                                       |
|   **1 --                                                              |
|  2 years**   1.63              **1 -- 2 years**                  0.36 |
|                                                                       |
|   **3 --                                                              |
|  5 years**   1.07              **3 -- 5 years**                  2.18 |
|                                                                       |
|   **6 --                                                              |
|  12 years**  1.47              **6 -- 12 years**                 1.43 |
|                                                                       |
|   **13 -                                                              |
| - 17 years** 1.25              **13 -- 17 years**                1.17 |
|                                                                       |
|   **                                                                  |
| Clinical                           **Clinical strata:**               |
|   st                                                                  |
| rata:**                                                               |
|                                                                       |
|   **High                                                              |
| risk**      18.50             **High risk**                     24.54 |
|                                                                       |
|   **Low                                                               |
| risk**       1.74              **Low risk**                      1.08 |
|   ------------------ -                                                |
| ---------------- -------------------- ------------ ------------------ |
+-----------------------------------------------------------------------+

***Status summary.*** Based on the current evidence base, from the
pediatric literature review pediatric panel review, and empirical
analyses, this indicator is recommended for inclusion in the pediatric
indicator set. Panelists rated this indicator favorably and with
agreement for both internal quality improvement and comparative
reporting purposes.

***Changes from AHRQ QI Implemented Prior to Pediatric Panel Review***

  ---------------------- ------------------------ ------------------------
  **AHRQ QI definition** **Pediatric indicator    **Reason implemented**
                         definition**             

  All ages               Age 0 -- 17              Pediatric age range

  Premature neonates     Exclude newborns with a  Excluded from all
  included.              birth weight less than   indicators due to very
                         500g.                    high risk nature and
                                                  bias related to delivery
                                                  practices (i.e.
                                                  attempting delivery vs.
                                                  allowing fetal death).

  Patients with          Exclude patients with    Congenital
  coagulopathies         any diagnosis code       coagulopathies
  identifiable through   indicating specified     constitute a higher
  administrative data    coagulopathies           percentage of total
  included (tend to be a                          coagulopathies than in
  small percentage in                             adults and pediatric
  adults).                                        patients are at higher
                                                  risk.
  ---------------------- ------------------------ ------------------------

***Changes Implemented to Pediatric Indicator as a Result of Pediatric
Panel Review***

  ---------------------- ------------------------ ------------------------
  **Pre-panel            **Post-panel indicator   **Reason implemented**
  definition**           definition**             

  All surgery types      Limit to elective        Limiting to elective
  included.              surgery patients.        surgery patients allows
                                                  for the inclusion of
                                                  high risk patients, as
                                                  these patients should be
                                                  adequately controlled if
                                                  surgery is elective.

  Exclude patients with  Stratify patients with   With proper prophylaxis,
  specified congenital   any coagulopathy         most serious bleeding in
  coagulopathies.        (acquired or congenital) patients with
                         ascertainable with       coagulopathies is
                         administrative data.     preventable in elective
                         Include patients on ECMO surgeries. Only a
                         in high risk stratum.    mimimal number of cases
                                                  where the coagulopathy
                                                  was not previously known
                                                  will be included.
                                                  Nonetheless, all
                                                  patients with
                                                  coagulopathy are at
                                                  higher risk for
                                                  bleeding, and are
                                                  generally cared for by
                                                  tertiary care centers.
  ---------------------- ------------------------ ------------------------

Changes considered, but not implemented

  ----------------------- ----------------------- -----------------------
  **AHRQ QI definition**  **Pediatric indicator   **Reason not
                          definition**            implemented**

  None.                                           
  ----------------------- ----------------------- -----------------------

***Clinical rationale***

This indicator is intended to flag cases of hemorrhage or hematoma
following a surgical procedure. It is based on an indicator developed as
part of the Complications Screening Program. This indicator limits
hemorrhage and hematoma codes to secondary procedure and diagnosis codes
in order to isolate those hemorrhages that can truly be linked to a
surgical procedure. High quality surgical technique and proper
prophylaxis in high risk patients may reduce the risk of this
complication.

***Summary of AHRQ QI clinician panel reviews***

This indicator was reviewed twice during our development of the Patient
Safety Indicators, which included a clinical panel review. For this
indicator, the first panel (multispecialty) consisted of 7 clinicians: a
cardiologist, a trauma surgeon, a critical care physician, a
hospitalist, an internist, and two nurse specialists. The second
(surgery specialist) panel consisted of 6 clinicians: a trauma surgeon,
a pediatric neurosurgeon, three orthopedic surgeons and a female
urologist. Both panels reviewed several other indicators. In the course
of review the panels advocated for the following:

-   Panelists argued for risk adjustment by procedure type and
    comorbidity as possible.

-   Surgeons removed seromas from the definition of this indicator,
    citing the insignificant nature of many seromas.

-   Surgeons noted that post-procedural (non OR) hemorrhages are not
    included in this indicator, although such procedure-related
    hemorrhages may be clinically significant.

-   Surgeons argued for the exclusion of patients on anticoagulant
    therapies or coagulopathies, although in adult populations few of
    the common conditions can be identified using administrative data.

***Results of pediatric clinician panel review***

As part of the current pediatric indicator development process, this
indicator was reviewed by a panel of twelve pediatric clinicians, one
pediatric critical care specialist, one pediatric hospitalist, one
pediatric anesthesiologist, one pediatric hematologist/oncologist, one
pediatric cardiologist, two pediatric surgeons, one pediatric
neurosurgeon, one pediatric urologist, two pediatric cardiovascular
surgeons, and one neonatologist. The panel reviewed several other
indicators. In the course of review the panel suggested the following:

-   First, panelists argued that this indicator should be limited to
    elective surgery patients, but should include children with
    congenital and acquired coagulopathies. This would then allow the
    indicator to evaluate peri-operative management of hemostasis in
    high risk groups in a, theoretically, controlled situation.

-   Despite the desire to track high risk children, panelists requested
    that rates in patients with coagulopathy be available separately,
    since children are not routinely screened for coagulopathy and may
    be more likely to have hemorrhage due to coagulopathy than adults.
    Panelists requested both acquired and congenital coagulopathies be
    included, since both types are related to higher risk of bleeding.

NOTES: Although not expressly discussed by the panel, we would like to
note that many coagulation disorders are diagnosed only after bleeding
occurs. Because of this, the strata of patients with coagulopathies
likely does not include all high risk patients, as some patients with
coagulopathies may not be diagnosed if they do not have a bleed. In
addition, some patients may have acquired coagulopathies due to
suboptimal care during the hospitalization. Consequently, an adverse
effect of providing a high risk stratum is the possibility for hospitals
to code more patients with coagulopathies in uncertain circumstances in
order to reduce the low risk strata rates. We suggest reviewing the
overall rate as well as the stratified rates.

During a discussion of a separate indicator, panelists noted that the
definition of elective may be "fuzzy" in some instances. Panelists
struggled to define exactly what elective surgery would entail, given
the use of the ATYPE variable. Although not expressly noted by the
panel, we should note that the validity of the ATYPE variable for
defining elective surgery in children is not known, although analyses in
adults have demonstrated that ATYPE often captures types of surgeries
which are commonly elective.^23^ Children may be at higher risk for
having "urgent" rather than "emergent" surgeries, due to the higher use
of tertiary care centers. It is unknown whether these "urgent" surgeries
are mis-classified as emergent in subsequent numbers to affect the
indicator.

**^Post-conference\ call\ panel\ ratings^**

  --------------------------- ------------- --------------------------------
  **Question**                **Median**    **Agreement status**

  Overall rating -- internal  7             Agreement
  QI                                        

  Overall rating -            7             Agreement
  comparative                               

  Not present on admission    8             Agreement

  Preventability              7             Agreement

  Due to medical error        4.5           Indeterminate agreement

  Charting by physicians      8             Agreement

  Lack of bias                6             Indeterminate agreement

  Final recommendation        Internal QI:  
                              Acceptable    
                              (+)           
                              Comparative   
                              purposes:     
                              Acceptable    
                              (+)           
  --------------------------- ------------- --------------------------------

***Empirical analyses to inform indicator definition***

The following empirical analysis aided in formulating the definition for
this indicator. Analyses were conducted using the 2000 Nationwide
Inpatient Sample.

Before panel review, we reviewed the risk of developing this
complication for patients with congenital coagulopathies (ICD-9-CM codes
286.0-286.4 and 286.5, although this former code may include some
acquired coagulopathies). The results of the analyses demonstrated an
almost 10-fold increased risk of coagulopathy, although the frequency of
these codes was rare (5 cases in the numerator and 696 in the
denominator). The inclusion of 286.5 was questioned at this stage,
because this code includes \"hyperheparinemia\".  In other words, if a
patient receives an overdose of heparin, with subsequent hemorrhage, the
diagnosis would be 286.5 plus an E-code from the drug toxicity chapter
of the ICD-9-CM coding system. None of the cases in the numerator had
code 286.5.

The panelists then suggested that all coagulopathies are of interest and
asked that all be stratified (i.e., congenital and acquired). A second
analysis, using the most current definition, was undertaken to determine
the relative risk for post-operative hemorrhage and hematoma
complication among patients with coagulopathies. When undergoing
elective surgery none of the 539 children with congenital/chronic
coagulopathies (ICD-9-CM codes 286.0-286.4, 287.1, 287.3, 287.8, 287.9)
had this complication.   The majority of the numerator events that
involved coagulopathies occurred among patients with the vaguest codes,
specifically \"other and unspecified coagulation defects (286.9)\" and
\"thrombocytopenia unspecified (287.5).\"  We recommend that this issue
be examined further using so-called "present on admission" data from the
CA and NY SID data. If through these follow-up analyses, it is confirmed
that these conditions are not known to be present on admission, then the
complication itself could trigger a diagnosis of the coagulopathy and
lead to substantial bias.

The following empirical analysis was completed after the initial panel
review using the 2003 KIDs' Inpatient Sample(NIS).

We examined the rate for patients undergoing ECMO. We found that the
number of patients in the denominator was very small (53 using the NIS),
but we also found that these patients were at much higher risk of this
complication (rate is 92.5 per 1000 patients with ECMO code vs. 2.12 per
1000 patients overall).

***Literature based evidence specific to pediatric population***

Post-operative hemorrhage or hematoma is an issue of significant concern
in the pediatric population. For example, one of the most common
surgical procedures performed in this patient group is tonsillectomy
(with or without adenoidectomy) and peri-operative or post-operative
bleeding is one of the most concerning complications.^24,\ 25^

The incidence of post-operative hemorrhage or hematoma, using the
original AHRQ PSI definition was investigated in pediatric populations
(e.g., 1.02 per 1,000 discharges at 0-17 years, 1.50 at 18-44 years,
1.99 at 45-64 years, and 2.54 at 65 or more years).^10^ Other groups
have analyzed rates of this indicator using the publicly available
indicator definition applied to a pediatric population; this definition
differs slightly from the definition proposed above. Miller and Zhan
analyzed HCUP data from 2000 and found 13 pediatric patients (0-18 years
of age) per 10,000 discharges with the complication of postoperative
hemorrhage or hematoma. Additionally, they found that this complication
resulted in an increased mean length of stay (by 7.9 days) and \$75,932
in increased charges in affected patients, with 3.5 times higher odds of
in-hospital mortality (after adjusting for age, gender, expected payer,
up to 30 comorbidities, and multiple hospital characteristics, including
ownership, teaching status, nursing expertise, urban location, bed size,
pediatric volume, coding intensity, ICU bed percentage, and surgical
discharge percentage).^11^ Sedman et al analyzed NACHRI data from
1999-2002 and found observed rates varying from 2.6 per 1,000 patients
in 2001 to 2.7 per 1,000 patients in 2002 (with no consistent trend over
time).^12^

+----------------------------------+-----------------------------------+
| ### POSTOPE                      |                                   |
| RATIVE RESPIRATORY FAILURE (PSI) |                                   |
+----------------------------------+-----------------------------------+
| **Indicator definition:**        |                                   |
|                                  |                                   |
| Number of patients with          |                                   |
| respiratory failure (see         |                                   |
| definition and exclusions below) |                                   |
| per 1000 eligible admissions     |                                   |
| (population at risk). See The    |                                   |
| Pediatric Quality Indicator      |                                   |
| Technical Specifications.        |                                   |
+----------------------------------+-----------------------------------+
| **Definition of respiratory      | **Definition of population at     |
| failure**:                       | risk:**                           |
|                                  |                                   |
|                                  | Patients eligible to be included  |
|                                  | in this indicator:                |
+----------------------------------+-----------------------------------+
| ***Secondary diagnosis code      | ***a. All elective surgical       |
| for:***                          | patients (defined by DRG and      |
|                                  | admission type), age 0-17 years,  |
| -   Acute respiratory failure    | except exclusions (see below).*** |
|     \[518.81, 518.84, 518.5\]    |                                   |
|                                  | ***b. Exclude patients with       |
| ***OR***                         | principal diagnosis code for      |
|                                  | acute respiratory failure,        |
| ***Secondary procedure code      | patient where a procedure for     |
| for:***                          | tracheostomy is the only          |
|                                  | operating room procedure or       |
| -   Insertion of endotracheal    | tracheostomy occurs before the    |
|     tube \[96.04\], when it      | first operating room              |
|     occurs one or more days      | procedure.***                     |
|     after the index surgery      |                                   |
|                                  | ***c. Exclude patients in MDC 4   |
| -   Continuous mechanical        | and 5 (respiratory and            |
|     ventilation of unspecified   | circulatory diseases).***         |
|     duration \[96.70\], when it  |                                   |
|     occurs two or more days      | ***d. Exclude newborns with a     |
|     after the index surgery      | birth weight less than 500g.***   |
|                                  |                                   |
| -   Continuous mechanical        | ***e. Exclude obstetric patients  |
|     ventilation                  | (MDC 14)***                       |
|                                  |                                   |
| > \< 96 consecutive hrs          | ***f. Exclude patients with       |
| > \[96.71\], when it occurs two  | neuromuscular disorders (e.g.     |
| > or more days after the index   | Muscular dystrophy and other      |
| > surgery                        | myopathies, myosthenia gravis,    |
|                                  | Guillain Barre)***                |
| -   Continuous mechanical        |                                   |
|     ventilation for 96           |                                   |
|     consecutive hrs or more      |                                   |
|     \[96.72\], when it occurs on |                                   |
|     the same day or after the    |                                   |
|     index surgery                |                                   |
+----------------------------------+-----------------------------------+

+-----------------------------------------------------------------------+
| **Rates based on year 2003 KIDs' Inpatient Database** (per 1000):     |
|                                                                       |
|   -----                                                               |
| ----------- ----------------- -------------------- ------------------ |
|   **OVERALL**      14.25                                              |
|                                                                       |
|   **Age stratified                                                    |
|   rates:**                                                            |
|                                                                       |
|   **Neonate, \<    411.26                                             |
|   2000g**                                                             |
|                                                                       |
|   **Neonate, ≥     98.08                                              |
|   2000g**                                                             |
|                                                                       |
|   **29 days -- 364 21.40                                              |
|   days**                                                              |
|                                                                       |
|   **1 -- 2 years** 21.49                                              |
|                                                                       |
|   **3 -- 5 years** 13.43                                              |
|                                                                       |
|   **6 -- 12        10.40                                              |
|   years**                                                             |
|                                                                       |
|   **13 -- 17       9.11                                               |
|   years**                                                             |
|                                                                       |
|                                                                       |
|                                                                       |
|   **Hospital                                                          |
|   type**                                                              |
|                                                                       |
|   **Children's**                     **Non-Children's**               |
|                                                                       |
|   **OVERALL**      17.03             **OVERALL**          6.76        |
|                                                                       |
|   **Age stratified                   **Age stratified                 |
|   rates:**                           rates:**                         |
|                                                                       |
|   **Neonate, \<    430.41            **Neonate, \<        508.43      |
|   2000g**                            2000g**                          |
|                                                                       |
|   **Neonate, ≥     91.20             **Neonate, ≥ 2000g** 17.97       |
|   2000g**                                                             |
|                                                                       |
|   **29 days -- 364 20.27             **29 days -- 364     17.34       |
|   days**                             days**                           |
|                                                                       |
|   **1 -- 2 years** 23.94             **1 -- 2 years**     12.83       |
|                                                                       |
|   **3 -- 5 years** 15.13             **3 -- 5 years**     5.87        |
|                                                                       |
|   **6 -- 12        12.07             **6 -- 12 years**    5.71        |
|   years**                                                             |
|                                                                       |
|   **13 -- 17       14.12             **13 -- 17 years**   4.01        |
|   years**                                                             |
|   -----                                                               |
| ----------- ----------------- -------------------- ------------------ |
+-----------------------------------------------------------------------+

***Status summary.*** Based on the current evidence base, from the
pediatric literature review, pediatric panel review, and empirical
analyses, this indicator is recommended for inclusion in the pediatric
quality indicator set. Panelists rated this indicator favorably with
agreement internal quality improvement use and favorably with
indeterminate agreement for comparative reporting purposes.

***Changes from AHRQ QI Implemented Prior to Pediatric Panel Review***

  ---------------------- ------------------------ ------------------------
  **AHRQ QI definition** **Pediatric indicator    **Reason implemented**
                         definition**             

  All ages               Age 0 -- 17              Pediatric age range

  Definition based on    Definition based on both Change prompted by chart
  diagnosis code only.\* diagnosis and procedure  review study (see below)
                         codes for mechanical     from an adult population
                         ventilation.             that demonstrated
                                                  inclusion of procedure
                                                  codes increased the
                                                  sensitivity of this
                                                  indicator.

  Premature neonates     Exclude newborns with a  Excluded from all
  included.              birth weight less than   indicators due to very
                         500g.                    high risk nature and
                                                  bias related to delivery
                                                  practices (i.e.
                                                  attempting delivery vs.
                                                  allowing fetal death).
  ---------------------- ------------------------ ------------------------

\*Note: The change incorporated in the pediatric indicator has also been
implemented for the current AHRQ QI, as part of the February 2006 PSI
update.

***Changes Implemented to Pediatric Indicator as a Result of Pediatric
Panel Review***

  ---------------------- ------------------------ ------------------------
  **Pre-panel            **Post-panel indicator   **Reason implemented**
  definition**           definition**             

  Patients with          Exclude patients with    Patients with
  neuromuscular          neuromuscular disorders  neuromuscular disorders
  disorders included.    (e.g. Muscular dystrophy are more likely to
                         and other myopayhies,    remain ventilated for a
                         myosthenia gravis,       longer period of time,
                         Guillain Barre)          regardless of quality of
                                                  care.
  ---------------------- ------------------------ ------------------------

Changes considered, but not implemented

  ----------------------- ----------------------- -----------------------
  **AHRQ QI definition**  **Pediatric indicator   **Reason not
                          definition**            implemented**

  None.                                           
  ----------------------- ----------------------- -----------------------

***Clinical rationale***

This indicator is intended to flag cases of postoperative respiratory
failure. This indicator limits the code for respiratory failure to
secondary diagnosis and procedure codes in order to eliminate
respiratory failure that was clearly present on admission. High quality
care may reduce the risk of this complication.

***Summary of AHRQ QI clinician panel reviews***

This indicator was reviewed twice during our development of the Patient
Safety Indicators, which included a clinical panel review. For this
indicator, the first panel (multispecialty) consisted of 6 clinicians: a
general surgeon, an internist, two adult hospitalists, and two
specialized nurses. The second (surgery specialist) panel consisted of
nine clinicians: a urologist, a transplant surgeon, two orthopedic
surgeons, a pediatric neurosurgeon, a neurosurgeon, and two colon and
rectal surgeons. Both panels reviewed several other indicators. In the
course of review the panels advocated for the following:

-   The panels reviewed an indicator called "Postoperative Pulmonary
    Compromise" which included additional complications such as acute
    edema of the lung. The surgical panel advocated for the retention
    only of acute respiratory failure, as this complication is
    clinically significant and somewhat preventable. In addition, acute
    respiratory failure, which requires mechanical ventilation, is less
    likely to be coded variably.

-   Both panels advocated for the population at risk to be limited to
    elective surgery patients.

-   Panelists noted that preventability varies greatly by case mix and
    type of surgery, and risk adjustment is necessary.

***Results of pediatric clinician panel review***

As part of the current pediatric indicator development process, this
indicator was reviewed by a panel of twelve pediatric clinicians, one
pediatric critical care specialist, one pediatric hospitalist, one
pediatric anesthesiologist, one pediatric hematologist/oncologist, one
pediatric cardiologist, two pediatric surgeons, one pediatric
neurosurgeon, one pediatric urologist, two pediatric cardiovascular
surgeons, and one neonatologist. The panel reviewed several other
indicators. In the course of review the panel suggested the following:

-   Panelists suggested that patients with neuromuscular disorders be
    excluded, since these patients may remain on the ventilator longer
    than other patients, even with high quality care.

-   Panelists also noted several other high risk groups, including
    infants undergoing cardiac surgery and tracheal reconstruction;
    these patients are excluded via MDC 4 and 5.

-   Finally, panelists noted that children with posterior fossa tumors
    may also remain on the ventilator longer than other patients, even
    with high quality care. However, we cannot identify these patients
    reliably using ICD-9-CM coding.

**^Post-conference\ call\ panel\ ratings^**

  --------------------------- ------------- --------------------------------
  **Question**                **Median**    **Agreement status**

  Overall rating -- internal  8             Agreement
  QI                                        

  Overall rating -            7             Indeterminate agreement
  comparative                               

  Not present on admission    8             Agreement

  Preventability              7             Indeterminate agreement

  Due to medical error        4.5           Indeterminate agreement

  Charting by physicians      8             Agreement

  Lack of bias                5             Indeterminate agreement

  Final recommendation        Internal QI:  
                              Acceptable    
                              (+)           
                              Comparative   
                              purposes:     
                              Acceptable    
                              (-)           
  --------------------------- ------------- --------------------------------

***Empirical analyses to inform indicator definition***

The following empirical analysis aided in formulating the definition for
this indicator. Analyses were conducted using the 2000 Nationwide
Inpatient Sample.

Following the panel suggestion that patients with neuromuscular
disorders be excluded, we identified potential ICD-9-CM codes and
conducted analyses to better understand the risk of postoperative
respiratory failure associated with these codes. Patients with
hereditary and idiopathic peripheral neuropathy (ICD-9-CM codes 356.X)
were not at increased risk for respiratory failure. Patients with
inflammatory and toxic neuropathy (ICD-9-CM codes 357.X) were also not
at increased risk, although these codes were rare, presumably because,
in addition to the disorders being rare, these patients do not generally
undergo elective surgery. Several other categories of codes were not
associated with increased risk, but were very rare or non-existent in
the denominator. Clinically these disorders may be associated with
higher risk, but we were not able to determine the risk empirically
using the NIS. These codes include myoneural disorders (358.X), myotonic
disorders (359.2), familial periodic paralysis (359.3), toxic myopathy
(359.4), myopathy in endocrine diseases classified elsewhere (359.5),
symptomatic inflammatory myopathy in diseases classified elsewhere
(359.6), other myopathies (critical illness myopathy, myopathies nec)
(359.81, 359.89), and myopathy, unspecified (359.9). Patients with
muscular dystrophy (ICD-9-CM codes 359.0, 359.1) were clearly at an
increased risk for this complication (relative risks ranged 11.1 -
18.0).

***Additional evidence not specific to pediatric population***

The original definition of this indicator was limited to diagnosis
codes. Subsequent work using linked administrative and clinical data
from the VA Healthcare System showed that the original definition had a
sensitivity of just 18% (i.e., capturing only 18% of the patients who
truly experienced postoperative respiratory failure) with a positive
predictive value of 74%. By modifying the definition to include
diagnosis or related procedure codes, the sensitivity increased
dramatically to 63% while the positive predictive review fell only
slightly to 66%.^22^

***Literature based evidence specific to pediatric population***

Post-operative respiratory failure is a potential complication after
pediatric surgery, as after adult surgery. The incidence of
post-operative respiratory failure, using the original AHRQ PSI
definition, was investigated in pediatric populations (e.g., 2.27 per
1,000 discharges at 0-17 years, 1.41 at 18-44 years, 2.32 at 45-64
years, and 3.85 at 65 or more years). Other groups have analyzed rates
of this indicator using the publicly available indicator definition
applied to a pediatric population; this definition differs slightly from
the definition proposed above. Using HCUP data from 2000, Miller and
Zhan found 33 pediatric patients (0-18 years of age) per 10,000
discharges with the complication of postoperative respiratory failure.
Additionally, they found that this complication resulted in an increased
mean length of stay (by 24.4 days) and \$140,507 in increased charges in
affected patients, with 76.6 times increased odds of in-hospital
mortality (after adjusting for age, gender, expected payer, up to 30
comorbidities, and multiple hospital characteristics, including
ownership, teaching status, nursing expertise, urban location, bed size,
pediatric volume, coding intensity, ICU bed percentage, and surgical
discharge percentage).^11^

+----------------------------------+-----------------------------------+
| ### POSTOPERATIVE SEPSIS (PSI)   |                                   |
+----------------------------------+-----------------------------------+
| **Indicator definition:**        |                                   |
|                                  |                                   |
| Number of patients with sepsis   |                                   |
| (see definition and exclusions   |                                   |
| below) per 1,000 eligible        |                                   |
| admissions (population at risk). |                                   |
+----------------------------------+-----------------------------------+
| **Definition of sepsis**:        | **Definition of population at     |
|                                  | risk:**                           |
|                                  |                                   |
|                                  | Patients eligible to be included  |
|                                  | in this indicator:                |
+----------------------------------+-----------------------------------+
| ***Secondary diagnosis code      | ***a. All surgical patients       |
| for:***                          | (defined by DRG and admission     |
|                                  | type), age 0-17 years, except     |
| -   Streptococcal septicemia     | exclusions (see below).***        |
|     \[038.0\]                    |                                   |
|                                  | ***b. Exclude patients with       |
| -   Staphylococcal septicemia    | principal diagnosis code for      |
|     \[038.1\]                    | sepsis, infection, or any patient |
|                                  | in DRG 164, 165 or 415.***        |
| -   Staphylococcal septicemia,   |                                   |
|     unspecified \[038.10\]       | ***c. Stratify by risk group:***  |
|                                  |                                   |
| -   Staphylococcal aureus        | ***i. High risk: Immunodeficient  |
|     septicemia \[038.11\]        | patients (HIV, AIDs, immune       |
|                                  | system disorders, transplant,     |
| -   Other staphylococcal         | short bowel syndrome, specified   |
|     septicemia \[038.19\]        | leukemias and lymphomas, renal    |
|                                  | failure and severe                |
| -   Pneumococcal septicemia      | malnutrition).***                 |
|     \[038.2\]                    |                                   |
|                                  | ***ii. Intermediate risk: Lupus,  |
| -   Septicemia due to anaerobes  | renal disease and other rare      |
|     \[038.3\]                    | autoimmune, hepatic failure,      |
|                                  | cachexia, spleen disorders.***    |
| -   Septicemia due to gram       |                                   |
|     negative organism,           | ***iii. Low risk: All other       |
|     unspecified \[038.40\]       | patients***                       |
|                                  |                                   |
| -   Septicemia due to hemophilus | ***c. Include only patients with  |
|     influenzae \[038.41\]        | a length of stay of 4 days or     |
|                                  | more.***                          |
| -   Septicemia due to            |                                   |
|     escherichia coli \[038.42\]  | ***d. Exclude all newborns and    |
|                                  | neonates (age\<28 days)           |
| -   Septicemia due to            | transferred from an acute care    |
|     pseudomonas \[038.43\]       | facility.***                      |
|                                  |                                   |
| -   Septicemia due to serratia   | ***e. Exclude obstetric patients  |
|     \[038.44\]                   | (MDC 14)***                       |
|                                  |                                   |
| -   Septicemia due to other      |                                   |
|     gram-negative organisms      |                                   |
|     \[038.49\]                   |                                   |
|                                  |                                   |
| -   Other specified septicemias  |                                   |
|     \[038.8\]                    |                                   |
|                                  |                                   |
| -   Unspecified septicemia       |                                   |
|     \[038.9\]                    |                                   |
|                                  |                                   |
| -   Septic shock \[785.52\]      |                                   |
|                                  |                                   |
| -   Other shock without mention  |                                   |
|     of trauma \[785.59\]         |                                   |
|                                  |                                   |
| -   Systemic inflammatory        |                                   |
|     response syndrome due to     |                                   |
|     infectious process without   |                                   |
|     organ dysfunction \[995.91\] |                                   |
|                                  |                                   |
| -   Systemic inflammatory        |                                   |
|     response syndrome due to     |                                   |
|     infectious process with      |                                   |
|     organ dysfunction \[995.92\] |                                   |
|                                  |                                   |
| -   Postoperative shock, NEC     |                                   |
|     \[998.0\]                    |                                   |
+----------------------------------+-----------------------------------+

+-----------------------------------------------------------------------+
| **Rates based on year 2003 KIDs' Inpatient Database** (per 1000):     |
|                                                                       |
|   ------------------ -                                                |
| ---------------- -------------------- ------------ ------------------ |
|   **                                                                  |
| OVERALL**        27.39                                                |
|                                                                       |
|   **                                                                  |
| Age stratified                                                        |
|   ra                                                                  |
| tes:**                                                                |
|                                                                       |
|   **                                                                  |
| Neonate, \<      230.85                                               |
|   20                                                                  |
| 00g**                                                                 |
|                                                                       |
|   **                                                                  |
| Neonate, ≥       82.70                                                |
|   20                                                                  |
| 00g**                                                                 |
|                                                                       |
|   **                                                                  |
| 29 days -- 364   55.79                                                |
|   da                                                                  |
| ys**                                                                  |
|                                                                       |
|   **                                                                  |
| 1 -- 2 years**   29.73                                                |
|                                                                       |
|   **                                                                  |
| 3 -- 5 years**   18.53                                                |
|                                                                       |
|   **                                                                  |
| 6 -- 12 years**  15.44                                                |
|                                                                       |
|   **                                                                  |
| 13 -- 17 years** 15.04                                                |
|                                                                       |
|                                                                       |
|                                                                       |
|                                                                       |
|   **                                                                  |
| Clinical                                                              |
|   st                                                                  |
| ratification**                                                        |
|                                                                       |
|   **                                                                  |
| Strata 1. Clean                                         9.10          |
|   Pr                                                                  |
| ocedures                                                              |
|   El                                                                  |
| ective**                                                              |
|                                                                       |
|   **                                                                  |
| Strata 2. Clean                                         18.10         |
|   Pr                                                                  |
| ocedures                                                              |
|   No                                                                  |
| n-Elective**                                                          |
|                                                                       |
|   **                                                                  |
| Strata 3.                                               24.82         |
|   Po                                                                  |
| tentially                                                             |
|   Co                                                                  |
| ntaminated                                                            |
|   El                                                                  |
| ective**                                                              |
|                                                                       |
|   **                                                                  |
| Strata 4.                                               48.93         |
|   Po                                                                  |
| tentially                                                             |
|   Co                                                                  |
| ntaminated                                                            |
|   No                                                                  |
| n-Elective**                                                          |
|                                                                       |
|   **                                                                  |
| Strata 5. Dirty                                                       |
|   or                                                                  |
|  Infected**                                                           |
|                                                                       |
|                                                                       |
|                                                                       |
|                                                                       |
|   **                                                                  |
| Hospital type**                                                       |
|                                                                       |
|   **                                                                  |
| Children's**                       **Non-Children's**                 |
|                                                                       |
|   **OVERA                                                             |
| LL**        31.33             **OVERALL**                       16.72 |
|                                                                       |
|   **                                                                  |
| Age stratified                     **Age stratified                   |
|   ra                                                                  |
| tes:**                             rates:**                           |
|                                                                       |
|   **Neonat                                                            |
| e, \<      246.53            **Neonate, \<                     132.30 |
|   20                                                                  |
| 00g**                              2000g**                            |
|                                                                       |
|   **Neona                                                             |
| te, ≥       84.06             **Neonate, ≥ 2000g**              56.31 |
|   20                                                                  |
| 00g**                                                                 |
|                                                                       |
|   **29 da                                                             |
| ys -- 364   57.68             **29 days -- 364                  44.91 |
|   da                                                                  |
| ys**                               days**                             |
|                                                                       |
|   **1 --                                                              |
| 2 years**   30.99             **1 -- 2 years**                  20.39 |
|                                                                       |
|   **3 --                                                              |
| 5 years**   19.78             **3 -- 5 years**                  16.79 |
|                                                                       |
|   **6 --                                                              |
|  12 years**  17.35             **6 -- 12 years**                 9.85 |
|                                                                       |
|   **13 --                                                             |
|  17 years** 18.08             **13 -- 17 years**                11.11 |
|                                                                       |
|   **                                                                  |
| Clinical                           **Clinical strata:**               |
|   st                                                                  |
| rata:**                                                               |
|                                                                       |
|   **Stra                                                              |
| ta 1**       9.68              **Strata 1**                      5.70 |
|                                                                       |
|   **Strat                                                             |
| a 2**       17.22             **Strata 2**                      16.23 |
|                                                                       |
|   **Stra                                                              |
| ta 3**       31.43             **Strata 3**                      8.78 |
|                                                                       |
|   **Strat                                                             |
| a 4**       57.87             **Strata 4**                      28.69 |
|                                                                       |
|   **                                                                  |
| Strata 5**                         **Strata 5**                       |
|   ------------------ -                                                |
| ---------------- -------------------- ------------ ------------------ |
+-----------------------------------------------------------------------+

***Status summary.*** Based on the current evidence base, from the
pediatric literature review, pediatric panel review, and empirical
analyses, this indicator will be included in the pediatric quality
indicator set. Panelists rated this indicator favorably with
indeterminate agreement for internal quality improvement, but did not
recommend the indicator for comparative reporting purposes.

***Changes from AHRQ QI Implemented Prior to Pediatric Panel Review***

  ---------------------- ------------------------ ------------------------
  **AHRQ QI definition** **Pediatric indicator    **Reason implemented**
                         definition**             

  All ages               Age 0 -- 17              Pediatric age range

  Premature neonates     Exclude newborns with a  Excluded from all
  included.              birth weight less than   indicators due to very
                         500g.                    high risk nature and
                                                  bias related to delivery
                                                  practices (i.e.
                                                  attempting delivery vs.
                                                  allowing fetal death).

  Newborns included.     Exclude all newborns.    Newborns may acquire
                                                  infection in utero or
                                                  during delivery
  ---------------------- ------------------------ ------------------------

***Changes Implemented to Pediatric Indicator as a Result of Pediatric
Panel Review***

  ---------------------- ------------------------ ------------------------
  **Pre-panel            **Post-panel indicator   **Reason implemented**
  definition**           definition**             

  Only elective surgery  Include high risk        As defined this
  patients included.     patients, (i.e.          complication is rare.
                         immunocompromised        Panelists felt the
                         patients) and all        indicator would be more
                         surgery types.           useful with the
                                                  inclusion of patients at
                                                  risk for this
                                                  complication.

  Exclude infection      Exclude infection based  Previous definition was
  based on DRG and       entirely on ICD-9-CM     not adequate for
  specific ICD-9-CM      codes.                   excluding infections in
  codes.                                          the pediatric population
                                                  and all surgery types.

  No stratification.     Stratify by procedure    Risk varies
                         field class (i.e. clean, substantially by
                         clean-contaminated,      procedure.
                         contaminated,            
                         dirty/infected).         
                         Categorization imputed   
                         from DRG and admission   
                         type (e.g. elective).    
  ---------------------- ------------------------ ------------------------

Changes considered, but not implemented

  ----------------------- ----------------------- -----------------------
  **AHRQ QI definition**  **Pediatric indicator   **Reason not
                          definition**            implemented**

  No outer time limit for Define outer time limit Not possible with data.
  developing sepsis.      after surgery for       
                          developing sepsis.      

  Definition only         Expand to track sepsis  Not possible with data.
  includes inpatients.    after outpatient        
                          surgery.                
  ----------------------- ----------------------- -----------------------

***Clinical rationale***

This indicator is intended to flag cases of nosocomial postoperative
sepsis. It is closely related to a complications indicator developed as
part of the Complications Screening Program. In order to better screen
out cases of sepsis that are likely to be present on admission, this
indicator limits its definition of sepsis to secondary diagnoses
(meaning sepsis was not labeled as the principal diagnosis). High
quality of care may reduce the risk for this complication.

***Summary of AHRQ QI clinician panel reviews***

This indicator was reviewed during our development of the Patient Safety
Indicators, which included a clinical panel review. For this indicator
the panel consisted of seven physicians: two general surgeons, a
geriatrician, two adult hospitalists, an internist, and a nurse
specialist. The panel reviewed several other indicators. In the course
of review the panel suggested the following:

-   Panelists reviewed an indicator called "Septicemia" which limited
    the population at risk to certain MDCs and DRGs for which it was
    judged that sepsis would be a potentially preventable complication.
    Panelists rejected this definition as too broad, and argued for the
    restriction of the population at risk to elective surgery patients.
    This complication was felt to be largely preventable in this
    population. This suggestion was implemented.

-   Panelists noted that varying definitions of "sepsis" may affect the
    rate of this indicator.

***Results of pediatric clinician panel review***

As part of the current pediatric indicator development process, this
indicator was reviewed by a panel of ten pediatric clinicians, including
one neonatologist, one infectious disease specialist, one ambulatory
care pediatrician, one pediatric hospitalist, one pediatric
cardiovascular surgeon, one pediatric oncologist, two pediatric
surgeons, one pediatric interventional radiologist, and one pediatric
critical care physician. The panel reviewed several other indicators. In
the course of review the panel suggested the following:

-   The panel felt that sepsis in pediatric patients following elective
    surgery was exceedingly rare. They felt that tracking sepsis after
    all surgeries was more useful in this population, and that tracking
    immunocompromised and other high risk patients would be desirable.
    They requested that rates be available for high risk patients
    separately from low risk patients. Further work is necessary to
    define the high and low risk groups.

-   Panelists discussed that neonates readmitted for elective surgery
    would be useful to track, especially if sepsis due to organisms
    known to be contracted from birth could be excluded (e.g. Group B
    strep). Some sepsis cases where the organism is unspecified will be
    attributable to these infections acquired in utero or during birth.

-   Some panelists desired changes that were not feasible using unlinked
    inpatient administrative data, these included tracking outpatient
    procedures, and setting an outer time limit post-surgery for the
    development of sepsis.

The same panel participated in a second round of rating, which included
preliminary rating, followed by a conference call, and a final rating.
The panel was identical except for the attrition of three panelists
(pediatric cardiovascular surgeon, pediatric oncologist, pediatric
hospitalist). The panel re-reviewed three other indicators. In the
course of review the panel further suggested the following, in addition
to the comments from the previous review:

-   Panelists argued that postoperative sepsis only accounts for a small
    percentage of postoperative infection. They advocated for the
    addition of indicators that examine other postoperative infections,
    including abscesses and wound infections. They agreed though that
    this narrower indicator was an important first step.

-   Panelists noted that many cases (approximately 1/3) would not have
    an organism identified, and without an isolated organism the term
    "sepsis" may actually be used to describe a variety of clinical
    scenarios, depending on the physician.

-   The panelists were presented with a stratification scheme based on
    comorbidities that would entail immunocompromised states. This
    stratification is now incorporated in risk adjustment. Panelists
    noted that such a scheme could never completely incorporate all the
    cases that would be at slightly elevated risk.

-   Panelists noted that the most important distinction meriting
    stratification is procedure type, specifically whether the surgical
    field is considered clean or contaminated. Surgical fields are
    standardly classified into four categories: clean,
    clean-contaminated, contaminated, and dirty/infected, which is
    assigned at the time of surgery. Although this information is not
    contained in the administrative record, panelists suggested that
    these categories be imputed based on the procedures performed.

-   Panelists discussed the inclusion of neonates in this indicator. It
    is difficult to distinguish perinatally acquired infection from
    postoperative sepsis, especially if an organism is not specified.
    Panelists noted that the rate of perinatally acquired infection is
    low, but that it would be cleaner to exclude if the indicator is
    used to inter-hospital comparisons. They did note that the
    complication is important, particularly in babies that are
    hospitalized for extended periods, and argued that sepsis in
    neonates be examined in a separate indicator.

-   Finally, panelists noted that infection control personnel are the
    best source of information on postoperative infection, since they
    look closely at each case and contributing factors.

**^Post-conference\ call\ panel\ ratings^**

  --------------------------- ------------- --------------------------------
  **Question**                **Median**    **Agreement status**

  Overall rating -- internal  7             Indeterminate agreement
  QI                                        

  Overall rating --           6             Indeterminate agreement
  comparative purposes                      

  Not present on admission    7             Indeterminate agreement

  Preventability              6             Indeterminate agreement

  Due to medical error        5             Indeterminate agreement

  Charting by physicians      7.5           Agreement

  Lack of bias                7             Indeterminate agreement

  Final recommendation        Internal QI:  
                              Acceptable    
                              (-)           
                              Comparative   
                              purposes: Not 
                              recommended   
  --------------------------- ------------- --------------------------------

***Empirical analyses to inform indicator definition***

The following empirical analysis aided in formulating the definition for
this indicator. Analyses were conducted using the 2000 Nationwide
Inpatient Sample.

Prior to panel review we investigated the procedure codes for neonates
with this complication in an attempt to better understand if this
complication was clearly associated with infection acquired in utero.
The associated procedure codes did not reveal a clear association, nor a
consistent pattern in types of procedures. As a result the question of
whether or not to exclude neonates (patients age 0-30 days) was posed to
the clinician panel.

The following empirical analyses were completed after the initial panel
review using the 2003 KIDs' Inpatient Database (KID).

Panelists proposed expanding this indicator to include all patients. We
found, as expected, that patients undergoing elective surgery had a
lower rate of this complication than other surgical patients (12.3 per
1000 vs. 31.7 per 1000). We also found that patients with infections
appeared in the non-elective surgery group, motivating a new definition
for the infection exclusion.

In examining this indicator, we recognized the need for a more
comprehensive definition of infection. We selected infection codes from
the ICD-9-CM coding manual that represented either: explicitly stated
bacterial infections (e.g. streptococcal pneumonia) or inflammatory
conditions that may reflect bacterial conditions (e.g. peritonitis). To
ensure that the non-specific codes we had selected were truly associated
with sepsis we calculated the relative risk of these codes as compared
to all cases in the indicator and found a significantly elevated rate
(RR = 4.19).

During panel review, one panelist noted that unspecified septicemia may
be clinically more subjective than those cases with an isolated
organism. Since organisms are often not isolated, we analyzed the
distribution of the codes in the numerator for the panelists'
information. We found that unspecified septicemia accounted for 1/3 of
all cases.

Also, in response to panel and peer reviewer suggestions, we
investigated expanding the definition of immunocompromised patients. We
examined each of the following strata, which are associated with
impaired immunity separately: HIV, primary immunodeficiencies,
transplant, high risk cancer (leukemia, lymphoma), other cancers, lupus,
other rare autoimmune diseases, juvenile rheumatoid arthritis, other
rheumatoid arthritis, short bowel syndrome, renal conditions treated
with immune suppressants, renal failure, hepatic failure, severe
malnutrition, cachexia and spleen disorders.

We found that patients with rheumatoid arthritis, lupus, other
autoimmune disorders, cachexia, and renal diseases were not at elevated
risk for this complication (relative risk less than 1.4), although the
sample size was low for these strata. Patients with spleen disorders and
cancers other than specific leukemia and lymphomas had a slightly
elevated risk (relative risk between 1.4 and 3). Hepatic failure, renal
failure, primary immunodeficiency or having undergone a transplant
procedure had a moderately elevated risk (relative risk between 3 and
9). Patients with HIV, specific leukemia or lymphomas, short bowel
syndrome, or severe malnutrition had a greatly elevated risk (relative
risk above 9).

Finally, panelists suggested that the type of surgery may be more
predictive of this complication than the comorbidities of a patient.
Specifically, they were interested in stratification by surgical field
class, a standard classification widely associated with postoperative
wound infection. The categories of clean, clean-contaminated,
contaminated, and dirty/infected, take into account various aspects of
the procedure field which are assessed at the time of surgery. However,
this classification is not contained in the administrative data. We
attempted to devise a scheme based on the likely classification of a
surgery. For instance, most heart procedures are typically clean,
whereas intestinal procedures would be clean-contaminated or
contaminated. By combining DRG and the admission type (i.e. elective,
non-elective), we devised a stratification scheme which predicts the
surgical type of the patient. For patients with more than one procedure,
the highest risk class is assigned.

In order to investigate this scheme we first analyzed the risk of
postoperative sepsis based on admission type. We confirmed that
admissions designated "elective" in the administrative record, were at
lower risk for this infection than patients designated "urgent" or other
(i.e. "emergent" "newborn" "invalid/missing" or "other") ( RR = 0.34,
1.95. and 1.36 respectively).

We then assigned each DRG to one of five risk classes. Risk class one
included DRGs for surgeries that were typically clean procedures if done
electively. Risk class 2 included DRGs for surgeries that were typically
clean contaminated if done electively. Risk class 3 included trauma and
cellulitis. Risk class 4 included DRGs for infections. Risk class 5
included DRGs where the procedure was not defined (e.g. major OR
procedures in patients with HIV). We excluded DRGs for burns and
transplants from this analysis since these patients are at higher risk
due to comorbid conditions rather than surgical field class and examined
these cases seperately. We examined the risk of postoperative sepsis,
applying all exclusions for the indicator, for each risk class and the
admission type and the interaction of the two. We found for each risk
class, elective admission type had lower relative risk than any other
admission type. Using these analyses we constructed a stratification
scheme which grouped together risk class/admission type combinations
with similar risk of postoperative sepsis. Five strata were identified.

***Additional evidence not specific to pediatric population***

Recent work using linked administrative and clinical data from the VA
Healthcare System showed that the current definition of this indicator
has a sensitivity of 25% (i.e., capturing only 25% of the patients who
truly experienced postoperative sepsis) with a positive predictive value
of 44%.^22^

***Literature based evidence specific to pediatric population***

As in adult surgery, post-operative sepsis is a potential complication
in pediatric surgery. The incidence of post-operative sepsis, using the
original AHRQ PSI definition, was investigated in pediatric populations
(e.g., 3.87 per 1,000 discharges at 0-17 years, 3.71 at 18-44 years,
9.08 at 45-64 years, and 11.16 at 65 or more years).^10^ Other groups
have analyzed rates of this indicator using the publicly available
indicator definition applied to a pediatric population; this definition
differs slightly from the definition proposed above. Using HCUP data
from 2000, a rate of 10.3 per 1,000 discharges was seen for the
complication of postoperative sepsis in pediatric patients 0-18 years of
age.^11^ Additionally, it was found that this complication resulted in
an increased mean length of stay (by 26 days) and \$117,815 in increased
charges in affected patients, with 11 times higher odds of in-hospital
mortality (after adjusting for age, gender, expected payer, up to 30
comorbidities, and multiple hospital characteristics, including
ownership, teaching status, nursing expertise, urban location, bed size,
pediatric volume, coding intensity, ICU bed percentage, and surgical
discharge percentage).^11^

+----------------------------------+-----------------------------------+
| ### POST                         |                                   |
| OPERATIVE WOUND DEHISCENCE (PSI) |                                   |
+----------------------------------+-----------------------------------+
| **Indicator definition:**        |                                   |
|                                  |                                   |
| Number of abdominopelvic surgery |                                   |
| patients with disruption of      |                                   |
| abdominal wall (see definition   |                                   |
| and exclusions below) per 1000   |                                   |
| eligible admissions (population  |                                   |
| at risk). See The Pediatric      |                                   |
| Quality Indicator Technical      |                                   |
| Specifications.                  |                                   |
+----------------------------------+-----------------------------------+
| **Definition of disruption of    | **Definition of population at     |
| abdominal wall**:                | risk:**                           |
|                                  |                                   |
|                                  | Patients eligible to be included  |
|                                  | in this indicator:                |
+----------------------------------+-----------------------------------+
| ***Secondary procedure code      | ***a. All abdominopelvic surgical |
| for:***                          | patients (defined by procedure    |
|                                  | codes), age 0-17 years, except    |
| -   Reclosure of disruption of   | exclusions (see below).***        |
|     abdominal wall \[54.61\]     |                                   |
|                                  | ***b. Exclude patients with       |
|                                  | procedure for reclosure of        |
|                                  | postoperative disruption of       |
|                                  | abdominal wall occurs before or   |
|                                  | on the same day as the first      |
|                                  | abdominopelvic surgery            |
|                                  | procedure.***                     |
|                                  |                                   |
|                                  | ***c. Exclude patients with any   |
|                                  | procedure code for gastroschisis  |
|                                  | repair OR umbilical hernia repair |
|                                  | in newborns (omphalacele repair)  |
|                                  | performed before the reclosure    |
|                                  | procedure.***                     |
|                                  |                                   |
|                                  | ***d. Exclude newborns with a     |
|                                  | birth weight less than 500g.***   |
|                                  |                                   |
|                                  | ***e. Exclude obstetric patients  |
|                                  | (MDC 14)***                       |
|                                  |                                   |
|                                  | ***f. Exclude patients with any   |
|                                  | diagnosis code for                |
|                                  | immunocompromised state (ie.      |
|                                  | Organ transplant, bone marrow or  |
|                                  | stem cell transplant, HIV or      |
|                                  | AIDs, humoral immunodeficiencies, |
|                                  | deficiencies of cell-mediated     |
|                                  | immunity, other specified and     |
|                                  | unspecified immunodeficiency,     |
|                                  | renal failure, severe             |
|                                  | malnutrition) .***                |
|                                  |                                   |
|                                  | ***g. Exclude patients with a     |
|                                  | length of stay of less than 2     |
|                                  | days.***                          |
+----------------------------------+-----------------------------------+

+-----------------------------------------------------------------------+
| **Rates based on year 2003 KIDs' Inpatient Database** (per 1000):     |
|                                                                       |
|   -----                                                               |
| ----------- ----------------- -------------------- ------------------ |
|   **OVERALL**      0.76                                               |
|                                                                       |
|   **Age stratified                                                    |
|   rates:**                                                            |
|                                                                       |
|   **Neonate, \<    2.46                                               |
|   2000g**                                                             |
|                                                                       |
|   **Neonate, ≥     0.77                                               |
|   2000g**                                                             |
|                                                                       |
|   **29 days -- 364 1.70                                               |
|   days**                                                              |
|                                                                       |
|   **1 -- 2 years** 0.68                                               |
|                                                                       |
|   **3 -- 5 years** 0.22                                               |
|                                                                       |
|   **6 -- 12        0.32                                               |
|   years**                                                             |
|                                                                       |
|   **13 -- 17       0.49                                               |
|   years**                                                             |
|                                                                       |
|                                                                       |
|                                                                       |
|   **Hospital                                                          |
|   type**                                                              |
|                                                                       |
|   **Children's**                     **Non-Children's**               |
|                                                                       |
|   **OVERALL**      0.82              **OVERALL**          0.69        |
|                                                                       |
|   **Age stratified                   **Age stratified                 |
|   rates:**                           rates:**                         |
|                                                                       |
|   **Neonate, \<    0.70              **Neonate, \<        5.21        |
|   2000g**                            2000g**                          |
|                                                                       |
|   **Neonate, ≥     0.33              **Neonate, ≥ 2000g** 1.25        |
|   2000g**                                                             |
|                                                                       |
|   **29 days -- 364 1.54              **29 days -- 364     1.86        |
|   days**                             days**                           |
|                                                                       |
|   **1 -- 2 years** 0.52              **1 -- 2 years**     1.33        |
|                                                                       |
|   **3 -- 5 years** 0.40              **3 -- 5 years**     0.00        |
|                                                                       |
|   **6 -- 12        0.43              **6 -- 12 years**    0.28        |
|   years**                                                             |
|                                                                       |
|   **13 -- 17       1.09              **13 -- 17 years**   0.30        |
|   years**                                                             |
|   -----                                                               |
| ----------- ----------------- -------------------- ------------------ |
+-----------------------------------------------------------------------+

***Status summary.*** Based on the current evidence base, from the
pediatric literature review, pediatric panel review, and empirical
analyses, this indicator is recommended for inclusion in the pediatric
indicator set, with the revised definition summarized above. Panelists
rated this indicator favorably, with agreement for both internal quality
improvement and comparative reporting purposes.

***Changes from AHRQ QI Implemented Prior to Pediatric Panel Review***

  ---------------------- ------------------------ ------------------------
  **AHRQ QI definition** **Pediatric indicator    **Reason implemented**
                         definition**             

  All ages.              Age 0 -- 17              Pediatric age range

  All birth weights      Exclude newborns with a  Excluded from all
  included.              birth weight less than   indicators due to very
                         500g.                    high risk nature and
                                                  bias related to delivery
                                                  practices (i.e.
                                                  attempting delivery vs.
                                                  allowing fetal death).

  All procedures         Exclude patients with    These staged procedures
  included.              any procedure code for   are planned re-openings
                         gastrochesis repair,     relatively common in
                         repair of umbilical      pediatrics that should
                         hernia with prosthesis,  not be included in the
                         or umbilical             indicator.
                         herniorrhaphy performed  
                         before the reclosure     
                         procedure.               
  ---------------------- ------------------------ ------------------------

***Changes Implemented to Pediatric Indicator as a Result of Pediatric
Panel Review***

  ---------------------- ------------------------ ------------------------
  **Pre-panel            **Post-panel indicator   **Reason implemented**
  definition**           definition**             

  Immunocompromised      Exclude                  Patients at high risk
  patients included      immunocompromised        for complication with
                         patients.                questionable
                                                  preventability.

  Include all length of  Exclude patients with    Admission patterns vary
  stay zero or one.      length of stay of zero   and these patients are
                         or one.                  unlikely to develop this
                                                  complication during a
                                                  short stay.
  ---------------------- ------------------------ ------------------------

Changes considered, but not implemented

  ----------------------- ------------------------ -----------------------
  **AHRQ QI definition**  **Pediatric indicator    **Reason not
                          definition**             implemented**

  Cancer patients         Decomposed rates for     One panelist felt that
  examined with all other cancer patients          cancer patients may be
  patients.               available.               at higher risk, but
                                                   complication may still
                                                   be preventable.
                                                   However, empirical
                                                   analyses demonstrated
                                                   that cancer patients
                                                   are not at higher risk.
  ----------------------- ------------------------ -----------------------

***Clinical rationale***

This indicator is intended to flag wound dehiscence in patients who have
undergone abdominal and pelvic surgery. A specific code is available to
detect wound dehiscence in this patient population. The indicator is
restricted to secondary diagnoses, and is intended to capture cases
occurring within the same hospitalization. High quality surgical
technique may reduce the risk for this complication.

***Summary of AHRQ QI clinician panel reviews***

This indicator was reviewed twice during our development of the Patient
Safety Indicators, which included a clinical panel review. For this
indicator the first panel (multispecialty) consisted of 6 clinicians: a
general surgeon, an internist, two adult hospitalists, and two
specialized nurses. The second (surgery specialist) panel consisted of
nine clinicians: a urologist, a transplant surgeon, two orthopedic
surgeons, a pediatric neurosurgeon, a neurosurgeon, and two colon and
rectal surgeons. Both panels reviewed several other indicators. In the
course of review, the panels advocated for the following:

-   Panelists rejected a diagnosis code for postoperative wound
    disruption, since the code did not distinguish between minor and
    severe dehiscence. Instead the panelists argued for an indicator
    based only on procedure codes.

-   Surgical panelists argued that trauma, cancer and immunocompromised
    patients be excluded.

-   Risk adjustment for comorbidity and procedure type was advocated.

***Results of pediatric clinician panel review***

As part of the current pediatric indicator development process, this
indicator was reviewed by a panel of twelve pediatric clinicians, one
pediatric critical care specialist, one pediatric hospitalist, one
pediatric anesthesiologist, one pediatric hematologist/oncologist, one
pediatric cardiologist, two pediatric surgeons, one pediatric
neurosurgeon, one pediatric urologist, two pediatric cardiovascular
surgeons, and one neonatologist. The panel reviewed several other
indicators. In the course of review the panel suggested the following:

-   Panelests suggested that immunocompromised patients be excluded
    since these patients are more at risk for non-preventable wound
    dehiscence.

-   Panelist requested that rates for cancer patients be available
    separately, since they also may be at a higher risk for these
    complications, but these complications in cancer patients are still
    important to monitor.

-   Panelists noted that some dehiscences may occur after discharge. In
    order to track these complications, an area level indicator will be
    developed for this indicator, which includes principal procedures to
    close operative wounds identified by this indicator, and which
    utilizes a population denominator. The area level indicator is
    intended to capture transfers and readmissions for wound dehiscence.
    It will be available in addition to this hospital-based indicator.

**^Post-conference\ call\ panel\ ratings^**

  --------------------------- ------------- --------------------------------
  **Question**                **Median**    **Agreement status**

  Overall rating -- internal  8             Agreement
  QI                                        

  Overall rating -            7.5           Agreement
  comparative                               

  Not present on admission    9             Agreement

  Preventability              7             Agreement

  Due to medical error        5.5           Indeterminate agreement

  Charting by physicians      8             Agreement

  Lack of bias                4             Indeterminate agreement

  Final recommendation        Internal QI:  
                              Acceptable    
                              (+)           
                              Comparative   
                              purposes:     
                              Acceptable    
                              (+)           
  --------------------------- ------------- --------------------------------

***Additional evidence not specific to pediatric population***

Recent unpublished work using linked administrative and clinical data
from the VA Healthcare System showed that the current definition of this
indicator has a sensitivity of 23% (i.e., capturing only 23% of the
patients who truly experienced postoperative wound dehiscence) with a
positive predictive value of 72%. The former finding is not surprising
because the VA clinical definition does not require surgical reclosure,
which the PSI definition does. ^22^

***Empirical analyses to inform indicator definition***

The following empirical analysis aided in formulating the definition for
this indicator. Analyses were conducted using the 2000 Nationwide
Inpatient Sample.

Prior to panel review, we conducted analyses to determine whether the
most frequent procedures captured by this indicator in the pediatric
population were planned staged procedures (as reported by chart review
from NACHRI and The Johns Hopkins Hospital).^4,\ 26^ We examined the
most frequent procedures found with the AHRQ QI version of this
indicator applied to a pediatric population. Results indicated that the
most frequent staged procedure captured was repair of gastroschisis
(ICD-9-CM code, 54.71). Many other procedures unlikely to be planned
staged procedures also were captured. Finally an analysis of the change
of the rate of complication after applying exclusions for known staged
procedures showed an expected reduction in rate.

One panelist suggested stratification of the indicator for patients with
cancer. However, empirical analyses demonstrated that the rate in this
population was not elevated, so this indicator was removed after the
panel review.

Following the initial panel review, in conjunction with analyses
completed for other indicators or in response to peer review, we
examined two additional aspects of this indicator. These empirical
analyses were completed using the 2003 KIDs' Inpatient Database (KID).

We examined the length of stay for patients with wound dehiscence given
peer review comments on patents with short stays (length of stay of zero
or one days). We found that almost 19% of the denominator patients had a
length of stay of less than 2 days, but none of these patients appeared
in the numerator.

We examined each of the following strata, which are associated with
impaired immunity separately: HIV, primary immunodeficiencies,
transplant, high risk cancer (leukemia, lymphoma), other cancers, lupus,
other rare autoimmune diseases, juvenile rheumatoid arthritis, other
rheumatoid arthritis, short bowel syndrome, renal conditions treated
with immune suppressants, renal failure, hepatic failure, severe
malnutrition, cachexia and spleen disorders.

Since this complication is relatively rare in children it is difficult
to note any increased risk in each of the potentially high-risk stratum,
but children with short bowel syndrome appear to be at higher risk with
the relative risk over 15 as compared with all patients in the
denominator. Children with spleen disorders also had an elevated risk,
with a relative risk of nearly 3.5. Since the desire was to develop a
stratification or classification scheme for immunocompromised patients
that could be applied to a number of indicators, results from other
indicators were also considered. Consistency across indicators and
modules is desired, and so in consideration of stratification of
pediatric indicators, we also considered the impact of these
comorbidities on an adult population. Some conditions that were rare in
children are less rare in adults and the impact on these complications
more apparent.

***Literature based evidence specific to pediatric population***

Post-operative abdominopelvic wound dehiscence is an issue of concern in
the pediatric surgical population. Other groups have analyzed rates of
this indicator using the publicly available indicator definition applied
to a pediatric population; this definition differs slightly from the
definition proposed above. The incidence of post-operative wound
dehiscence was investigated in pediatric patients in several studies
(e.g., 1.25 per 1,000 discharges at 0-17 years, 1.74 at 18-44 years,
2.65 at 45-64 years, and 3.77 at 65 or more years).^10^ HCUP data from
1997 showed a rate of 2.9 per 10,000 discharges for a broader definition
of post-operative wound disruption (based on either a diagnosis code or
a procedure code). Using HCUP data from 2000, a rate of 8 per 10,000
discharges was seen for the complication of postoperative wound
dehiscence in pediatric patients 0-18 years of age.^11,\ 17^
Additionally, it was found that this complication resulted in an
increased mean length of stay (by 21.1 days) and \$76,737 in increased
charges in affected patients, with 5.7 times higher odds of in-hospital
mortality (after adjusting for age, gender, expected payer, up to 30
comorbidities, and multiple hospital characteristics, including
ownership, teaching status, nursing expertise, urban location, bed size,
pediatric volume, coding intensity, ICU bed percentage, and surgical
discharge percentage).^11^ Sedman et al found a range of observed rates
for post-operative wound dehiscence from 1.7 per 1,000 in 2002 to 1.2
per 10,000 in 1999 using NACHRI data (i.e., a slight downward trend over
time).^12^

+----------------------------------+-----------------------------------+
| ### SELECTED INF                 |                                   |
| ECTION DUE TO MEDICAL CARE (PSI) |                                   |
+----------------------------------+-----------------------------------+
| **Indicator definition:**        |                                   |
|                                  |                                   |
| Number of patients with specific |                                   |
| infection codes (see definition  |                                   |
| and exclusions below) per 1,000  |                                   |
| eligible admissions (population  |                                   |
| at risk). See The Pediatric      |                                   |
| Quality Indicator Technical      |                                   |
| Specifications.                  |                                   |
+----------------------------------+-----------------------------------+
| **Definition of infection**:     | **Definition of population at     |
|                                  | risk:**                           |
|                                  |                                   |
|                                  | Patients eligible to be included  |
|                                  | in this indicator:                |
+----------------------------------+-----------------------------------+
| ***Secondary diagnosis code      | a\. All medical and surgical      |
| for:***                          | patients (defined by DRG), age    |
|                                  | 0-17 years, except exclusions     |
| -   Other infection (Infection,  | (see below).                      |
|     sepsis or septicemia         |                                   |
|     following infusion,          | b\. Exclude patients with         |
|     injection, transfusion, or   | principal diagnosis code of 999.3 |
|     vaccination) \[999.3\]       | or 996.62.                        |
|                                  |                                   |
| -   Infection and inflammatory   | c\. Stratify patients by three    |
|     reaction due to other        | risk groups                       |
|     vascular device, implant,    |                                   |
|     and graft \[996.62\]         | i\. High risk: High risk          |
|                                  | immunodeficient patients (HIV,    |
| Note: These codes identify a     | immune system disorders,          |
| variety of infections, but       | transplant, short bowel syndrome, |
| primarily catheter and IV        | cancer, renal failure and severe  |
| related infections.              | malnutrition.)                    |
|                                  |                                   |
|                                  | ii\. Intermediate risk: Cystic    |
|                                  | fibrosis, Hemophilia,             |
|                                  | Intermediate risk immunodeficient |
|                                  | patients (lupus, renal disease    |
|                                  | and other rare autoimmune,        |
|                                  | hepatic failure, cachexia, spleen |
|                                  | disorders).                       |
|                                  |                                   |
|                                  | iii\. Low risk: All other         |
|                                  | patients                          |
|                                  |                                   |
|                                  | d\. Exclude patients with length  |
|                                  | of stay of less than 2 days.      |
|                                  |                                   |
|                                  | e\. Exclude all newborns (born    |
|                                  | in-hospital) and neonates (age    |
|                                  | \<28 days) transferred from an    |
|                                  | acute care facility.              |
|                                  |                                   |
|                                  | f\. Exclude obstetric patients    |
|                                  | (MDC 14)                          |
+----------------------------------+-----------------------------------+

+-----------------------------------------------------------------------+
| **Rates based on year 2003 KIDs' Inpatient Database** (per 1000):     |
|                                                                       |
|   ------------------ -                                                |
| ---------------- -------------------- ------------ ------------------ |
|   **                                                                  |
| OVERALL**        3.25                                                 |
|                                                                       |
|   **                                                                  |
| Age stratified                                                        |
|   ra                                                                  |
| tes:**                                                                |
|                                                                       |
|   **                                                                  |
| Neonate, \<      9.69                                                 |
|   20                                                                  |
| 00g**                                                                 |
|                                                                       |
|   **                                                                  |
| Neonate, ≥       1.29                                                 |
|   20                                                                  |
| 00g**                                                                 |
|                                                                       |
|   **                                                                  |
| 29 days -- 364   3.58                                                 |
|   da                                                                  |
| ys**                                                                  |
|                                                                       |
|   **                                                                  |
| 1 -- 2 years**   3.44                                                 |
|                                                                       |
|   **                                                                  |
| 3 -- 5 years**   3.57                                                 |
|                                                                       |
|   **                                                                  |
| 6 -- 12 years**  3.12                                                 |
|                                                                       |
|   **                                                                  |
| 13 -- 17 years** 3.18                                                 |
|                                                                       |
|                                                                       |
|                                                                       |
|                                                                       |
|   **                                                                  |
| Clinical                                                              |
|   st                                                                  |
| ratification**                                                        |
|                                                                       |
|   **                                                                  |
| High risk: High                                         24.23         |
|   ri                                                                  |
| sk                                                                    |
|   im                                                                  |
| munodeficient                                                         |
|   pa                                                                  |
| tients (HIV,                                                          |
|   im                                                                  |
| mune system                                                           |
|   di                                                                  |
| sorders,                                                              |
|   tr                                                                  |
| ansplant, short                                                       |
|   bo                                                                  |
| wel syndrome,                                                         |
|   ca                                                                  |
| ncer, renal                                                           |
|   fa                                                                  |
| ilure and severe                                                      |
|   ma                                                                  |
| lnutrition.)**                                                        |
|                                                                       |
|   **                                                                  |
| Intermediate                                            7.61          |
|   ri                                                                  |
| sk: Cystic                                                            |
|   fi                                                                  |
| brosis,                                                               |
|   He                                                                  |
| mophilia,                                                             |
|   In                                                                  |
| termediate risk                                                       |
|   im                                                                  |
| munodeficient                                                         |
|   pa                                                                  |
| tients ( lupus,                                                       |
|   re                                                                  |
| nal disease and                                                       |
|   ot                                                                  |
| her rare                                                              |
|   au                                                                  |
| toimmune,                                                             |
|   he                                                                  |
| patic failure,                                                        |
|   ca                                                                  |
| chexia, spleen                                                        |
|   di                                                                  |
| sorders).**                                                           |
|                                                                       |
|   **                                                                  |
| Low risk: All                                           1.68          |
|   ot                                                                  |
| her patients**                                                        |
|                                                                       |
|                                                                       |
|                                                                       |
|                                                                       |
|   **                                                                  |
| Hospital type**                                                       |
|                                                                       |
|   **                                                                  |
| Children's**                       **Non-Children's**                 |
|                                                                       |
|   **OVER                                                              |
| ALL**        6.15              **OVERALL**                       1.10 |
|                                                                       |
|   **                                                                  |
| Age stratified                     **Age stratified                   |
|   ra                                                                  |
| tes:**                             rates:**                           |
|                                                                       |
|   **Neon                                                              |
| ate, \<      19.75             **Neonate, \<                     5.32 |
|   20                                                                  |
| 00g**                              2000g**                            |
|                                                                       |
|   **Neon                                                              |
| ate, ≥       3.05              **Neonate, ≥ 2000g**              0.43 |
|   20                                                                  |
| 00g**                                                                 |
|                                                                       |
|   **29 d                                                              |
| ays -- 364   7.00              **29 days -- 364                  1.06 |
|   da                                                                  |
| ys**                               days**                             |
|                                                                       |
|   **1 --                                                              |
|  2 years**   6.93              **1 -- 2 years**                  0.87 |
|                                                                       |
|   **3 --                                                              |
|  5 years**   6.13              **3 -- 5 years**                  1.35 |
|                                                                       |
|   **6 --                                                              |
|  12 years**  5.33              **6 -- 12 years**                 1.01 |
|                                                                       |
|   **13 -                                                              |
| - 17 years** 6.20              **13 -- 17 years**                1.39 |
|                                                                       |
|   **                                                                  |
| Clinical                           **Clinical strata:**               |
|   st                                                                  |
| rata:**                                                               |
|                                                                       |
|   **Low R                                                             |
| isk**       25.88             **Low Risk**                      17.26 |
|                                                                       |
|   **Inte                                                              |
| rmediate     9.27              **Intermediate                    3.82 |
|   Ri                                                                  |
| sk**                               Risk**                             |
|                                                                       |
|   **High                                                              |
|  Risk**      3.27              **High Risk**                     0.68 |
|   ------------------ -                                                |
| ---------------- -------------------- ------------ ------------------ |
+-----------------------------------------------------------------------+

***Status summary.*** Based on the current evidence base, from the
pediatric literature review, pediatric panel review, and empirical
analyses, this indicator will be included in the pediatric quality
indicator set. Panelists rated this indicator favorably with
indeterminate agreement for internal quality improvement, but did not
recommend the indicator for comparative reporting purposes.

***Changes from AHRQ QI Implemented Prior to Pediatric Panel Review***

  ---------------------- ------------------------ ------------------------
  **AHRQ QI definition** **Pediatric indicator    **Reason implemented**
                         definition**             

  All ages               Age 0 -- 17              Pediatric age range

  Cancer,                Exclude patients with    Patients with these
  immunocompromised      any code of              conditions tend to have
  state and short bowel  immunocompromised state  long term indwelling
  patients included.     or cancer or short bowel catheters that are prone
                         syndrome.                to infection.

  Normal newborns        Exclude normal newborns. Normal newborns do not
  included.                                       typically have lines
                                                  that would put them at
                                                  risk for this
                                                  complication.

  Premature neonates. .  Exclude newborns with a  Excluded from all
  .                      birth weight less than   indicators due to very
                         500g.                    high risk nature and
                                                  bias related to delivery
                                                  practices (i.e.
                                                  attempting delivery vs.
                                                  allowing fetal death).

  No stratification.     Stratify by low birth    Small neonates are at
                         weight neonate (2000 g   higher risk for
                         or less) and other       infection due to
                         patients.                underdeveloped immune
                                                  systems.
  ---------------------- ------------------------ ------------------------

***Changes Implemented to Pediatric Indicator as a Result of Pediatric
Panel Review***

  ---------------------- ------------------------ ------------------------
  **Pre-panel            **Post-panel indicator   **Reason implemented**
  definition**           definition**             

  High risk patients,    High risk patients       Panelists felt that this
  with long term lines   (those with long term    indicator is most useful
  or immunocompromised   lines and                if high risk patients
  state excluded.        immunocompromised state) are included, since
                         included and stratified. these patients are the
                                                  patients for which
                                                  interventions could be
                                                  targeted.

  Include all length of  Exclude patients with    Admission patterns vary
  stay zero or one.      length of stay of zero   and these patients are
                         or one.                  unlikely to develop this
                                                  complication during a
                                                  short stay.

  Exclude only normal    Exclude all newborns and Difficult to distinguish
  newborns and those \<  neonates transferred     infection captured by
  500 g.                 from another acute care  this indicator from
                         facility.                perinatally acquired
                                                  infection. Consider
                                                  newborn infections in a
                                                  separate novel indicator
                                                  in future.
  ---------------------- ------------------------ ------------------------

Changes considered, but not implemented

  ----------------------- ----------------------- -----------------------
  **AHRQ QI definition**  **Pediatric indicator   **Reason implemented**
                          definition**            

  Separate vaccination    Panelist suggested that Not possible using
  and injection related   vaccination and         codes, but most likely
  infection from line     injection related       few cases are
  infections.             infections are very     vaccination/injection
                          different from line     related.
                          infections and should   
                          be removed from the     
                          indicator.              
  ----------------------- ----------------------- -----------------------

***Clinical rationale***

This indicator is intended to capture infections that are due to medical
care, but are limited to those easily captured using administrative
data. This indicator likely captures mostly line and other vascular
access related infections. High quality care is likely to reduce the
risk for this complication.

***Summary of AHRQ QI clinician panel reviews***

This indicator was reviewed during our development of the Patient Safety
Indicators, which included a clinical panel review. For this indicator
the panel consisted of physicians: two general surgeons, a geriatrician,
two adult hospitalists, an internist, and a nurse specialist. The panel
reviewed several other indicators. In the course of review the panel
suggested the following:

-   Panelists rejected codes for "infection due to contaminated or
    infected blood or other substance" as they felt these complications
    were out of the control of the health care provider. This suggestion
    was implemented.

-   Panelists rejected an exclusion of trauma patients, arguing that
    these patients should be tracked, but argued for the exclusion of
    immunocompromised patients. This suggestion was implemented.

-   Panelists noted that not all infections are preventable and that
    these infections will be charted variably.

***Results of pediatric clinician panel review***

As part of the current pediatric indicator development process, this
indicator was reviewed by a panel of ten pediatric clinicians, including
one neonatologist, one infectious disease specialist, one ambulatory
care pediatrician, one pediatric hospitalist, one pediatric
cardiovascular surgeon, one pediatric oncologist, two pediatric
surgeons, one pediatric interventional radiologist, and one pediatric
critical care physician. The panel reviewed several other indicators. In
the course of review the panel suggested the following:

-   The panel felt that it was important to track line infections in all
    patients, even those at high risk in the pediatric population,
    including tracking immunocompromised patients and those with
    long-term lines. They requested that rates be available for high
    risk patients separately from low risk patients.

-   Panelists suggested that vaccination and injection related
    infections are very different from line infections and should be
    removed from the indicator. However, it is likely that very few of
    the infections detected by the indicator are related to these
    procedures.

The same panel participated in a second round of rating, which included
preliminary rating, followed by a conference call, and a final rating.
The panel was identical except for the attrition of three panelists
(pediatric cardiovascular surgeon, pediatric oncologist, pediatric
hospitalist). The panel re-reviewed three other indicators. In the
course of review the panel further suggested the following, in addition
to the comments from the previous review:

-   Panelists noted that the intermediate risk group used for
    stratification could not be comprehensive as many small patient
    groups are at higher risk of infection. However, they agreed that
    most of the important groups were included. They suggested that we
    also examine patients undergoing cardiovascular surgery and trauma.
    Risk adjustment approaches will account for differences in risk
    between trauma and cardiovascular surgery patients.

-   Panelists also noted that line associated infections are already
    tracked at many hospitals. The clinical based data has been used
    with the administrative based data to confirm actual rates.

**^Post-conference\ call\ panel\ ratings^**

  --------------------------- ------------- --------------------------------
  **Question**                **Median**    **Agreement status**

  Overall rating -- internal  7             Indeterminate agreement
  QI                                        

  Overall rating --           6.5           Indeterminate agreement
  comparative purposes                      

  Not present on admission    7.5           Agreement

  Preventability              7             Indeterminate agreement

  Due to medical error        6             Agreement

  Charting by physicians      6             Indeterminate agreement

  Lack of bias                5             Indeterminate agreement

  Final recommendation        Internal QI:  
                              Acceptable    
                              (-)           
                              Comparative   
                              purposes: Not 
                              recommended   
  --------------------------- ------------- --------------------------------

***Empirical analyses to inform indicator definition***

The following empirical analyses were completed after the initial panel
review using the 2003 KIDs' Inpatient Database (KID).

Panelists suggested we stratify this indicator by risk. We examined the
risk of this complication for several groups theorized to have higher
risked. We the following to be at highly elevated risk: short bowel
syndrome (RR=97.69), immunocompromised state (RR = 29.61), lymphosarcoma
and reticolosarcoma (RR = 34.17), myeloid leukemia (RR = 38.69),
monocytic leukemia (RR = 77.43), leukemia of unspecified cell type
(RR=51.43). The following patients were at intermediate risk: cystic
fibrosis (RR=8.81), hemophilia (RR=14.26), Hodgkin's disease (RR=10.49),
other malignant neoplasms of lymphoid and histiocytic tissue (RR=17.00),
lymphoid leukemia (RR=18.95), and all other cancer (RR=15.60).

In order to further investigate the definition of immunocompromised
state, we examined each of the following strata, which are associated
with impaired immunity separately: HIV, primary immunodeficiencies,
transplant, high risk cancer (leukemia, lymphoma), other cancers, lupus,
other rare autoimmune diseases, juvenile rheumatoid arthritis, other
rheumatoid arthritis, short bowel syndrome, renal conditions treated
with immune suppressants, renal failure, hepatic failure, severe
malnutrition, cachexia and spleen disorders.

We found that patients with rheumatoid arthritis were not at elevated
risk for this complication (relative risk less than 1.4). Patients with
spleen disorders had a slightly elevated risk (relative risk between 1.4
and 3). Patients with lupus, other rare autoimmune diseases, renal
diseases, hepatic failure and cachexia had a moderately elevated risk
(relative risk between 3 and 9). Patients with primary
immunodeficiencies, all types of cancer, short bowel syndrome, renal
failure, or severe malnutrition or having undergone a transplant
procedure had a greatly elevated risk (relative risk above 9).

In a separate analysis, we examined the length of stay for patients with
nosocomial infections given peer review comments on patients with short
stays (length of stay of zero or one days). We found that almost 22% of
the denominator patients had a length of stay of less than 2 days, but
only 1.8% of numerator patients had a length of stay of less than 2
days.

***Literature based evidence specific to pediatric population***

Infections due to medical devices are of great concern to those caring
for critically ill infants and children. These infections represent a
significant iatrogenic problem in pediatric health care. Bloodstream
infections associated with a central intravascular line were found to be
the most common infection site in a sample of pediatric intensive care
units between 1992 and 1997.^27^ Guidelines have been published in an
attempt to decrease the rates of intravascular catheter-related
infections.^28,\ 29^

Other groups have analyzed rates of this indicator using the publicly
available indicator definition applied to a pediatric population; this
definition differs slightly from the definition proposed above. This
indicator was applied to pediatric hospital populations (e.g., 1.89 per
1,000 discharges at 0-17 years, 1.89 at 18-44 years, 2.50 at 45-64
years, and 1.66 at 65 or more years).^10^ Miller and colleagues analyzed
HCUP data from 1997 and found an incidence of "infection attributed to
procedures" (999.3 alone) of 0.13 per 1,000 discharges among children
aged 0-18 years.^17^ In the HCUP data from 2000, using the current PSI
definition, they found a rate of 1.3 per 1,000 discharges for "infection
as a result of medical care".^11^ Sedman et al found observed rates
varying from 3.2 per 1,000 in 1999 to 4.0 per 1,000 in 2002 in the
NACHRI database (i.e., a slight upward trend over time).^12^
Additionally, Miller & Zhan found that this complication resulted in an
increased mean length of stay (by 30 days) and \$121,010 in increased
charges in affected patients, with 2.2 times higher odds of in-hospital
mortality (after adjusting for age, gender, expected payer, up to 30
comorbidities, and multiple hospital characteristics, including
ownership, teaching status, nursing expertise, urban location, bed size,
pediatric volume, coding intensity, ICU bed percentage, and surgical
discharge percentage).^11^

+----------------------------------+-----------------------------------+
| ### TRANSFUSION REACTION (PSI)   |                                   |
+----------------------------------+-----------------------------------+
| **Indicator definition:**        |                                   |
|                                  |                                   |
| Number of patients with          |                                   |
| transfusion reaction (see        |                                   |
| definition and exclusions below) |                                   |
| per 1,000 eligible admissions    |                                   |
| (population at risk). See The    |                                   |
| Pediatric Quality Indicator      |                                   |
| Technical Specifications.        |                                   |
+----------------------------------+-----------------------------------+
| **Definition of transfusion      | **Definition of population at     |
| reaction**:                      | risk:**                           |
|                                  |                                   |
|                                  | Patients eligible to be included  |
|                                  | in this indicator:                |
+----------------------------------+-----------------------------------+
| ***Secondary diagnosis code      | ***a. All medical and surgical    |
| for:***                          | patients (defined by DRG), age    |
|                                  | 0-17 years, except exclusions     |
| -   ABO incompatibility reaction | (see below).***                   |
|     \[999.6\]                    |                                   |
|                                  | ***b. Exclude patients with       |
| -   Rh incompatibility reaction  | principal diagnosis code for      |
|     \[999.7\]                    | transfusion reaction.***          |
|                                  |                                   |
| -   Mismatched blood in          | ***c. Exclude all neonates.***    |
|     transfusion \[E876.0\]       |                                   |
|                                  | ***d. Exclude obstetric patients  |
|                                  | (MDC 14)***                       |
+----------------------------------+-----------------------------------+
| **Rates based on year 2003 KIDs' |                                   |
| Inpatient Database** (per 1000): |                                   |
|                                  |                                   |
|   ---------------- -----         |                                   |
| -- -------------------- -------- |                                   |
|   **OVERALL**                    |                                   |
|    0.002                         |                                   |
|                                  |                                   |
|   **Age stratifi                 |                                   |
| ed                               |                                   |
|   rates:**                       |                                   |
|                                  |                                   |
|                                  |                                   |
|   **Neonate, \<                  |                                   |
|                                  |                                   |
|   2000g**                        |                                   |
|                                  |                                   |
|                                  |                                   |
|   **Neonate, ≥                   |                                   |
|                                  |                                   |
|   2000g**                        |                                   |
|                                  |                                   |
|                                  |                                   |
|   **29 days -- 3                 |                                   |
| 64 0.004                         |                                   |
|   days**                         |                                   |
|                                  |                                   |
|                                  |                                   |
|   **1 -- 2 years                 |                                   |
| ** 0.000                         |                                   |
|                                  |                                   |
|   **3 -- 5 years                 |                                   |
| ** 0.000                         |                                   |
|                                  |                                   |
|   **6 -- 12                      |                                   |
|    0.000                         |                                   |
|   years**                        |                                   |
|                                  |                                   |
|                                  |                                   |
|   **13 -- 17                     |                                   |
|    0.003                         |                                   |
|   years**                        |                                   |
|                                  |                                   |
|                                  |                                   |
|                                  |                                   |
|                                  |                                   |
|                                  |                                   |
|   **Hospital                     |                                   |
|                                  |                                   |
|   type**                         |                                   |
|                                  |                                   |
|                                  |                                   |
|   **Children's**                 |                                   |
|            **Non-Children's**    |                                   |
|                                  |                                   |
|   **OVERALL**      0.            |                                   |
| 002   **OVERALL**          0.000 |                                   |
|                                  |                                   |
|   **Age stratifi                 |                                   |
| ed         **Age stratified      |                                   |
|   rates:**                       |                                   |
|            rates:**              |                                   |
|                                  |                                   |
|   **Neonate, \<                  |                                   |
|            **Neonate, \<         |                                   |
|   2000g**                        |                                   |
|            2000g**               |                                   |
|                                  |                                   |
|   **Neonate, ≥                   |                                   |
|            **Neonate, ≥ 2000g**  |                                   |
|   2000g**                        |                                   |
|                                  |                                   |
|                                  |                                   |
|   **29 days -- 364 0.            |                                   |
| 000   **29 days -- 364     0.000 |                                   |
|   days**                         |                                   |
|            days**                |                                   |
|                                  |                                   |
|   **1 -- 2 years** 0.            |                                   |
| 000   **1 -- 2 years**     0.000 |                                   |
|                                  |                                   |
|   **3 -- 5 years** 0.            |                                   |
| 000   **3 -- 5 years**     0.000 |                                   |
|                                  |                                   |
|   **6 -- 12        0.            |                                   |
| 000   **6 -- 12 years**    0.000 |                                   |
|   years**                        |                                   |
|                                  |                                   |
|                                  |                                   |
|   **13 -- 17       0.            |                                   |
| 009   **13 -- 17 years**   0.000 |                                   |
|   years**                        |                                   |
|                                  |                                   |
|   ---------------- -----         |                                   |
| -- -------------------- -------- |                                   |
+----------------------------------+-----------------------------------+

***Status summary.*** Based on the current evidence base, from the
pediatric literature review, pediatric panel review, and empirical
analyses, this indicator will be included in the pediatric quality
indicator set. Panelists rated this indicator favorably, with agreement
for internal quality improvement, and favorably with indeterminate
agreement for comparative reporting purposes.

***Changes from AHRQ QI Implemented Prior to Pediatric Panel Review***

  ---------------------- ------------------------ ------------------------
  **AHRQ QI definition** **Pediatric indicator    **Reason implemented**
                         definition**             

  All ages               Age 0 -- 17              Pediatric age range

  Newborns included.     Exclude all newborns.    Empirical analyses
                                                  revealed a high rate of
                                                  "reactions" in
                                                  uncomplicated newborns.
                                                  These are likely
                                                  miscoding of
                                                  maternal-fetal ABO or Rh
                                                  incompatibility The high
                                                  rate of miscoding
                                                  suggests lack of
                                                  validity for this
                                                  population.
  ---------------------- ------------------------ ------------------------

***Changes Implemented to Pediatric Indicator as a Result of Pediatric
Panel Review***

  ---------------------- ------------------------ ------------------------
  **Pre-panel            **Post-panel indicator   **Reason implemented**
  definition**           definition**             

  No additional.                                  
  ---------------------- ------------------------ ------------------------

Changes considered, but not implemented

  ----------------------- ----------------------- -----------------------
  **AHRQ QI definition**  **Pediatric indicator   **Reason not
                          definition**            implemented**

  None.                                           
  ----------------------- ----------------------- -----------------------

***Clinical rationale***

This indicator is intended to flag ABO and Rh incompatibility reactions.
High quality care is likely to reduce the incidence of this
complication.

***Summary of AHRQ QI clinician panel reviews***

This indicator was reviewed twice during our development of the Patient
Safety Indicators, which included a clinical panel review. For this
indicator the first panel (multispecialty) consisted of 5 clinicians: a
critical care physician, an adult hospitalist, two specialized nurses,
and an anesthesiologist. The second (surgery specialist) panel consisted
of 6 clinicians: a spine surgeon, a pediatric neurosurgeon, a transplant
surgeon, a urologist (female specialty), a colon and rectal surgeon, and
an orthopedic surgeon. Both panels reviewed several other indicators. In
the course of review the panels suggested the following:

-   Panelists advocated for the inclusion of only ABO and Rh
    incompatibility reactions.

-   Panelists argued that trauma patients should be included, despite
    the occasional deliberate use of mismatched blood.

***Results of pediatric clinician panel review***

This indicator was also reviewed, during the current development process
by a panel of eleven pediatric clinicians, including one general
pediatrician, one pediatric hospitalist, one pediatric critical care
physician, one neonatologist, one pediatric infectious disease
specialist, one pediatric hematologist/oncologist, one pediatric
cardiothoracic surgeon, one pediatric emergency medicine specialist, on
pediatric interventional radiologist, and two pediatric surgeons. In the
course of review the panels suggested or noted the following:

-   Panelists noted that some reactions may result from outpatient
    therapy. In order to track these complications, an area level
    indicator will be developed for this indicator, which includes
    principal diagnoses for transfusion reactions identified by this
    indicator, and which utilizes a population denominator. The area
    level indicator is intended to capture transfers and readmissions
    for transfusion reaction. It will be available in addition to this
    hospital-based indicator.

**^Post-conference\ call\ panel\ ratings^**

  --------------------------- ------------- --------------------------------
  **Question**                **Median**    **Agreement status**

  Overall rating -- internal  8             Agreement
  QI                                        

  Overall rating --           8             Indeterminate agreement
  comparative purposes                      

  Not present on admission    8             Agreement

  Preventability              8             Agreement

  Due to medical error        8             Agreement

  Charting by physicians      8             Agreement

  Lack of bias                7.25          Indeterminate agreement

  Final recommendation        Internal QI:  
                              Acceptable    
                              (+)           
                              Comparative   
                              purposes:     
                              Acceptable    
                              (+)           
  --------------------------- ------------- --------------------------------

***Empirical analyses to inform indicator definition***

The following empirical analysis aided in formulating the definition for
this indicator. Analyses were conducted using the 2000 Nationwide
Inpatient Sample.

Prior to panel review, using the AHRQ QI definition applied to a
pediatric population, we found that the rate of transfusion reaction was
much higher in children than adults (more than 10 fold increase in
children). We suspected that these cases may be in children undergoing
"exchange transfusions" for severe hyperbilirubinemia. Analysis showed
that none of these patients had a procedure code for an exchange
transfusion. In order to better understand this increased rate we looked
at DRGs for patients with transfusion reaction. All but two patients of
66 with transfusion reaction were neonates and most of those neonates
were in a DRG for normal newborns and had a normal birthweight. Due to
this information, we began to suspect miscoding as a cause for the
higher rate in children. To verify we examined the secondary diagnosis
and procedure codes for children with transfusion reaction. Most
children had no diagnosis code that would suggest a severely ill infant
that would have received a transfusion, and therefore truly seemed to be
normal newborns. Many only had one diagnosis code for normal delivery in
addition to the transfusion reaction code. As a result, we concluded
that these cases are most likely miscoding, perhaps of maternal-fetal Rh
incompatibility.

***Literature based evidence specific to pediatric population***

Transfusion reactions due to ABO or Rh incompatibility remain a very
rare but preventable patient safety issue in pediatrics. HCUP data from
1997 showed an event rate of 0.17 per 1,000 discharges for transfusion
reaction among children 0-18 years of age, using a broader definition
that included "other transfusion reaction."^10^ Other groups have
analyzed rates of this indicator using the publicly available indicator
definition applied to a pediatric population; this definition differs
slightly from the definition proposed above. The incidence of
transfusion reactions was investigated in pediatric patients (e.g.,
0.003 per 1,000 discharges at 0-17 years, 0.003 at 18-44 years, 0.006 at
45-64 years, and 0.005 at 65 or more years). ^17^ Using HCUP data from
2000, a rate of 0.006 per 1,000 discharges was seen. Given the rarity of
this complication, Miller and Zhan were not able to determine whether it
was associated with increased mean length of stay, mean hospital
charges, or in-hospital mortality.^11^

+-----------------------------------------------------------------------+
| ### ASTHMA ADMISSION RATE (PQI)                                       |
|                                                                       |
| (AREA LEVEL INDICATOR)                                                |
+-----------------------------------------------------------------------+
| **Indicator definition:**                                             |
|                                                                       |
| Number of patients admitted for asthma (see definition and exclusions |
| below) per 100,000 population. See The Pediatric Quality Indicator    |
| Technical Specifications.                                             |
+-----------------------------------------------------------------------+
| **Included admissions**:                                              |
+-----------------------------------------------------------------------+
| ***All patients 2-17 years old with a principal diagnosis code for    |
| asthma.***                                                            |
|                                                                       |
| -   Extrinsic asthma (unspecified, with status asthmaticus, with      |
|     acute exacerbation \[493.00-2\])                                  |
|                                                                       |
| -   Intrinsic asthma (unspecified, with status asthmaticus, with      |
|     acute exacerbation \[493.10-2\])                                  |
|                                                                       |
| -   Chronic obstructive asthma (unspecified, with status asthmaticus, |
|     with acute exacerbation \[493.20-2\])                             |
|                                                                       |
| -   Exercise induced bronchospasm \[493.81\]                          |
|                                                                       |
| -   Cough variant asthma \[493.82\]                                   |
|                                                                       |
| -   Asthma (unspecified, with status asthmaticus, with acute          |
|     exacerbation \[493.90-2\])                                        |
|                                                                       |
| ***Exclude patients transferring from another institution, MDC 14     |
| (pregnancy, childbirth, and puerperium), or MDC 15 (newborns and      |
| neonates).***                                                         |
|                                                                       |
| ***Exclude patients with any diagnosis for cystic fibrosis, chronic   |
| lung disease of prematurity, anomalies of upper respiratory system,   |
| congenital cystic lung, anomalies of the lungs and accessory lobes,   |
| anomalies of respiratory system, including mediastinal cysts and      |
| pleural anomalies, tracheoesophageal fistula, esophageal atresia and  |
| stenosis, ciliary dismotility syndrome and vascular ring/sling.***    |
+-----------------------------------------------------------------------+
| **Rates based on year 2003 KIDs' Inpatient Sample(per 100,000):**     |
|                                                                       |
|   -------------- ---------------------------------------------------  |
|   **OVERALL**    177.9                                                |
|                                                                       |
|   **Age                                                               |
|   stratified                                                          |
|   rates:**                                                            |
|                                                                       |
|   **2 years**    624.1                                                |
|                                                                       |
|   **3 -- 5       270.4                                                |
|   years**                                                             |
|                                                                       |
|   **6 -- 12      157.2                                                |
|   years**                                                             |
|                                                                       |
|   **13 -- 17     68.3                                                 |
|   years**                                                             |
|   -------------- ---------------------------------------------------  |
+-----------------------------------------------------------------------+

***Status summary.*** Based on the current evidence base, from the
pediatric literature review, pediatric panel review, and empirical
analyses, this indicator is recommended for inclusion in the pediatric
quality indicator set. Panelists rated the indicator favorably with
agreement for internal quality improvement within an area and favorably
with indeterminate agreement for comparative purposes.

***Changes from AHRQ QI Implemented Prior to Pediatric Panel Review***

  ---------------------- ------------------------ ------------------------
  **AHRQ QI definition** **Pediatric indicator    **Reason implemented**
                         definition**             

  All ages               Age 2 -- 17              Pediatric age range.
                                                  Lower age limit changed
                                                  from zero to two, since
                                                  diagnosis in younger
                                                  children may be
                                                  difficult to distinguish
                                                  from bronchospasm.
  ---------------------- ------------------------ ------------------------

***Changes Implemented to Pediatric Indicator as a Result of Pediatric
Panel Review***

  ---------------------- ------------------------ ------------------------
  **Pre-panel            **Post-panel indicator   **Reason implemented**
  definition**           definition**             

  All complicated        Exclude patients with    Patients with
  patients included.     cystic fibrosis, chronic respiratory disorders
                         lung disease of          may have complications
                         prematurity, anomalies   requiring admission.
                         of upper respiratory     
                         system, congenital       
                         cystic lung, anomalies   
                         of the lungs and         
                         accessory lobes,         
                         anomalies of respiratory 
                         system, including        
                         mediastinal cysts and    
                         pleural anomalies,       
                         tracheoesophageal        
                         fistula, esophageal      
                         atresia and stenosis,    
                         ciliary dismotility      
                         syndrome and vascular    
                         ring/sling.              
  ---------------------- ------------------------ ------------------------

Changes considered, but not implemented

  ----------------------- ----------------------- -----------------------
  **AHRQ QI definition**  **Pediatric indicator   **Reason not
                          definition**            implemented**

  Only asthma codes       Include codes for       Panelists felt that
  including in numerator. bronchospasm and        these codes are more
                          wheezing in numerator.  likely to represent
                                                  first presentation
                                                  (e.g., not preventable)
                                                  or other conditions.
  ----------------------- ----------------------- -----------------------

***Clinical rationale***

This indicator is intended to identify hospitalizations for asthma,
where asthma is identified as the principal reason for hospitalization.
Guidelines outline maintenance therapy, including drug treatments, which
may reduce the incidence of acute exacerbations requiring
hospitalization.

This indicator was developed as part of the Prevention Quality Indicator
measure set, and is adapted from indicators developed independently by
John Billings^30^ and Weissman and colleagues^31^ after favorable
evaluation by physician panels.

***Results of pediatric clinician panel review***

This indicator was not reviewed as part of our AHRQ QI indicator
development process, since it is a Prevention Quality Indicator with a
strong evidentiary base in the literature.

As part of the current pediatric indicator development process, this
indicator was reviewed by a panel of eleven pediatric clinicians,
including two ambulatory care pediatricians, one ambulatory care
pediatric nurse practitioner, one family practitioner, one pediatric
hospitalist, one pediatric emergency medicine physician, two pediatric
pulmonologists, one pediatric endocrinologist, and two pediatric
surgeons. The panel reviewed several other indicators. In the course of
review the panel suggested the following:

-   The panel advocated excluding patients with cystic fibrosis and
    anomalies of the respiratory system as these patients represent
    highly complicated cases that may require hospitalization. This
    exclusion was added to the definition.

-   The panel discussed adding codes for bronchospasm and wheezing, but
    agreed that these should not be added, as panelists felt that these
    may be more likely to represent a first presentation (that would not
    be avoidable) or conditions other than asthma.

-   Panelists expressed concern that certain patients may be less likely
    to seek timely care regardless of access to quality care. These
    patients may present with advanced disease. Panelists argued, as for
    all potentially preventable hospitalizations, that this indicator be
    adjusted for socioeconomic status and that differences in cultural
    groups be considered when analyzing results.

-   Panelists also noted that areas with hospitals that have short stay
    units or similar practice patterns (e.g. holding patients in the ER
    instead of admitting) may appear to have lower rates without
    actually having higher quality of care. Given data limitations, no
    changes to the indicator definition could be made to address this
    issue. However, users of the indicator could explore admitting
    patterns with additional data.

**^Post-conference\ call\ panel\ ratings^**

  --------------------------- ------------- --------------------------------
  **Question**                **Median**    **Agreement status**

  Overall rating -- internal  8             Agreement
  QI area                                   

  Overall rating --           7             Indeterminate agreement
  comparative purposes                      

  Access to quality           7             Indeterminate agreement
  outpatient care                           

  Charting by physicians      8             Indeterminate agreement

  Lack of bias                5             Indeterminate agreement

  Final recommendation        Internal QI:  
                              Acceptable    
                              (+)           
                              Comparative   
                              purposes:     
                              Acceptable    
                              (-)           
  --------------------------- ------------- --------------------------------

***Literature based evidence***

Numerous studies have shown that asthma hospitalization rates are
associated with socioeconomic factors, including median household income
(at the area level) and lack of insurance (at the individual level).^31^
A study of asthma hospitalization rates in California in 1993 (ages
0-64) found that areas with median household incomes under \$35,000 had
hospitalization rates that were 1.5 times higher than areas with higher
median incomes.^32^ In Boston, in 1992, age and gender standardized
hospitalization rates (all ages) were correlated with percentage poverty
in an area (r=0.68), percentage holding a bachelor's degree (r=-0.61),
and income (r=-0.51).^33^ Within New York City in 1994, asthma
hospitalization rates were negatively correlated with a zip code area's
median household income (r=-0.67), and positively correlated with the
percentage of minorities in the population (r=0.82).^33^ These findings
confirm an earlier study by Billings et al.,^30^ who reported 6.4-fold
variation in asthma hospitalization rates (age 0-64) at the zip code
level in New York City in 1988, with 70% of this variation explainable
by the percentage of households with annual income below \$15,000.
Millman et al.^34^ reported that low-income zip codes had 5.8 times more
asthma hospitalizations per capita (age 0-64) than high-income zip codes
in 11 states in 1988. Using New York State data, Lin et al showed that
hospitalization rates were higher in areas with higher poverty,
unemployment, minority populations, and lower education levels.^35^ Even
in England, 45% of the variation in asthma hospitalization rates across
90 family health services authorities in 1990-95 was attributable to
socioeconomic factors, plus the availability of secondary care.^36^ To
our knowledge, only one study has reported partial correlations;^37^ it
found that in New York City, the percentage of African-American
residents (age 0-34) was the strongest predictor, and median household
income was the next strongest predictor, of asthma hospitalization
rates.

The observation that asthma admission rates are higher in areas with low
socio-economic status (SES) has led some researchers to hypothesize that
lack of access to care, or poor quality outpatient care, may lead to
higher admission rates. Although analyses of the National Health and
Nutrition Examination Survey found that Medicaid enrollment and Spanish
language preference were associated with inadequate asthma therapy,
these deficiencies in care were not directly linked to hospitalizations
in children.^38^ Studies from other settings have shown that
African-American asthmatics tend to have fewer scheduled primary care
visits, and more hospitalizations and emergency room visits, than White
asthmatics.^39,\ 40^ African-Americans' use of asthma medications in
children may also be less consistent with current practice
guidelines.^41^

Few studies have directly linked high-quality processes of outpatient
care with lower hospitalization rates at either the area or the
individual level. An in-depth study of asthma treatment practices in New
Haven, Boston, and Rochester found that the community with the highest
asthma hospitalization rate (Boston) also had lower use of inhaled
anti-inflammatory agents and oral steroids. The threshold for admission
also appeared to be lower in Boston, as fewer of the admitted children
were hypoxemic, relative to the other cities.^42^ One case control study
from a large health maintenance organization established that not having
a written asthma management plan was a strong risk factor for asthma
hospitalization in children (after adjusting for severity of asthma),
but the use of anti-inflammatory medications was not.^43^ More recent
studies have confirmed that continuity of care with the same provider
and a comprehensive asthma care program decrease the risk of ED visits
and hospitalization for asthma. The risk of hospital admission was lower
when clinical pathways were used for asthmatic children in ERs of
Australian hospitals.^44^ In another Australian study, having a written
asthma action plan contributed to a reduction in hospital and emergency
department attendance.^45^

With patient and parent education, good medical therapy, and outreach
programs, adverse outcomes for children can be reduced
considerably.^43,\ 46^ For example, Medicaid HMO enrollees had higher
age-gender-race adjusted asthma hospital discharge rates than Medicaid
recipients enrolled in primary care case management program under
fee-for-service reimbursement.^47^ On the other hand, experience with
Child Health Plus (CHPlus), a health insurance program providing
ambulatory and ED coverage for uninsured and low-income children (0-13
years) in New York, suggests that some access-improving interventions do
NOT reduce asthma hospitalization rates. Visit rates, follow-up visits,
and total visits to primary care providers were significantly higher
during CHPlus than before enrollment. There was no significant
association between CHPlus coverage and ED visits or hospitalizations
for asthma, although specialty utilization increased .^48^

+-----------------------------------------------------------------------+
| ### DIABETES SHORT-TERM COMPLICATIONS ADMISSION RATE (PQI)            |
|                                                                       |
| (AREA LEVEL INDICATOR)                                                |
+-----------------------------------------------------------------------+
| **Indicator definition:**                                             |
|                                                                       |
| Number of patients admitted for diabetes short-term complications     |
| (ketoacidosis, hyperosmolarity, coma) (see definition and exclusions  |
| below) per 100,000 population. See The Pediatric Quality Indicator    |
| Technical Specifications.                                             |
+-----------------------------------------------------------------------+
| **Included admissions**:                                              |
+-----------------------------------------------------------------------+
| ***All patients 6-17 years old with a principal diagnosis code for    |
| ketoacidosis, hyperosmolarity or coma.***                             |
|                                                                       |
| -   Diabetes with ketoacidosis (includes type II and type I, stated   |
|     as uncontrolled and not stated as uncontrolled \[250.1x\])        |
|                                                                       |
| -   Diabetes with hyperosmolarity (includes type II and type I,       |
|     stated as uncontrolled and not stated as uncontrolled \[250.2x\]) |
|                                                                       |
| -   Diabetes with other coma (includes type II and type I, stated as  |
|     uncontrolled and not stated as uncontrolled \[250.3x\])           |
|                                                                       |
| ***Exclude patients transferring from another institution, MDC 14     |
| (pregnancy, childbirth, and puerperium), or MDC 15 (newborns and      |
| neonates)***                                                          |
+-----------------------------------------------------------------------+
| **Rates based on year 2003 KIDs' Inpatient Sample(per 100,000):**     |
|                                                                       |
|   -------------- ---------------------------------------------------  |
|   **OVERALL**    30.7                                                 |
|                                                                       |
|   **Age                                                               |
|   stratified                                                          |
|   rates:**                                                            |
|                                                                       |
|   **6 -- 12      22.9                                                 |
|   years**                                                             |
|                                                                       |
|   **13 -- 17     41.3                                                 |
|   years**                                                             |
|   -------------- ---------------------------------------------------  |
+-----------------------------------------------------------------------+

***Status summary.*** Based on the current evidence base, from the
pediatric literature review, pediatric panel review, and empirical
analyses, this indicator is recommended for inclusion in the pediatric
quality indicator set. Panelists rated this indicator favorably with
agreement for internal quality improvement within an area but rated the
indicator less favorably for comparative purposes.

***Changes from AHRQ QI Implemented Prior to Pediatric Panel Review***

  ---------------------- ------------------------ ------------------------
  **AHRQ QI definition** **Pediatric indicator    **Reason implemented**
                         definition**             

  All ages               Age 6 -- 17              Pediatric age range.
                                                  Lower age limit
                                                  increased to six years
                                                  to reduce the incidence
                                                  of first time
                                                  presentations included
                                                  in numerator.
  ---------------------- ------------------------ ------------------------

***Changes Implemented to Pediatric Indicator as a Result of Pediatric
Panel Review***

  ---------------------- ------------------------ ------------------------
  **Pre-panel            **Post-panel indicator   **Reason implemented**
  definition**           definition**             

  No additional.                                  
  ---------------------- ------------------------ ------------------------

Changes considered, but not implemented

  ----------------------- ----------------------- -----------------------
  **AHRQ QI definition**  **Pediatric indicator   **Reason not
                          definition**            implemented**

  Uncontrolled diabetes   Uncontrolled diabetes   Panelists felt that
  code in separate        included in numerator.  this code is likely to
  complimentary                                   represent high quality
  indicator.                                      care, with appropriate
                                                  intervention for
                                                  uncontrolled diabetes.
  ----------------------- ----------------------- -----------------------

***Indicator status summary***

Based on the evidentiary base and the pediatric clinician panel review,
this indicator will be included in the pediatric quality indicator set.
For details of the panel discussion see below.

***Clinical rationale***

This indicator is intended to identify hospitalizations for diabetic
ketoacidosis, coma, and hyperosmolarity. With good disease management,
these complications are avoidable.

This indicator was developed as part of the Prevention Quality Indicator
measure set, and is adapted from an indicator developed by John
Billings^30^ and colleagues after favorable evaluation by a physician
panel.

***Results of pediatric clinician panel review***

This indicator was not reviewed as part of our AHRQ QI indicator
development process, since it is a Prevention Quality Indicator with a
strong evidentiary base in the literature.

As part of the current pediatric indicator development process, this
indicator was reviewed by a panel of eleven pediatric clinicians,
including two ambulatory care pediatricians, one ambulatory care
pediatric nurse practitioner, one family practitioner, one pediatric
hospitalist, one pediatric emergency medicine physician, two pediatric
pulmonologists, one pediatric endocrinologist, and two pediatric
surgeons. The panel reviewed several other indicators. In the course of
review the panel suggested the following:

-   The panel discussed the possibility of adding a code for
    uncontrolled diabetes, as is included in the Healthy People 2010
    indicator. Panelists felt that admissions with this code may
    actually be indicative of good care, indicating an attempt to
    pinpoint reasons for uncontrolled diabetes that may be unrelated to
    medical care (e.g. social factors). This code was not added to the
    definition.

-   It would be most desirable to eliminate admissions for initial
    diagnoses; however, it is not possible to do so given coding
    constraints. Panelists felt that a relatively high age lower limit
    (as the 6 year age limit) would aid in reducing the number of first
    time presentations captured by this indicator.

-   Panelists expressed concern that certain patients may be less likely
    to seek timely care regardless of access to quality care. These
    patients may present with less controlled disease. Panelists argued,
    as for all potentially preventable hospitalizations, that this
    indicator be adjusted for socioeconomic status and that differences
    in cultural groups be considered when analyzing results.

**^Post-conference\ call\ panel\ ratings\ --\ Diabetes^**

  --------------------------- ------------- --------------------------------
  **Question**                **Median**    **Agreement status**

  Overall rating -- internal  7             Agreement
  QI area                                   

  Overall rating --           6             Indeterminate agreement
  comparative purposes                      

  Access to quality           7             Indeterminate agreement
  outpatient care                           

  Charting by physicians      8             Agreement

  Lack of bias                7             Indeterminate agreement

  Final recommendation        Internal QI:  
                              Acceptable    
                              (+)           
                              Comparative   
                              purposes: Not 
                              recommended   
  --------------------------- ------------- --------------------------------

***Empirical analyses to inform indicator definition***

The following empirical analysis aided in formulating the definition for
this indicator. Analyses were conducted using the 2000 Nationwide
Inpatient Sample.

Type I and Type II diabetes have differing risk of acute complications
and incidence in the pediatric population. Type II is expected to vary
systematically by area leading to more bias potentially. We examined the
relative incidence of Type I and Type II in this indicator. If Type II
diabetes were frequent enough, a stratified rate may be in order. Over
95% of cases were due to Type I diabetes. Further, a handful of "Type
II" cases (14) occurred in children under 5 years of age, and may be
miscoded, since this type of diabetes is extremely rare in this age
group. Based on this information, stratification was not recommended,
and the age range for the indicator was implemented as 6 years and
above.

***Literature based evidence***

Precipitating events leading to admission may include physiologic
causes, as discussed above, or the cessation of treatment due to access
to care or non-compliance issues. Evidence that such causes are or are
not due to access to care contributes to the construct validity of this
indicator. However, such evidence has not been strongly shown. Access to
care in relation to admissions has been explicitly studied and reported.
Weissman^31^ found that uninsured patients had a higher risk of
admission for DKA and coma than privately insured patients (age 0-64)
(adjusted O.R. 2.18 -- 2.77). Two studies of ACSC indicators reported
validation work for diabetes independent of measure sets. Millman et
al.^34^ reported that low-income zip codes had 4.1 times more diabetes
hospitalizations per capita (age 0-64) than high-income zip codes in 11
states in 1988. Billings et al.^30^ found that low-income zip codes in
New York City (where at least 60% of households earned less than
\$15,000 in 1988, based on adjusted 1980 Census data) had 6.3 times more
diabetes hospitalizations per capita (age 0-64) than high-income zip
codes (where less than 17.5% of households earned less than \$15,000).
Household income explained 52% of the variation in short term diabetes
complication hospitalization rates at the zip code level. These findings
suggest that this indicator may be a marker for poor access to
outpatient care.

+-----------------------------------------------------------------------+
| ### DEHYDRATION ADMISSION RATE (PQI)                                  |
|                                                                       |
| (AREA LEVEL INDICATOR)                                                |
+-----------------------------------------------------------------------+
| **Indicator definition:**                                             |
|                                                                       |
| Number of patients admitted for dehydration (see definition and       |
| exclusions below) per 100,000 population.                             |
+-----------------------------------------------------------------------+
| **Included admissions**:                                              |
+-----------------------------------------------------------------------+
| ***All patients 0-17 years old with a principal diagnosis code for    |
| hypovolemia \[276.5\].***                                             |
|                                                                       |
| ***Exclude patients transferring from another institution, MDC 14     |
| (pregnancy, childbirth, and puerperium), or MDC 15 (newborns and      |
| neonates)***                                                          |
|                                                                       |
| ***Exclude patients with any diagnosis code for immunocompromised     |
| state.***                                                             |
+-----------------------------------------------------------------------+
| **Rates based on year 2003 KIDs' Inpatient Sample**(per 100,000):     |
|                                                                       |
|   -------------- ---------------------------------------------------  |
|   **OVERALL**                                                         |
|                                                                       |
|   **Age                                                               |
|   stratified                                                          |
|   rates:**                                                            |
|                                                                       |
|   **0 -- 30                                                           |
|   days**                                                              |
|                                                                       |
|   **31 days --                                                        |
|   364 days**                                                          |
|                                                                       |
|   **1 -- 2                                                            |
|   years**                                                             |
|                                                                       |
|   **3 -- 5                                                            |
|   years**                                                             |
|                                                                       |
|   **6 -- 12                                                           |
|   years**                                                             |
|                                                                       |
|   **13 -- 17                                                          |
|   years**                                                             |
|   -------------- ---------------------------------------------------  |
+-----------------------------------------------------------------------+

***Status summary.*** Based on current evidence base and the pediatric
panel literature review, this indicator was eliminated from further
review. A subset of patients were added to the indicator for
gastroenteritis admission.

***Changes from AHRQ QI Implemented Prior to Pediatric Panel Review***

  ---------------------- ------------------------ ------------------------
  **AHRQ QI definition** **Pediatric indicator    **Reason implemented**
                         definition**             

  All ages               Age 0 -- 17              Pediatric age range

  Immunocompromised      Immunocompromised        Immunocompromised
  patients included.     patients excluded.       patients are more likely
                                                  to develop complications
                                                  requiring
                                                  hospitalization.
  ---------------------- ------------------------ ------------------------

***Changes Implemented to Pediatric Indicator as a Result of Pediatric
Panel Review***

  ---------------------- ------------------------ ------------------------
  **Pre-panel            **Post-panel indicator   **Reason implemented**
  definition**           definition**             

  Patients with          Patients with principle  Panelists felt that this
  principle diagnosis of diagnosis of dehydration change more accurately
  dehydration in         and a secondary          reflected
  separate indicator.    diagnosis of             gastroenteritis hospital
                         gastroenteritis included admissions and that
                         in the numerator         other types of
                         (gastroenteritis         dehydration admissions
                         indicator).              were not important.
  ---------------------- ------------------------ ------------------------

Changes considered, but not implemented

  ----------------------- ----------------------- -----------------------
  **AHRQ QI definition**  **Pediatric indicator   **Reason not
                          definition**            implemented**

  None.                                           
  ----------------------- ----------------------- -----------------------

***Clinical rationale***

This indicator is intended to identify hospitalizations for dehydration,
where dehydration is identified as the principal reason for
hospitalization. Many cases of dehydration can be treated in an
outpatient setting effectively with early identification, oral
rehydration therapy and IV fluids.

This indicator was developed as part of the Prevention Quality Indicator
measure set, and is adapted from an indicator developed by John
Billings^30^ and colleagues after favorable evaluation by a physician
panel.

***Literature based evidence***

We found little literature on admission for dehydration as an ambulatory
care sensitive condition indicator. Millman et al.^34^ reported that
low-income zip codes had 2.1 times more dehydration hospitalizations per
capita (age 0-64) than high-income zip codes in 11 states in 1988.
Billings et al.^30^ found that low-income zip codes in New York City
(where at least 60% of households earned less than \$15,000 in 1988,
based on adjusted 1980 Census data) had 2.0 times more dehydration
hospitalizations per capita (age 0-64) than high-income zip codes (where
less than 17.5% of households earned less than \$15,000). Household
income explained 42% of the variation in dehydration hospitalization
rates at the zip code level. These findings suggest that this indicator
may be marker for poor access to outpatient care.

In a before and after study conducted on the effectiveness of a clinical
pathway for gastroenteritis in the emergency department of the
Children\'s Hospital at Westmead, the admission rate was reduced from
20.0% in 1996 to 9.1% in 1999 (P \< 0.05) without adverse sequelae.^44^
This finding is consistent with the hypothesis that timely and effective
care for gastroenteritis reduces the severity of dehydration and hence
the risk of hospitalization.

***Results of pediatric clinician panel review***

This indicator was not reviewed as part of our AHRQ QI indicator
development process, since it is a Prevention Quality Indicator with a
strong evidentiary base in the literature.

As part of the current pediatric indicator development process, this
indicator was reviewed by a panel of eleven pediatric clinicians,
including two ambulatory care pediatricians, one ambulatory care
pediatric nurse practitioner, one family practitioner, one pediatric
hospitalist, one pediatric emergency medicine physician, two pediatric
pulmonologists, one pediatric endocrinologist, and two pediatric
surgeons. The panel reviewed several other indicators. In the course of
review the panel suggested the following:

-   Panelists suggested that this indicator be combined with the
    gastroenteritis indicator. That indicator will now include patients
    admitted with a principal diagnosis of dehydration and a secondary
    diagnosis of gastroenteritis as well as patients with a principal
    diagnosis of gastroenteritis. Before this recommendation, there was
    a separate indicator for dehydration. The combination of the
    dehydration and gastroenteritis indicators allowed for
    gastroenteritis patients to be more fully captured in one indicator.
    Patients admitted for dehydration that is not due to gastroenteritis
    will no longer be captured.

***Empirical analyses to inform indicator definition***

The following empirical analysis aided in formulating the definition for
this indicator. Analyses were conducted using the 2000 Nationwide
Inpatient Sample.

We examined the secondary diagnosis codes for patients in the numerator
to better understand the clinical mixture of this indicator. We found
that approximately half of the diagnosis codes were related to
gastroenteritis, supporting the panelists' suggestion to change the
gastroenteritis admission rate indicator and remove the dehydration
indicator from consideration.

+-----------------------------------------------------------------------+
| ### GASTROENTERITIS ADMISSION RATE (PQI)                              |
|                                                                       |
| (AREA LEVEL INDICATOR)                                                |
+-----------------------------------------------------------------------+
| **Indicator definition:**                                             |
|                                                                       |
| Number of patients admitted for gastroenteritis (see definition and   |
| exclusions below) per 100,000 population. See The Pediatric Quality   |
| Indicator Technical Specifications.                                   |
+-----------------------------------------------------------------------+
| **Included admissions**:                                              |
+-----------------------------------------------------------------------+
| ***All patients 3 months - 17 years old with a principal diagnosis    |
| code for gastroenteritis OR a principal diagnosis of dehydration      |
| accompanied by a secondary diagnosis of gastroenteritis.***           |
|                                                                       |
| ***Enteritis due to:***                                               |
+-----------------------------------------------------------------------+
| -   Rotavirus \[008.61\]                                              |
|                                                                       |
| -   Adenovirus \[008.62\]                                             |
|                                                                       |
| -   Norwalk virus \[008.63\]                                          |
|                                                                       |
| -   Other small round virus \[008.64\]                                |
|                                                                       |
| -   Calicivirus \[008.65\]                                            |
|                                                                       |
| -   Astrovirus \[008.66\]                                             |
|                                                                       |
| -   Enterovirus, not elsewhere classified \[008.67\]                  |
|                                                                       |
| -   Other viral enteritis \[008.69\]                                  |
|                                                                       |
| -   Other organism, not otherwise specified (viral) \[00.88\]         |
|                                                                       |
| -   Infectious colitis, enteritis and gastroenteritis not otherwise   |
|     specified \[009.0\]                                               |
|                                                                       |
| -   Colitis, enteritis, and gastroenteritis of presumed infectious    |
|     origin \[009.1\]                                                  |
|                                                                       |
| -   Infectious diarrhea \[009.2\]                                     |
|                                                                       |
| -   Diarrhea of presumed infectious origin \[009.3\]                  |
|                                                                       |
| -   Other and unspecified noninfectious gastroenteritis and colitis   |
|     \[558.9\]                                                         |
+-----------------------------------------------------------------------+
| ***Exclude patients transferring from another institution, MDC 14     |
| (pregnancy, childbirth, and puerperium), or MDC 15 (newborns and      |
| neonates)***                                                          |
|                                                                       |
| ***Exclude patients with any diagnosis code for bacterial             |
| gastroenteritis and gastrointestinal abnormalities.***                |
+-----------------------------------------------------------------------+
| **Rates based on year 2003 KIDs' Inpatient Sample(per 100,000):**     |
|                                                                       |
|   -------------- ---------------------------------------------------  |
|   **OVERALL**    180.80                                               |
|                                                                       |
|   **Age                                                               |
|   stratified                                                          |
|   rates:**                                                            |
|                                                                       |
|   **61 days --   1029.94                                              |
|   364 days**                                                          |
|                                                                       |
|   **1 -- 2       672.01                                               |
|   years**                                                             |
|                                                                       |
|   **3 -- 5       164.86                                               |
|   years**                                                             |
|                                                                       |
|   **6 -- 12      67.15                                                |
|   years**                                                             |
|                                                                       |
|   **13 -- 17     32.44                                                |
|   years**                                                             |
|   -------------- ---------------------------------------------------  |
+-----------------------------------------------------------------------+

***Status summary.*** Based on the current evidence base, from the
pediatric literature review, pediatric panel review, and empirical
analyses, this indicator is recommended for inclusion in the pediatric
quality indicator set. Panelists rated the indicator favorably with
indeterminate agreement for internal quality improvement but rated the
indicator less favorably for comparative purposes.

***Changes from AHRQ QI Implemented Prior to Pediatric Panel Review***

  ---------------------- ------------------------ ------------------------
  **AHRQ QI definition** **Pediatric indicator    **Reason implemented**
                         definition**             

  All ages               Age 0 -- 17              Pediatric age range
  ---------------------- ------------------------ ------------------------

***Changes Implemented to Pediatric Indicator as a Result of Pediatric
Panel Review***

  ---------------------- ------------------------ ------------------------
  **Pre-panel            **Post-panel indicator   **Reason implemented**
  definition**           definition**             

  Patients with          Patients with principle  Panelists felt that this
  principle diagnosis of diagnosis of dehydration change more accurately
  dehydration in         and a secondary          reflected
  separate indicator.    diagnosis of             gastroenteritis hospital
                         gastroenteritis included admissions.
                         in the numerator.        

  No specific exclusion  Exclude patients with    Bacterial
  for bacterial          any diagnosis code for   gastroenteritis may
  gastroenteritis        bacterial                require hospitalization,
                         gastroenteritis.         despite high quality
                                                  outpatient care.

  Age range 0-17         Age range 3 mo.-17       Infants 2 months or
                         years.                   younger often better
                                                  managed as inpatients.

  Include                Exclude patients with    Gastrointestinal
  gastrointestinal       any diagnosis codes of   abnormalities may cause
  abnormalities.         gastrointestinal         diarrhea that may mimic
                         abnormalities.           infectious diarrhea.
  ---------------------- ------------------------ ------------------------

Changes considered, but not implemented

  ----------------------- ----------------------- -----------------------
  **AHRQ QI definition**  **Pediatric indicator   **Reason not
                          definition**            implemented**

  Include patients with   Exclude patients with   Patients are not at
  immunocompromised       immunocompromised       higher risk for being
  state.                  state.                  admitted.
  ----------------------- ----------------------- -----------------------

***Clinical rationale***

This indicator is intended to identify hospitalizations for
gastroenteritis, where gastroenteritis is identified as the principal
reason for hospitalization. Timely and effective care for
gastroenteritis, such as oral rehydration therapy, may reduce the need
for hospitalization.

This indicator was developed as part of the Prevention Quality Indicator
measure set, and is adapted from an indicator developed by John Billings
and colleagues after favorable evaluation by a physician panel.

***Results of pediatric clinician panel review***

This indicator was not reviewed as part of our AHRQ QI indicator
development process, since it is a Prevention Quality Indicator with a
strong evidentiary base in the literature.

As part of the current pediatric indicator development process, this
indicator was reviewed by a panel of eleven pediatric clinicians,
including two ambulatory care pediatricians, one ambulatory care
pediatric nurse practitioner, one family practitioner, one pediatric
hospitalist, one pediatric emergency medicine physician, two pediatric
pulmonologists, one pediatric endocrinologist, and two pediatric
surgeons. The panel reviewed several other indicators. In the course of
review the panel suggested the following:

-   Panelists suggested that this indicator include patients admitted
    with a principal diagnosis of dehydration and a secondary diagnosis
    of gastroenteritis as well as patients with a principal diagnosis of
    gastroenteritis. Before this recommendation, there was a separate
    indicator for dehydration. The combination of the dehydration and
    gastroenteritis indicators allowed for gastroenteritis patients to
    be more fully captured in one indicator.

-   Panelists agreed that patients with immunocompromised state should
    be excluded since these patients may be at increased risk for
    complications due to gastroenteritis, requiring hospitalization.

-   Panelists argued that patients two months of age or less should not
    be included since they felt that these patients have less reserves
    to cope with gastroenteritis / dehydration or additional underlying
    illness and are often best managed in an inpatient setting.

-   Panelists expressed concern that certain patients may be less likely
    to seek timely care regardless of access to quality care. These
    patients may present with advanced disease. Panelists argued, as for
    all potentially preventable hospitalizations, that this indicator be
    adjusted for socioeconomic status and that differences in cultural
    groups be considered when analyzing results.

-   Panelists also noted that areas with hospitals that have short stay
    units or similar practice patterns (e.g. holding patients in the ER
    instead of admitting) may appear to have lower rates without
    actually having higher quality of care. Given data limitations, no
    changes to the indicator definition could be made to address this
    issue. However, users of the indicator could explore admitting
    patterns with additional data.

**^Post-conference\ call\ panel\ ratings\ --\ Gastroenteritis^**

  --------------------------- ------------- --------------------------------
  **Question**                **Median**    **Agreement status**

  Overall rating -- internal  7             Indeterminate agreement
  QI area                                   

  Overall rating --           6             Indeterminate agreement
  comparative purposes                      

  Access to quality           6             Indeterminate agreement
  outpatient care                           

  Charting by physicians      7             Agreement

  Lack of bias                4             Indeterminate agreement

  Final recommendation        Internal QI:  
                              Acceptable    
                              (-)           
                              Comparative   
                              purposes: Not 
                              recommended   
  --------------------------- ------------- --------------------------------

***Literature based evidence***

No published studies have specifically addressed the relationship of the
gastroenteritis hospitalization rate to quality of outpatient care. John
Billings' original study from New York reported 1.87-fold variation in
gastroenteritis hospitalization rates for ages 0-64, with a coefficient
of variation of 0.438 and 22% of variance explained by household
income.^30^ Millman et al.^34^ reported that low-income zip codes had
1.9 times more pediatric gastroenteritis hospitalizations per capita
than high-income zip codes in the same 11 states in 1988. Similarly, a
retrospective analysis of the 1995-96 cohort of infants born in Western
Australia showed that aboriginal infants were hospitalized for
gastroenteritis 8 times more frequently, and readmitted 2.7 times more
frequently than their non-Aboriginal peers.^49^ These findings suggest
that this indicator may be marker for poor access to outpatient care.

In a before and after study conducted on the effectiveness of a clinical
pathway for gastroenteritis in the emergency department of the
Children\'s Hospital at Westmead, the admission rate was reduced from
20.0% in 1996 to 9.1% in 1999 (P \< 0.05) without adverse sequelae.^44^
This finding is consistent with the hypothesis that timely and effective
care for gastroenteritis reduces the need for hospitalization.

+----------------------------------+-----------------------------------+
| ### BACTERIA                     |                                   |
| L PNEUMONIA ADMISSION RATE (PQI) |                                   |
|                                  |                                   |
| (AREA LEVEL INDICATOR)           |                                   |
+----------------------------------+-----------------------------------+
| **Indicator definition:**        |                                   |
|                                  |                                   |
| Number of patients admitted for  |                                   |
| bacterial pneumonia (see         |                                   |
| definition and exclusions below) |                                   |
| per 100,000 population. See The  |                                   |
| Pediatric Quality Indicator      |                                   |
| Technical Specifications.        |                                   |
+----------------------------------+-----------------------------------+
| **Included admissions**:         |                                   |
+----------------------------------+-----------------------------------+
| ***All patients 3 mo. -- 17      |                                   |
| years old with a principal       |                                   |
| diagnosis code for bacterial     |                                   |
| pneumonia.***                    |                                   |
|                                  |                                   |
| ***Pneumonia due to:***          |                                   |
+----------------------------------+-----------------------------------+
| -   Pneumococcus \[481\]         | -   Bacterial pneumonia NOS       |
|                                  |     \[482.9\]                     |
| -   H. influenzae \[482.2\]      |                                   |
|                                  | -   Mycoplasma \[483.0\]          |
| -   Streptococcus unspecified    |                                   |
|     \[482.30\]                   | -   Clamydia \[483.1\]            |
|                                  |                                   |
| -   Group A streptococcus        | -   Other specified organism      |
|     \[482.31\]                   |     \[483.8\]                     |
|                                  |                                   |
| -   Group B streptococcus        | -   Broncopneumonia, organism     |
|     \[482.32\]                   |     unspec \[485\]                |
|                                  |                                   |
| -   Other streptococcus          | -   Organism unspecified \[486\]  |
|     \[482.39\]                   |                                   |
+----------------------------------+-----------------------------------+
| ***Exclude patients transferring |                                   |
| from another institution, MDC 14 |                                   |
| (pregnancy, childbirth, and      |                                   |
| puerperium), or MDC 15 (newborns |                                   |
| and neonates).***                |                                   |
|                                  |                                   |
| ***Exclude patients with any     |                                   |
| diagnosis code for sickle cell   |                                   |
| anemia, HB-S disease, cystic     |                                   |
| fibrosis, immunocompromised      |                                   |
| state (ie. Organ transplant,     |                                   |
| bone marrow or stem cell         |                                   |
| transplant, HIV or AIDs, humoral |                                   |
| immunodeficiencies, deficiencies |                                   |
| of cell-mediated immunity, other |                                   |
| specified and unspecified        |                                   |
| immunodeficiency), chronic lung  |                                   |
| disease of prematurity,          |                                   |
| anomalies of upper respiratory   |                                   |
| system, congenital cystic lung,  |                                   |
| anomalies of the lungs and       |                                   |
| accessory lobes, anomalies of    |                                   |
| respiratory system, including    |                                   |
| mediastinal cysts and pleural    |                                   |
| anomalies, tracheoesophageal     |                                   |
| fistula, esophageal atresia and  |                                   |
| stenosis, ciliary dismotility    |                                   |
| syndrome and vascular            |                                   |
| ring/sling.***                   |                                   |
+----------------------------------+-----------------------------------+
| **Rates based on year 2003 KIDs' |                                   |
| Inpatient Sample(per 100,000):** |                                   |
|                                  |                                   |
|   ----                           |                                   |
| --------- ---------------------- |                                   |
|   **OVERALL**                    |                                   |
|                                  |                                   |
|   **Age                          |                                   |
|   stratified                     |                                   |
|   rates:**                       |                                   |
|                                  |                                   |
|   **3 mo -- \<                   |                                   |
|   1 year**                       |                                   |
|                                  |                                   |
|   **1 -- 2                       |                                   |
|   years**                        |                                   |
|                                  |                                   |
|   **3 -- 5                       |                                   |
|   years**                        |                                   |
|                                  |                                   |
|   **6 -- 12                      |                                   |
|   years**                        |                                   |
|                                  |                                   |
|   **13 -- 17                     |                                   |
|   years**                        |                                   |
|   ----                           |                                   |
| --------- ---------------------- |                                   |
+----------------------------------+-----------------------------------+

***Status summary.*** Based on the current evidence base, from the
pediatric literature review, pediatric panel review, and empirical
analyses, this indicator is not recommended for inclusion in the
pediatric indicator set. Panelists disagreed regarding the usefulness of
this indicator.

***Changes from AHRQ QI Implemented Prior to Pediatric Panel Review***

  ---------------------- ------------------------ ------------------------
  **AHRQ QI definition** **Pediatric indicator    **Reason implemented**
                         definition**             

  All ages               Age 3 mo. -- 17          Pediatric age range.
                                                  Lower range raised to
                                                  reflect standard
                                                  practice of admitting
                                                  very young infants.

  Exclude patients with  Exclude patients with    Patients are at higher
  sickle cell anemia and cystic fibrosis,         risk for developing
  related diseases.      immunocompromised state, complications with
                         anomalies of upper       pneumonia requiring
                         respiratory system,      hospitalization.
                         congenital cystic lung,  
                         anomalies of the lungs   
                         and accessory lobes,     
                         anomalies of respiratory 
                         system, including        
                         mediastinal cysts and    
                         pleural anomalies,       
                         tracheoesophageal        
                         fistula, esophageal      
                         atresia and stenosis,    
                         ciliary dismotility      
                         syndrome and vascular    
                         ring/sling.              
  ---------------------- ------------------------ ------------------------

***Changes Implemented to Pediatric Indicator as a Result of Pediatric
Panel Review***

  ---------------------- ------------------------ ------------------------
  **Pre-panel            **Post-panel indicator   **Reason implemented**
  definition**           definition**             

  No additional.                                  
  ---------------------- ------------------------ ------------------------

Changes considered, but not implemented

  ----------------------- ----------------------- -----------------------
  **AHRQ QI definition**  **Pediatric indicator   **Reason not
                          definition**            implemented**

  None.                                           
  ----------------------- ----------------------- -----------------------

***Clinical rationale***

This indicator is intended to capture cases of hospitalization, where
bacterial pneumonia is identified as the primary reason for the
hospitalization. Bacterial pneumonia is for the most part treatable with
antibiotics, and timely and appropriate treatment may reduce the need
for hospitalization. Patients who are more likely to develop
complications requiring hospitalization despite good quality outpatient
care are excluded, including those with sickle cell diseases, cystic
fibrosis, immunocompromised state, and anomalies of the respiratory
system.

This indicator was developed as part of the Prevention Quality Indicator
measure set, and is adapted from an indicator developed by Weissman et
al. ^31^ after favorable evaluation by a physician panel.

***Results of pediatric clinician panel review***

This indicator was not reviewed as part of our AHRQ QI indicator
development process, since it is a Prevention Quality Indicator with a
strong evidentiary base in the literature.

As part of the current pediatric indicator development process, this
indicator was reviewed by a panel of eleven pediatric clinicians,
including two ambulatory care pediatricians, one ambulatory care
pediatric nurse practitioner, one family practitioner, one pediatric
hospitalist, one pediatric emergency medicine physician, two pediatric
pulmonologists, one pediatric endocrinologist, and two pediatric
surgeons. The panel reviewed several other indicators. In the course of
review the panel suggested the following:

-   Panelists expressed concern that certain patients may be less likely
    to seek timely care regardless of access to quality care. These
    patients may present with advanced disease. Panelists argued, as for
    all potentially preventable hospitalizations, that this indicator be
    adjusted for socioeconomic status and that differences in cultural
    groups be considered when analyzing results.

-   Panelists also noted that areas with hospitals that have short stay
    units or similar practice patterns (e.g. holding patients in the ER
    instead of admitting) may appear to have lower rates without
    actually having higher quality of care. Given data limitations, no
    changes to the indicator definition could be made to address this
    issue. However, users of the indicator could explore admitting
    patterns with additional data.

-   Ideally, bacterial pneumonia could easily be distinguished from
    viral pneumonia in pediatric patients, since viral pneumonia is
    largely seen as less preventable. However, since it is not standard
    practice to culture the respiratory tract in children (due to the
    difficulty of obtaining material), this indicator will invariably
    pick up some viral pneumonias in addition to unspecified and
    specified bacterial pneumonia.

**^Post-conference\ call\ panel\ ratings\ --\ Pneumonia^**

  --------------------------- ------------- --------------------------------
  **Question**                **Median**    **Agreement status**

  Overall rating -- internal  6             Disagreement
  QI area                                   

  Overall rating --           6             Disagreement
  comparative purposes                      

  Access to quality outpt     6.5           Indeterminate agreement
  care                                      

  Charting by physicians      7             Indeterminate agreement

  Lack of bias                5             Indeterminate agreement

  Final recommendation        Not           
                              recommended   
                              for internal  
                              QI or         
                              comparative   
                              purposes.     
  --------------------------- ------------- --------------------------------

***Literature based evidence***

We found little literature on admission for pneumonia as an indicator of
access to quality outpatient care. Millman et al.^34^ reported that
low-income zip codes had 5.4 times more pneumonia hospitalizations per
capita (age 0-64) than high-income zip codes in 11 states in 1988.
Billings et al.^30^ found that low-income zip codes in New York City
(where at least 60% of households earned less than \$15,000 in 1988,
based on adjusted 1980 Census data) had 5.4 times more pneumonia
hospitalizations per capita (age 0-64) than high-income zip codes (where
less than 17.5% of households earned less than \$15,000). Household
income explained 53% of the variation in pneumonia hospitalization rates
at the zip code level. In a Swedish study using hospital episode
statistics and population census data, deprivation was associated with
increased admission rates for all respiratory infections and all
age-groups. The greatest effect was among those 0-4 years of age, who
had admission rates 91% higher in the most deprived areas compared to
the least deprived.^50^These findings suggest that this indicator may be
marker for poor access to outpatient care.

Numerous studies among adults have shown that influenza vaccination and
pneumoccccal vaccination reduce hospitalization rates for pneumonia and
influenza.^50-56^ We are not aware of any evidence linking reduced
pneumonia hospitalization rates among children to specific improvements
in the process of care, although it is certainly plausible that timely
initiation of outpatient antibiotics may obviate the need for
hospitalization. Supportive evidence comes from Washington,^57^ who
found that African-American children admitted to US hospitals for
pneumonia were less likely to require bronchoscopy or mechanical
ventilation, and hence less sick at presentation, than white children.

+-----------------------------------------------------------------------+
| ### PERFORATED APPENDIX ADMISSION RATE (PQI)                          |
|                                                                       |
| (AREA LEVEL INDICATOR)                                                |
+-----------------------------------------------------------------------+
| **Indicator definition:**                                             |
|                                                                       |
| Number of patients admitted for perforated appendix (see definition   |
| and exclusions below) per 100 admissions for appendicitis within an   |
| area. See The Pediatric Quality Indicator Technical Specifications.   |
+-----------------------------------------------------------------------+
| **Included admissions**:                                              |
+-----------------------------------------------------------------------+
| ***All patients 1-17 years old with any diagnosis code for            |
| perforation or abscess of appendix.***                                |
|                                                                       |
| -   Acute appendicitis with generalized peritonitis \[540.0\]         |
|                                                                       |
| -   Acute appendicitis with peritoneal abscess \[540.1\]              |
|                                                                       |
| ***Exclude patients transferring from another institution, MDC 14     |
| (pregnancy, childbirth, and puerperium), or MDC 15 (newborns and      |
| neonates).***                                                         |
+-----------------------------------------------------------------------+
| **Rates based on year 2003 KIDs' Inpatient Sample(per 100)**:         |
|                                                                       |
|   ------                                                              |
| ------------ ------- ------------------------------------------------ |
|   **OVERALL**        31.36                                            |
|                                                                       |
|   **Age stratified                                                    |
|   rates:**                                                            |
|                                                                       |
|   **1 -- 2 years**   69.21                                            |
|                                                                       |
|   **3 -- 5 years**   51.79                                            |
|                                                                       |
|   **6 -- 12 years**  31.90                                            |
|                                                                       |
|   **13 -- 17 years** 25.53                                            |
|                                                                       |
|   **Rates by type of                                                  |
|   hospital:**                                                         |
|                                                                       |
|   **Children's               39.25                                    |
|   hospitals**                                                         |
|                                                                       |
|   **Non-children's           28.61                                    |
|   hospitals**                                                         |
|                                                                       |
|   **Unknown**                34.34                                    |
|   ------                                                              |
| ------------ ------- ------------------------------------------------ |
+-----------------------------------------------------------------------+

***Status summary.*** Based on the current evidence base, from the
pediatric literature review, pediatric panel review, and empirical
analyses, this indicator is recommended for inclusion in the pediatric
quality indicator set. Panelists rated the indicator favorably with
indeterminate agreement both internal quality improvement within an area
and for comparative purposes.

***Changes from AHRQ QI Implemented Prior to Pediatric Panel Review***

  ---------------------- ------------------------ ------------------------
  **AHRQ QI definition** **Pediatric indicator    **Reason implemented**
                         definition**             

  All ages               Age 1-- 17               Pediatric age range.
                                                  Lower age range set to 1
                                                  year, given difficulty
                                                  in diagnosing
                                                  appendicitis in infants.
  ---------------------- ------------------------ ------------------------

***Changes Implemented to Pediatric Indicator as a Result of Pediatric
Panel Review***

  ---------------------- ------------------------ ------------------------
  **Pre-panel            **Post-panel indicator   **Reason implemented**
  definition**           definition**             

  No additional.                                  
  ---------------------- ------------------------ ------------------------

Changes considered, but not implemented

  ----------------------- ----------------------- -----------------------
  **AHRQ QI definition**  **Pediatric indicator   **Reason not
                          definition**            implemented**

  None.                                           
  ----------------------- ----------------------- -----------------------

***Clinical rationale***

This indicator is intended to identify cases of perforated appendix.
Timely identification of appendicitis may avert perforation.

This indicator was developed as part of the Prevention Quality Indicator
measure set, and is adapted from an indicator developed by Weissman^31^
and colleagues after favorable evaluation by a physician panel.

***Results of pediatric clinician panel review***

This indicator was not reviewed as part of our all-age indicator
development process, since it is a Prevention Quality Indicator with a
strong evidentiary base in the literature.

As part of the current pediatric indicator development process, this
indicator was reviewed by a panel of eleven pediatric clinicians,
including two ambulatory care pediatricians, one ambulatory care
pediatric nurse practitioner, one family practitioner, one pediatric
hospitalist, one pediatric emergency medicine physician, two pediatric
pulmonologists, one pediatric endocrinologist, and two pediatric
surgeons. The panel reviewed several other indicators. In the course of
review the panel suggested the following:

-   Panelists expressed concern that certain patients may be less likely
    to seek timely care regardless of access to quality care. These
    patients may present with rupture. Panelists argued, as for all
    potentially preventable hospitalizations, that this indicator be
    adjusted for socioeconomic status and that differences in cultural
    groups be considered when analyzing results.

**^Post-conference\ call\ panel\ ratings^**

  --------------------------- ------------- --------------------------------
  **Question**                **Median**    **Agreement status**

  Overall rating -- internal  7             Indeterminate agreement
  QI area                                   

  Overall rating --           7             Indeterminate agreement
  comparative purposes                      

  Access to quality           6             Indeterminate agreement
  outpatient care                           

  Charting by physicians      8             Agreement

  Lack of bias                7             Indeterminate agreement

  Final recommendation        Internal QI:  
                              Acceptable    
                              (-)           
                              Comparative   
                              purposes:     
                              Acceptable    
                              (-)           
  --------------------------- ------------- --------------------------------

***Literature based evidence***

In the seminal study of this topic, Braveman et al. examined the
likelihood of ruptured appendix among adults 18 to 64 years old who were
hospitalized for acute appendicitis in California from 1984 to 1989.
After controlling for age, sex, psychiatric diagnoses, substance abuse,
diabetes, poverty, race or ethnic group, and hospital characteristics,
ruptured appendix was more likely among both Medicaid-covered and
uninsured patients with appendicitis than among patients with private
capitated coverage (OR 1.49 and 1.46, 1.39 to 1.54, respectively).^58^

Several more recent studies have focused on the pediatric population.
Using hospital discharge data from Washington state, Bratton et al.
found that the risk-adjusted odds ratio for complicated disease
(perforation or peritoneal abscess) among children with Medicaid as the
primary payer was 1.3 (95% CI: 1.2-1.4). The risk of complicated disease
for children without any medical insurance was not significantly
elevated. Children who received care in centers with \>10 000 annual
admissions had a 1.8-fold increased odds of perforation, compared with
children treated at smaller facilities. Patients initially managed in
the emergency department were less likely to have complicated disease,
compared with children who were referred from an office practice (OR:
0.7; 95% CI: .7-.8).^59^ This last finding was confirmed by a study of
both children and adults from San Diego, which reported that patients
with appendicitis directly admitted from outpatient sources were 1.62
(95% CI: 1.28-2.05) times more likely to have rupture than were those
admitted through the hospital ED.^60^ Guagliardo et al. analyzed acute
appendicitis cases from California and New York (4-18 years of age) and
identified several independent risk factors for rupture in California:
Hispanic ethnicity (OR=1.30, 95% CI: 1.14-1.48), public insurance
(OR=1.29, 95% CI: 1.14--1.46), self-pay (OR=1.36, 95% CI: 1.07--1.74),
median zip code \<\$25,000 (OR=1.22, 95% CI: 1.03--1.45), and non-ED
referral (OR=1.15, 95% CI: 1.02--1.30). In New York, Hispanic ethnicity,
insurance, and low income were not associated with rupture, but
African-American race and non-ED referral were associated with rupture
(OR=1.28, 95% CI: 1.05--1.57).^61^ Finally, Ponsky et al. reviewed data
on children aged 5 to 17 years from 36 pediatric hospitals, and found
that Asian and black children were more likely to have appendiceal
rupture than white children (OR=1.66, 95% CI: 1.24-2.23; OR=1.13, 95%
CI, 1.01-1.30). Children with public insurance had a greater risk of
rupture than children with private insurance (OR=1.48; 95% CI
1.34-1.64), as did children who were self-insured (OR=1.36; 95% CI,
1.22-1.53). Hospital experience, defined by the volume of appendectomies
performed, was not associated with appendiceal rupture rate (*r* = 0.03;
*P* = .86) regardless of adjustments for race, sex, age, and insurance
status.^62^

Another study in a pediatric population examined reasons for delay to
surgery and insurance status in a New York pediatric population through
retrospective chart review. They noted that Medicaid or uninsured
children had both a higher perforation rate and a longer duration of
symptoms before presenting to a health care professional as compared to
HMO or private fee for service insured children. There were no
differences between the types of insurance in the time to surgery after
presentation.^63^ Unfortunately the authors did not analyze how much of
the variance in perforated appendix could be explained by delays in
seeking care. Based on Maryland Medicaid claims and hospital discharge
data for children from 1989 to 1994, the probability of ruptured
appendicitis was inversely related to age (OR = 0.86, 95% CI 0.81-0.91),
white race (OR = 0.35, 95% CI 0.17-0.71) and preventive care visits (OR
= 0.19, 95% CI 0.05-0.77).^64^ In this model, the number of preventive
care visits may serve as a marker for access to care.

Weissman et al., in their analysis of avoidable hospitalizations, found
that the uninsured had a relative risk of 1.14-1.20 of admission for
ruptured appendix after adjusting for age and sex (age 0-64). Medicaid
patients had a relative risk of .45-.58, suggesting that in at least
this case, Medicaid patients are not at increased risk for ruptured
appendix.^31^

+-----------------------------------------------------------------------+
| ### URINARY TRACT INFECTION ADMISSION RATE (PQI)                      |
|                                                                       |
| (AREA LEVEL INDICATOR)                                                |
+-----------------------------------------------------------------------+
| **Indicator definition:**                                             |
|                                                                       |
| Number of patients admitted for urinary tract infection (see          |
| definition and exclusions below) per 100,000 population. See The      |
| Pediatric Quality Indicator Technical Specifications.                 |
+-----------------------------------------------------------------------+
| **Included admissions**:                                              |
+-----------------------------------------------------------------------+
| ***All patients 3 mo. - 17 years old with a principal diagnosis code  |
| for urinary tract infection.***                                       |
|                                                                       |
| -   Chronic pyelonephritis without lesion of renal medullary necrosis |
|     \[590.00\]                                                        |
|                                                                       |
| -   Chronic pyelonephritis with lesion of renal medullary necrosis    |
|     \[590.01\]                                                        |
|                                                                       |
| -   Acute pyelonephritis without lesion of renal medullary necrosis   |
|     \[590.10\]                                                        |
|                                                                       |
| -   Acute pyelonephritis with lesion of renal medullary necrosis      |
|     \[590.11\]                                                        |
|                                                                       |
| -   Renal and pernephric abscess \[590.2\]                            |
|                                                                       |
| -   Pyeloureteritis cystica \[590.3\]                                 |
|                                                                       |
| -   Pyelonephritis, unspecified \[590.80\]                            |
|                                                                       |
| -   Pyelitis or pyelonephritis in diseases classified elsewhere       |
|     \[590.81\]                                                        |
|                                                                       |
| -   Infection of kidney, unspecified \[590.9\]                        |
|                                                                       |
| -   Acute cystitis \[595.0\]                                          |
|                                                                       |
| -   Cystitis, unspecified \[595.9\]                                   |
|                                                                       |
| -   Urinary tract infection, site not specified \[599.0\]             |
|                                                                       |
| ***Exclude patients transferring from another institution, MDC 14     |
| (pregnancy, childbirth, and puerperium), or MDC 15 (newborns and      |
| neonates)***                                                          |
|                                                                       |
| ***Exclude patients with any diagnosis of immunocompromised state     |
| (ie. Organ transplant, bone marrow or stem cell transplant, HIV or    |
| AIDs, humoral immunodeficiencies, deficiencies of cell-mediated       |
| immunity, other specified and unspecified immunodeficiency) or        |
| kidney/urinary tract disorders (e.g. chronic pyelonephritis,          |
| vesicoureteral reflux, congenital anomalies of urinary system, renal  |
| agenesis or dysgenesis, cystic kidney disease, exstrophy of bladder,  |
| atresia and stenosis of bladder neck, obstructive defects of renal    |
| pelvis and ureter, other anomalies of urinary system).***             |
+-----------------------------------------------------------------------+
| **Rates based on year 2003 KIDs' Inpatient Sample(per 100,000):**     |
|                                                                       |
|   -------------- ---------------------------------------------------  |
|   **OVERALL**    52.28                                                |
|                                                                       |
|   **Age                                                               |
|   stratified                                                          |
|   rates:**                                                            |
|                                                                       |
|   **3 mo. -- \<  410.34                                               |
|   1 year**                                                            |
|                                                                       |
|   **1 -- 2       74.50                                                |
|   years**                                                             |
|                                                                       |
|   **3 -- 5       38.61                                                |
|   years**                                                             |
|                                                                       |
|   **6 -- 12      26.16                                                |
|   years**                                                             |
|                                                                       |
|   **13 -- 17     34.93                                                |
|   years**                                                             |
|   -------------- ---------------------------------------------------  |
+-----------------------------------------------------------------------+

***Status summary.*** Based on the current evidence base, from the
pediatric literature review, pediatric panel review, and empirical
analyses, this indicator is recommended for inclusion in the pediatric
quality indicator set. Panelists rated the indicator favorably with
indeterminate agreement for internal quality improvement within an area
but rated the indicator less favorably for comparative purposes.

***Changes from AHRQ QI Implemented Prior to Pediatric Panel Review***

  ---------------------- ------------------------ ------------------------
  **AHRQ QI definition** **Pediatric indicator    **Reason implemented**
                         definition**             

  All ages               Age 3 mo. -- 17          Pediatric age range. Age
                                                  range raised to 3 months
                                                  to reflect standard
                                                  practice of admitting
                                                  young infants.

  All patients included. Exclude patients with    Patients are at higher
                         any diagnosis of         risk for developing
                         immunocompromised state  complications with
                         or kidney/urinary tract  urinary tract infection
                         disorders (e.g. chronic  requiring
                         pyelonephritis,          hospitalization.
                         vesicoureteral reflux,   
                         congenital anomalies of  
                         urinary system, renal    
                         agenesis or dysgenesis,  
                         cystic kidney disease,   
                         exstrophy of bladder,    
                         atresia and stenoosis of 
                         bladder neck,            
                         obstructive defects of   
                         renal pelvis and ureter, 
                         other anomalies of       
                         urinary system).         
  ---------------------- ------------------------ ------------------------

***Changes Implemented to Pediatric Indicator as a Result of Pediatric
Panel Review***

  ---------------------- ------------------------ ------------------------
  **Pre-panel            **Post-panel indicator   **Reason implemented**
  definition**           definition**             

  No additional.                                  
  ---------------------- ------------------------ ------------------------

Changes considered, but not implemented

  ----------------------- ----------------------- -----------------------
  **AHRQ QI definition**  **Pediatric indicator   **Reason not
                          definition**            implemented**

  None.                                           
  ----------------------- ----------------------- -----------------------

***Clinical rationale***

This indicator is intended to identify hospitalizations for urinary
tract infection, where UTI is identified as the principal reason for
hospitalization. Many cases of UTI can be treated in an outpatient
setting effectively with early identification and appropriate antibiotic
treatment, and will not progress to pyelonephritis. Patients who are
more likely to develop complications requiring hospitalization despite
good quality outpatient care are excluded, including those with,
immunocompromised state, and anomalies of the urinary tract and kidneys.

This indicator was developed as part of the Prevention Quality Indicator
measure set, and is adapted from an indicator developed by John
Billings^30^ and Weissman^31^ and colleagues after favorable evaluation
by a physician panel.

***Results of pediatric clinician panel review***

This indicator was not reviewed as part of our AHRQ QI indicator
development process, since it is a Prevention Quality Indicator with a
strong evidentiary base in the literature.

As part of the current pediatric indicator development process, this
indicator was reviewed by a panel of eleven pediatric clinicians,
including two ambulatory care pediatricians, one ambulatory care
pediatric nurse practitioner, one family practitioner, one pediatric
hospitalist, one pediatric emergency medicine physician, two pediatric
pulmonologists, one pediatric endocrinologist, and two pediatric
surgeons. The panel reviewed several other indicators. In the course of
review the panel suggested the following:

-   Panelists expressed concern that certain patients may be less likely
    to seek timely care regardless of access to quality care. These
    patients may present with advanced disease. Panelists argued, as for
    all potentially preventable hospitalizations, that this indicator be
    adjusted for socioeconomic status and that differences in cultural
    groups be considered when analyzing results.

-   Panelists also noted that areas with hospitals that have short stay
    units or similar practice patterns (e.g. holding patients in the ER
    instead of admitting) may appear to have lower rates without
    actually having higher quality of care. Given data limitations, no
    changes to the indicator definition could be made to address this
    issue. However, users of the indicator could explore admitting
    patterns with additional data.

-   Panelists noted that practice patterns regarding evaluation for
    causative factors such as urinary tract malformations vary from
    hospital to hospital and may affect rates. Some hospitals always
    evaluate patients in-hospital, and when excludable abnormalities are
    found, these patients will be excluded. In other areas, this
    evaluation is done on an outpatient basis and therefore the patient
    will be included in the indicator, despite having an excludable
    comorbidity.

**^Post-conference\ call\ panel\ ratings\ --\ UTI^**

  --------------------------- ------------- --------------------------------
  **Question**                **Median**    **Agreement status**

  Overall rating -- internal  7             Indeterminate agreement
  QI area                                   

  Overall rating --           6             Indeterminate agreement
  comparative purposes                      

  Access to quality outpt     6             Indeterminate agreement
  care                                      

  Charting by physicians      7             Indeterminate agreement

  Lack of bias                5             Indeterminate agreement

  Final recommendation        Internal QI:  
                              Acceptable    
                              (-)           
                              Comparative   
                              purposes: Not 
                              recommended   
  --------------------------- ------------- --------------------------------

***Literature based evidence***

We found little literature on admission for urinary infection as an
indicator of access to quality outpatient care. Millman et al.^34^
reported that low-income zip codes had 2.8 times more UTI
hospitalizations per capita (age 0-64) than high-income zip codes in 11
states in 1988. Billings et al.^30^ found that low-income zip codes in
New York City (where at least 60% of households earned less than
\$15,000 in 1988, based on adjusted 1980 Census data) had 2.2 times more
UTI hospitalizations per capita (age 0-64) than high-income zip codes
(where less than 17.5% of households earned less than \$15,000).
Household income explained 28% of the variation in UTI hospitalization
rates at the zip code level. These findings suggest that this indicator
may be marker for poor access to outpatient care.

Although there is ample literature indicating that most adolescents and
adults with urinary tract infections can be safely managed with
outpatient antibiotics, we are not aware of any evidence linking reduced
UTI hospitalization rates among children to specific improvements in the
process of care.

+-----------------------------------------------------------------------+
| ### CRANIOTOMY MORTALITY RATE (IQI)                                   |
+-----------------------------------------------------------------------+
| **Indicator definition:**                                             |
|                                                                       |
| Number of deaths per 100 patients undergoing craniotomy. See The      |
| Pediatric Quality Indicator Technical Specifications.                 |
+-----------------------------------------------------------------------+
| **Included procedures**:                                              |
+-----------------------------------------------------------------------+
| ***All patients 0-17 years old in a craniotomy DRG.***                |
|                                                                       |
| -   Craniotomy, Age 0-17 \<003\>                                      |
|                                                                       |
| -   Intracranial Vascular Procedure with Principal Diagnosis of       |
|     Hemorrhage \<528\>                                                |
|                                                                       |
| -   Ventricular Shunt Procedures with Complications and Comorbidities |
|     \<529\>                                                           |
|                                                                       |
| -   Ventricular Shunt Procedures without Complications and            |
|     Comorbidities \<530\>                                             |
|                                                                       |
| -   Craniotomy with Implantation of Chemotherapeutic Agent or Acute   |
|     Complex Central Nervous System Principal Diagnosis \<543\>        |
|                                                                       |
| ***Exclude patients transferring to another institution, MDC 14       |
| (pregnancy, childbirth, and puerperium), or MDC 15 (newborns and      |
| neonates).***                                                         |
|                                                                       |
| ***Exclude patients with a principal diagnosis of head trauma.***     |
|                                                                       |
| ***Exclude newborns with a birthweight of less than 500 grams.***     |
|                                                                       |
| ***Stratify into major risk groups by type of surgery: 1) Major       |
| craniotomies (tumors, epilepsy, vascular malformation and aneurysms)  |
| 2) craniosynostosis, 3) hydrocephalus (endoscopic third               |
| ventriculostomies, shunt procedure), 4) Chiari malformations***       |
+-----------------------------------------------------------------------+
| **Rates based on year 2003 KIDs' Inpatient Sample(per 100)\***        |
|                                                                       |
|   -------------- ---------------------------------------------------  |
|   **OVERALL**                                                         |
|                                                                       |
|   **Age                                                               |
|   stratified                                                          |
|   rates:**                                                            |
|                                                                       |
|   **Neonate, \<                                                       |
|   2000g**                                                             |
|                                                                       |
|   **Neonate, ≥                                                        |
|   2000g**                                                             |
|                                                                       |
|   **29 days --                                                        |
|   364 days**                                                          |
|                                                                       |
|   **1 -- 2                                                            |
|   years**                                                             |
|                                                                       |
|   **3 -- 5                                                            |
|   years**                                                             |
|                                                                       |
|   **6 -- 12                                                           |
|   years**                                                             |
|                                                                       |
|   **13 -- 17                                                          |
|   years**                                                             |
|   -------------- ---------------------------------------------------  |
+-----------------------------------------------------------------------+

***Status summary.*** Based on the current evidence base, from the
pediatric literature review, pediatric panel review, and empirical
analyses, this indicator is recommended for inclusion in the pediatric
quality indicator set. However, further redefinition and consultation
with specialists is required before this indicator can be implemented.
Panelists rated this indicator favorably, with agreement for both for
quality improvement and comparative uses.

***Changes from AHRQ QI Implemented Prior to Pediatric Panel Review***

  ---------------------- ------------------------ ------------------------
  **AHRQ QI definition** **Pediatric indicator    **Reason implemented**
                         definition**             

  All ages               Age 0 -- 17              Pediatric age range
  ---------------------- ------------------------ ------------------------

***Changes Implemented to Pediatric Indicator as a Result of Pediatric
Panel Review***

  ---------------------- ------------------------ ------------------------
  **Pre-panel            **Post-panel indicator   **Reason implemented**
  definition**           definition**             

  All craniotomies       Stratify into major risk Risk of mortality varies
  examined together.     groups by type of        greatly by type of
                         surgery: 1) Major        procedure.
                         craniotomies (tumors,    
                         epilepsy, vascular       
                         malformation and         
                         aneurysms) 2)            
                         craniosynostosis, 3)     
                         hydrocephalus            
                         (endoscopic third        
                         ventriculostomies, shunt 
                         procedure), 4) Chiari    
                         malformations.           
  ---------------------- ------------------------ ------------------------

Changes considered, but not implemented

  ----------------------- ----------------------- -----------------------
  **AHRQ QI definition**  **Pediatric indicator   **Reason not
                          definition**            implemented**

  None.                                           
  ----------------------- ----------------------- -----------------------

***Clinical rationale***

This indicator was developed as part of the Inpatient Quality Indicator
measure set. The indicator includes all DRGs for craniotomy in children,
and excludes head trauma patients, as in previous coding conventions
these patients were assigned to separate DRGs. This exclusion maintains
consistency in the denominator group over time.

***Results of pediatric clinician panel review***

This indicator was not reviewed as part of our AHRQ QI indicator
development process, since it is an Inpatient Quality Indicator with a
strong evidentiary base in the literature.

As part of the current pediatric indicator development process, this
indicator was reviewed by a panel of twelve pediatric clinicians, one
pediatric critical care specialist, one pediatric hospitalist, one
pediatric anesthesiologist, one pediatric hematologist/oncologist, one
pediatric cardiologist, two pediatric surgeons, one pediatric
neurosurgeon, one pediatric urologist, two pediatric cardiovascular
surgeons, and one neonatologist. The panel reviewed several other
indicators. In the course of review the panel suggested the following:

-   At the onset of the review, this indicator was not stratified, and
    all patients undergoing any type of craniotomy were combined into
    one rate. Panelists argued that in order for this indicator to be
    fair due to case mix differences and useful for quality improvement
    the indicator should be stratified by four major risk groups, as
    outlined above. Panelists felt that this would be more informative,
    since different craniotomy procedures have vastly different risks
    for mortality.

-   Panelists noted that risk adjustment is an important factor for this
    indicator. We will apply a general risk adjustment derived from
    administrative data, although it will likely not be tailored
    specifically to this indicator.

**^Post-conference\ call\ panel\ ratings^**

  --------------------------- ------------- --------------------------------
  **Question**                **Median**    **Agreement status**

  Overall rating -- internal  8             Agreement
  QI                                        

  Overall rating -            8             Agreement
  comparative                               

  Not present on admission    N/A           N/A

  Preventability              7             Indeterminate agreement

  Due to medical error        5.5           Indeterminate agreement

  Charting by physicians      9             Agreement

  Lack of bias                5.5           Indeterminate agreement

  Final recommendation        Internal QI:  
                              Acceptable    
                              (+)           
                              Comparative   
                              purposes:     
                              Acceptable    
                              (-)           
  --------------------------- ------------- --------------------------------

***Empirical analyses to inform indicator definition***

The following empirical analyses aided in formulating the definition for
this indicator. Analyses were conducted using the 2000 Nationwide
Inpatient Sample.

This indicator includes procedures of varying complexity and risk. To
better understand the breakdown we conducted a series of analysis. First
we examined the risk of mortality for procedures associated with the DRG
003 (the only DRG in use in 2000 for the pediatric population). As
anticipated, mortality rates varied widely, from no mortality for
relatively simple procedures, to over 18% mortality. Also of note is
that DRG 3 had a much higher % of ventricular shunt placements and
revisions (25% in DRG3 versus 7% in DRG1 and 0.6% in DRG2).  These
procedures have lower mortality, in general, than other craniotomy
procedures, especially in the pediatric population (just 0.77%). This
analysis underscored the need for risk adjustment beyond the basic DRG
and existing comorbidity adjustment for this indicator (in adults this
adjustment is accomplished using APR-DRGs).

Panelists suggested stratification as one technique to account for
different risk. Analyses were undertaken to explore the correct
stratification. First we explored morality rates for pediatric
craniotomy by principal diagnosis (4-digit) and procedure code,
excluding procedures with fewer than 20 cases in the denominator. Due to
the relative infrequency of some procedures we ran this analysis using
three years of NIS data.

An attempt to classify cases according to risk strata included the
following classification

1.  any procedure with fewer than 5 cases in the three-year window
    (2000-02) was assigned to risk category 0

2.  any procedure with no deaths was assigned to risk category 0

3.  procedures with at least 5 cases and at least 1 death were assigned
    to risk categories 1-4 based on relative risk of mortality (roughly
    four equal groups)

4.  any diagnosis-procedure combination with at least 20 cases was
    re-assigned to a new risk category if the relative risk was higher
    or lower than the original risk category (if the risk category was
    the same then the diagnosis was ignored)

5.  In some cases diagnosis codes resulted in re-assignment. For
    example, procedure code 01.18 OTHER BRAIN DX PROCEDURE is assigned
    to risk category 4, unless it is associated with diagnosis codes
    348.2x PSEUDOTUMOR CEREBRI or 996.2x MALFUN NEURO DEVICE/GRAF, in
    which case it is re-assigned to risk category 3.

A second set of groupings was calculated using specific diagnosis and
procedure codes, based on panelist recommendations and a review of the
ICD-9-CM codes.

For each stratum and each procedure code the raw rate and relative risk
of mortality was calculated.

As a result of these analyses we found that the strata recommended by
the panelists were too broad to differentiate risk of mortality. Further
analyses and consultation with specialists will be required to define
strata.

***Literature based evidence***

Most of the evidence for this indicator is based on several studies in
adult populations. These studies found that providers who perform more
than 30 procedures annually have lower mortality than those performing
fewer procedures.^65,\ 66^ In another study, adult patients who were
referred to a large medical center for treatment of subarachnoid
hemorrhage were less likely to die early, younger, and had fewer severe
indications, including lower clinical grade, prevalence of coma, and
diastolic blood pressure.^67^

***Pediatric-specific evidence*.** Only one relevant study has focused
on the pediatric craniotomy population. A cross-sectional study of
pediatric craniotomies for brain tumors, based on the Nationwide
Inpatient Sample (i.e., administrative data) for 1998-2000, reported
that adjusted mortality was significantly lower at high-volume hospitals
than at low-volume hospitals (e.g., 2.3% at hospitals with 4 or fewer
annual admissions versus 1.4% at hospitals with more than 20 annual
admissions). There was a nonsignificant trend toward lower mortality
after surgery performed by high-volume surgeons.

Empirical analyses have shown that a disproportionate percentage of
children who undergo craniotomy have a primary indication of
hydrocephalus or another defect requiring shunt placement to relieve
intracranial pressure. Shunt placement is associated with a
substantially lower probability of post-craniotomy mortality, such that
it accounts for about 25% of all craniotomies, but only about 4% of
post-craniotomy deaths.

+-----------------------------------------------------------------------+
| ### PEDIATRIC HEART SURGERY MORTALITY RATE (IQI)                      |
+-----------------------------------------------------------------------+
| **Indicator definition:**                                             |
|                                                                       |
| Number of in-hospital deaths in patients undergoing surgery for       |
| congenital heart disease per 100 patients. See The Pediatric Quality  |
| Indicator Technical Specifications.                                   |
+-----------------------------------------------------------------------+
| **Included admissions**:                                              |
+-----------------------------------------------------------------------+
| Discharges with a procedure codes for surgical intervention for       |
| congenital heart disease in any field or non-specific heart surgery   |
| in any field with a diagnosis code of congenital heart disease in any |
| field.                                                                |
|                                                                       |
| Age less than 18 years old.                                           |
|                                                                       |
| Exclude:                                                              |
|                                                                       |
| a\. MDC 14 (pregnancy, childbirth and pueperium)                      |
|                                                                       |
| b\. patients with transcatheter interventions as single cardiac       |
| procedures, performed without bypass but with catheterization         |
|                                                                       |
| c\. patients with septal defects (4P) as single cardiac procedures    |
| without bypass                                                        |
|                                                                       |
| d\. heart transplant                                                  |
|                                                                       |
| e\. premature infants with PDA closure as only cardiac procedure      |
|                                                                       |
| f\. age less than 30 days with PDA closure as only cardiac procedure  |
|                                                                       |
| g\. missing discharge disposition                                     |
|                                                                       |
| h\. transferring to another short-term hospital                       |
+-----------------------------------------------------------------------+
| **Rates based on year 2003 KIDs' Inpatient Sample (per 1000)**:       |
|                                                                       |
|   -------------- ---------------------------------------------------  |
|   **OVERALL**    46.66                                                |
|                                                                       |
|   **Age                                                               |
|   stratified                                                          |
|   rates:**                                                            |
|                                                                       |
|   **Neonate, \<  152.70                                               |
|   2000g**                                                             |
|                                                                       |
|   **Neonate, ≥   144.74                                               |
|   2000g**                                                             |
|                                                                       |
|   **29 days --   39.09                                                |
|   364 days**                                                          |
|                                                                       |
|   **1 -- 2       21.14                                                |
|   years**                                                             |
|                                                                       |
|   **3 -- 5       12.53                                                |
|   years**                                                             |
|                                                                       |
|   **6 -- 12      11.77                                                |
|   years**                                                             |
|                                                                       |
|   **13 -- 17     9.57                                                 |
|   years**                                                             |
|   -------------- ---------------------------------------------------  |
+-----------------------------------------------------------------------+

***Status summary.*** Based on the current evidence base, from the
pediatric literature review, pediatric panel review, and empirical
analyses, this is recommended for inclusion in the pediatric indicator
set.

***Changes from AHRQ QI Implemented Prior to Pediatric Panel Review***

  ---------------------- ------------------------ ------------------------
  **AHRQ QI definition** **Pediatric indicator    **Reason implemented**
                         definition**             

  None.                                           
  ---------------------- ------------------------ ------------------------

***Changes Implemented to Pediatric Indicator as a Result of Pediatric
Panel Review***

  ---------------------- ------------------------ ------------------------
  **Pre-panel            **Post-panel indicator   **Reason implemented**
  definition**           definition**             

  No additional                                   
  ---------------------- ------------------------ ------------------------

Changes considered, but not implemented

  ----------------------- ----------------------- -----------------------
  **AHRQ QI definition**  **Pediatric indicator   **Reason not
                          definition**            implemented**

  None.                                           
  ----------------------- ----------------------- -----------------------

***Clinical rationale***

This indicator was developed as part of our Inpatient Quality Indicator
measure set and is based on an indicator developed by Kathy Jenkins and
colleagues. Dr. Jenkins developed this indicator based on physician
input and empirical analyses.^68^ Unlike other Inpatient Quality
Indicators, this indicator also includes a tailored risk adjustment
system, which estimates risk for patients based on procedure.

***Results of pediatric clinician panel review***

As part of the current pediatric indicator development process, this
indicator was reviewed by a panel of twelve pediatric clinicians, one
pediatric critical care specialist, one pediatric hospitalist, one
pediatric anesthesiologist, one pediatric hematologist/oncologist, one
pediatric cardiologist, two pediatric surgeons, one pediatric
neurosurgeon, one pediatric urologist, two pediatric cardiovascular
surgeons, and one neonatologist. The panel reviewed several other
indicators. In the course of review the panel suggested the following:

-   Panelists noted that the validity of this indicator lays primarily
    in the ability to risk adjust the measure. The panelists were
    presented with one system, the RACHS risk adjustment system,
    developed by Kathy Jenkins and colleagues, and panelists discussed
    the use of other clinically based systems, such as Aristotle. They
    recommended that the relative performance of risk adjustment
    feasible with administrative data be evaluated.

***Literature based evidence***

This indicator was developed as part of our Inpatient Quality Indicator
measure set and is based on an indicator developed by Kathy Jenkins and
colleagues. Dr. Jenkins developed this indicator based on physician
input and empirical analyses.^68^ Unlike other Inpatient Quality
Indicators, this indicator also includes a tailored risk adjustment
system, which estimates risk for patients based on procedure.

The evidence for the validity of this indicator comes from two sources.
First, three studies (including one that used prospectively collected
clinical data) have reported an association between hospital volume and
mortality following pediatric cardiac surgery. Using a multivariate
model that included age, complexity category, and four comorbidities,
Hannan et al.^69^ found 8.26% risk-adjusted mortality at hospitals with
fewer than 100 cases per year, versus 5.95% at higher volume hospitals
(an effect limited to surgeons who performed at least 75 cases per
year). Two other studies using hospital discharge data from California
and Massachusetts found similar effects of hospital volume .^68,\ 70^
The consistent association between volume and risk-adjusted mortality
supports the validity of both measures of performance, and is consistent
with the hypothesis that more experience leads to improved technical
skills and better outcomes. Other studies from single centers have
confirmed this hypothesis by demonstrating improvements in mortality
over time for a variety of procedures.^71-73^

The second source of evidence is that cardiopulmonary bypass or aortic
cross­clamp time has been repeatedly associated with postoperative
mortality, adjusting for a variety of patient characteristics.^74-77^
This relationship has been demonstrated not just for the Fontan
procedure, but also for the Norwood procedure for hypoplastic left heart
syndrome. ^78^ Experienced surgeons and surgical teams should be able to
reduce cardiopulmonary bypass or aortic cross-clamp time, thereby
improving postoperative mortality. It should be noted that patient-level
reduction in mortality does not necessarily correspond with
provider-level mortality. It is unknown how implementing these processes
of care would actually affect provider-level mortality rates.

# Conclusion

The dedicated research effort described in this report tailors the AHRQ
QIs specifically for pediatric populations, and offers an example of
developing pediatric indicators using routinely collected inpatient
data. These indicators will be released as the AHRQ Pediatric Quality
Indicator set to provide a tool for screening for quality of care for
the millions of hospitalized children each year, as well as for
assessing the rate of potentially preventable hospitalizations. The
potential uses of the indicators span many arenas, from public health to
internal quality improvement.

Consistent with previous indicator sets, each of the thirteen
provider-level indicators is particularly applicable to quality
improvement efforts. Hospitals may use existing data to identify
indicators with higher than expected rates, flagging potential quality
concerns. These areas of concern may be investigated further in order to
identify the underlying cause of the poorer than expected performance.
In some cases, incorrect coding practices may be identified, in other
cases closer examination of system-level factors may be in order.
Interventions may be devised to improve performance, and hospitals may
track their own performance over time to identify areas of improvement.

The ability to track quality of care for a wide range of patients is an
important consideration for quality improvement. Community hospitals,
who admit nearly 2/3 of pediatric cases, may not treat a substantial
number of patients with some specialized conditions. As a result,
indicators that only apply to such conditions (e.g., cancer,
cardiothoracic surgery, cystic fibrosis, neonatal surgeries) may not be
as useful for non-children's hospitals. All but two of the thirteen
selected provider-level measures are cross-cutting, applying to children
admitted for a variety of procedures and/or conditions. Pediatric heart
surgery mortality and volume may be more applicable to children's
hospitals than community hospitals, although some community hospitals
perform less complex heart surgeries. For indicators where hospital case
mix is expected to vary, stratification is available to allow a hospital
with a more complex case mix to examine rates by risk groups separately
and pinpoint quality concerns further.

Given the historical use of the AHRQ QIs, the provider-level indicators
are also likely to be used for inter-hospital comparisons. In
anticipation of this potential application, each indicator was assessed
for overall usefulness for two dimensions, internal quality improvement
and comparative purposes. Ten of the provider level indicators were
rated by panelists as useful for inter-hospital comparisons. These
ratings provide additional information to policy makers selecting
indicators for inter-hospital comparisons. Of course additional factors
may also influence the selection of indicators, and risk adjustment for
case mix will remain an important consideration.

Existing risk adjustment strategies for pediatric patients were not
suitable for use with the Pediatric QIs. Most available schemes apply to
specific clinical groups and utilize clinical data not available in
administrative databases. The APR-DRGs, used for risk adjustment for the
AHRQ Inpatient Quality Indicators, has considered pediatric populations
when developing algorithms. However, APR-DRGs are not suitable for
adjusting for complications since complications are part of the
adjustment algorithm (resulting in over-adjustment). As a result, we
investigated alternative risk adjustment strategies, and identified
three important risk adjustment factors: 1.) reason for admission
(including principal procedure) 2.) comorbidities and 3.) age and
gender. Using a modified-DRG risk adjustment combined with comorbidity
adjustment based on the AHRQ Clinical Classification System (CCS) and
age and gender adjustment, the AHRQ PedQIs include a novel and
specialized risk adjustment system. Using Present on Admission (POA)
data from New York and California, potential comorbidities were explored
to minimize bias from complications being mislabeled as comorbidities.
However, this system is only a first step in the development of
pediatric-specific risk adjustment. Both the DRG and CCS system may
prove to be too broad in some pediatric applications, grouping together
important co-morbidities or procedures with many low-risk conditions.

Another approach to accounting for case mix is stratification. The
original AHRQ QIs tended to use exclusion of high risk groups and risk
adjustment to account for difference in case mix. However, since
children are generally healthy, the high risk groups offer the best
option for intervention. Stratification allows hospitals to identify
which segment of the pediatric population accounts for any elevation in
rates, creating more user-friendly indicators. Tailored stratification
schemes are available for six complications indicators: Accidental
puncture and laceration, decubitus ulcer, iatrogenic pneumothorax,
postoperative hemorrhage and hematoma, postoperative sepsis, selected
infection due to medical care.

Despite these efforts to account for risk, we anticipate that further
research on pediatric risk adjustment will be important for targeting
quality improvement appropriately. Certainly no risk adjustment system
can account for all differences in risk and comparison between hospitals
must be pursued with this caveat. Comparisons between similar types of
hospitals, such as comparing tertiary care children's hospitals with
other children's hospitals, will further facilitate fair comparisons
between hospitals.

In addition to the provider-level indicators, the PedQIs also include
five area level indicators. These indicators track potentially
preventable hospitalizations, and allow policy makers to target specific
groups that appear to be developing more severe disease requiring
hospitalization. Higher than anticipated rates may reflect poor access
to care (e.g., from lack of insurance or too few primary care
physicians), barriers to timely care (e.g., clinics that require daytime
appointments), barriers to adherence to medical advice (e.g., language
barriers), cultural influences that preclude seeking early treatment, or
higher prevalence of poor health behaviors (e.g., smoking).
Interventions may address any of these factors.

Area level indicators are prone to bias due to cultural factors that may
be outside of a health systems control. For instance, an area with a
high number of illegal immigrants may have patients presenting with more
advanced disease, because patients delay seeking care for fear of
deportation. In addition, factors such as smoking or obesity may be more
prevalent in certain areas. Panelists felt that risk adjustment should
include these factors. Since we cannot directly adjust for these
factors, we have applied adjustment for socioeconomic status as a proxy.
However, risk adjustment for socioeconomic groups may mask true
differences in access to good quality care. For this reason it is
recommended that risk adjusted rates be considered alongside raw
unadjusted rates.

Future directions

The current PedQI indicator set and accompanying risk adjustment are
only the initial step in pediatric indicator development. These
indicators extend our previous indicator development efforts, but the
eighteen indicators do not address some important areas of inpatient
pediatric care, such as neonatal intensive care. A second phase of
development will examine novel indicators based on administrative data,
building from published literature and nominations from clinical and
other professional organizations.

Along with the expansion of the indicator set, the Pediatric QIs will
benefit from additional validation efforts. As the indicators are
utilized, needed improvements to the indicators will be illuminated.
Chart review efforts will provide better information on the sensitivity
and specificity of the indicators, and may guide further the most
appropriate applications of the indicators. Validation efforts may also
demonstrate the usefulness of the indicators for facilitating quality
improvement. Finally, further investigation and refinement of the risk
adjustment system will be essential both for quality improvement and
comparative reporting efforts.

Application of the indicator set requires high quality data. Currently
few data standards exist for pediatrics, and since pediatric data in
general does not fall under the auditing authority of the Centers for
Medicare and Medicaid Services (CMS), variation in coding practices is
of particular concern. Implementation of data standards for pediatrics
would aid in further development and utility of the AHRQ Pediatric QIs.
In addition, expansion of data sets to include data elements such as
"present on admission," linked data sets, or limited clinical data, such
as laboratory or pharmacy data, would also allow for improvement in the
sensitivity and specificity of existing indicators and the expansion of
the indicator set to include indicators targeted to important clinical
groups, such as asthma patients and special need children.

# References

1\. Owens P, Thompson J, Elixhauser A, et al. Care of children and
adolscents in us hospitals. Rockville, MD: Agency for Healthcare
Research and Quality; 2003. Report No.: HCUP Fact Book No. 4; AHRQ
Publication No. 04-0004.

2\. Beal AC, Co JP, Dougherty D, et al. Quality measures for children\'s
health care. Pediatrics 2004;113(1 Pt 2):199-209.

3\. Simpson LA, al DDe. Measures of children\'s health care quality:
Building towards consensus. Manuscript in preparation: Background paper
prepared for National Quality Forum; 2003 September 19.

4\. National Association of Children\'s Hospitals and Related
Institutions. In; 2005.

5\. Green L, Lewis F. Measurement and evaluation in health education and
health promotion. Mountain View, CA: Mayfield Publishing Company; 1998.

6\. Fitch K, Bernstein S, Aguilar M, et al. The rand/ucla
appropriateness method user\'s manual: RAND; 2001.

7\. Campbell SM, Roland MO, Quayle JA, et al. Quality indicators for
general practice: Which ones can general practitioners and health
authority managers agree are important and how useful are they? J Public
Health Med 1998;20(4):414-21.

8\. Campbell SM, Roland MO, Shekelle PG, et al. Development of review
criteria for assessing the quality of management of stable angina, adult
asthma, and non-insulin dependent diabetes mellitus in general practice.
Qual Health Care 1999;8(1):6-15.

9\. Sanderson C, Dixon J. Conditions for which onset or hospital
admission is potentially preventable by timely and effective ambulatory
care. J Health Serv Res Policy 2000;5(4):222-30.

10\. National healthcare quality report. In: Agency for Healthcare
Research and Quality; 2003.

11\. Miller MR, Zhan C. Pediatric patient safety in hospitals: A
national picture in 2000. Pediatrics 2004;113(6):1741-6.

12\. Sedman A, Harris JM, Schulz K, Schwalenstocker E, Remus D, Scanlon
M, Bahl V. Relevance of the agency for healthcare research and quality
patient safety indicators for children\'s hospitals. Pediatrics
2004;115(1):135-145.

13\. Lund C. Prevention and management of infant skin breakdown. Nursing
Clinics of North America 1999;34(4):907-920.

14\. Quigley S, Curley, MA. Skin integrity in the pediatrics population:
Preventing and managing pressure ulcers. Journal of the Society of
Pediatric Nursing 1996;1(1):7-18.

15\. Curley M, Quigley, SM, Lin, M. Pressure ulcers in pediatric
intensive care: Incidence and associated factors. Pediatric Critical
Care Medicine 2003;4(3):284-290.

16\. Curley M, Razmus, IS, Roberts, KE, Wypij, D. Predicting pressure
ulcer risk in pediatric patients - the branden q scale. Nursing Research
2003;52(1):22-33.

17\. Miller MR, Elixhauser A, Zhan C. Patient safety events during
pediatric hospitalizations. Pediatrics 2003;111(6 Pt 1):1358-66.

18\. Soll RF, Andruscavage L. The principles and practice of
evidence-based neonatology. Pediatrics 1999;103(1 Suppl E):215-24.

19\. Soll RF, Morley CJ. Prophylactic versus selective use of surfactant
in preventing morbidity and mortality in preterm infants.\[update of
cochrane database syst rev. 2000;(2):Cd000510; pmid: 10796379\].
Cochrane Database of Systematic Reviews 2001(2):CD000510.

20\. Walker MW, Shoemaker M, Riddle K, et al. Clinical process
improvement: Reduction of pneumothorax and mortality in high-risk
preterm infants. Journal of Perinatology 2002;22(8):641-5.

21\. Auten RL, Vozzelli M, Clark RH. Volutrauma. What is it, and how do
we avoid it? Clinics in Perinatology 2001;28(3):505-15.

22\. In: Personal communication. Work in progress.

23\. Elixhauser A. In; 2002.

24\. Derkay CS. A cost-effective approach for preoperative hemostatic
assessment in children undergoing adenotonsillectomy.\[see comment\].
Archives of Otolaryngology \-- Head & Neck Surgery 2000;126(5):688.

25\. Hartnick CJ, Ruben RJ. Preoperative coagulation studies prior to
tonsillectomy.\[see comment\]. Archives of Otolaryngology \-- Head &
Neck Surgery 2000;126(5):684-6.

26\. Miller MR. In; 2005.

27\. Richards MJ, Edwards JR, Culver DH, et al. Nosocomial infections in
pediatric intensive care units in the united states. National nosocomial
infections surveillance system. Pediatrics 1999;103(4):e39.

28\. O\'Grady NP, Alexander M, Dellinger EP, et al. Guidelines for the
prevention of intravascular catheter-related infections. The hospital
infection control practices advisory committee, center for disese
control and prevention, u.S. Pediatrics 2002;110(5):e51.

29\. Garland JS, Henrickson K, Maki DG, et al. The 2002 hospital
infection control practices advisory committee centers for disease
control and prevention guideline for prevention of intravascular
device-related infection. Pediatrics 2002;110(5):1009-13.

30\. Billings J, Zeital L, Lukomnik J, et al. Analysis of variation in
hospital admission rates associated with area income in new york city.
In: Unpublished Report.; 1992.

31\. Weissman JS, Gatsonis C, Epstein AM. Rates of avoidable
hospitalization by insurance status in massachusetts and maryland. Jama
1992;268(17):2388-94.

32\. Ray NF, Thamer M, Fadillioglu B, et al. Race, income, urbanicity,
and asthma hospitalization in california: A small area analysis. Chest
1998;113(5):1277-84.

33\. Gottlieb DJ, Beiser AS, O\'Connor GT. Poverty, race, and medication
use are correlates of asthma hospitalization rates. A small area
analysis in boston. Chest 1995;108(1):28-35.

34\. Millman M, editor. Committee on monitoring access to personal
health care services. Washington, D.C.: National Academy Press; 1993.

35\. Lin S, Fitzgerald E, Hwang SA, et al. Asthma hospitalization rates
and socioeconomic status in new york state (1987-1993). J Asthma
1999;36(3):239-51.

36\. Giuffrida A, Gravelle H, Roland M. Measuring quality of care with
routine data: Avoiding confusion between performance indicators and
health outcomes. Bmj 1999;319(7202):94-8.

37\. Carr W, Zeitel L, Weiss K. Variations in asthma hospitalizations
and deaths in new york city. Am J Public Health 1992;82(1):59-65.

38\. Halterman JS, Aligne CA, Auinger P, et al. Inadequate therapy for
asthma among children in the united states. Pediatrics 2000;105(1 Pt
3):272-6.

39\. Murray MD, Stang P, Tierney WM. Health care use by inner-city
patients with asthma. J Clin Epidemiol 1997;50(2):167-74.

40\. Lozano P, Connell FA, Koepsell TD. Use of health services by
african-american children with asthma on medicaid. Jama
1995;274(6):469-73.

41\. Bosco LA, Gerstman BB, Tomita DK. Variations in the use of
medication for the treatment of childhood asthma in the michigan
medicaid population, 1980 to 1986. Chest 1993;104(6):1727-32.

42\. Homer CJ, Szilagyi P, Rodewald L, et al. Does quality of care
affect rates of hospitalization for childhood asthma? Pediatrics
1996;98(1):18-23.

43\. Lieu TA, Quesenberry CP, Jr., Capra AM, et al. Outpatient
management practices associated with reduced risk of pediatric asthma
hospitalization and emergency department visits. Pediatrics
1997;100(3):334-41.

44\. Browne GJ, Giles H, McCaskill ME, et al. The benefits of using
clinical pathways for managing acute paediatric illness in an emergency
department. J Qual Clin Pract 2001;21(3):50-5.

45\. Woolcock AJ, Bastiampillai SA, Marks GB, et al. The burden of
asthma in australia. Med J Aust 2001;175(3):141-5.

46\. Greineder DK, Loane KC, Parks P. A randomized controlled trial of a
pediatric asthma outreach program. J Allergy Clin Immunol 1999;103(3 Pt
1):436-40.

47\. Lwebuga-Mukasa JS, Pszonak R. Patterns of inpatient and outpatient
care for asthma in erie and niagara counties, western new york state. J
Asthma 2001;38(2):155-60.

48\. Anis AH, Lynd LD, Wang XH, et al. Double trouble: Impact of
inappropriate use of asthma medication on the use of health care
resources. Cmaj 2001;164(5):625-31.

49\. Gracey M, Lee AH, Yau KK. Hospitalisation for gastroenteritis in
western australia. Arch Dis Child 2004;89(8):768-72.

50\. Christenson B, Hedlund J, Lundbergh P, et al. Additive preventive
effect of influenza and pneumococcal vaccines in elderly persons. Eur
Respir J 2004;23(3):363-8.

51\. Herzog NS, Bratzler DW, Houck PM, et al. Effects of previous
influenza vaccination on subsequent readmission and mortality in elderly
patients hospitalized with pneumonia. Am J Med 2003;115(6):454-61.

52\. Crocetti E, Arniani S, Bordoni F, et al. Effectiveness of influenza
vaccination in the elderly in a community in italy. Eur J Epidemiol
2001;17(2):163-8.

53\. Davis JW, Lee E, Taira DA, et al. Influenza vaccination,
hospitalizations, and costs among members of a medicare managed care
plan. Med Care 2001;39(12):1273-80.

54\. Nordin J, Mullooly J, Poblete S, et al. Influenza vaccine
effectiveness in preventing hospitalizations and deaths in persons 65
years or older in minnesota, new york, and oregon: Data from 3 health
plans. J Infect Dis 2001;184(6):665-70.

55\. Christenson B, Lundbergh P. Comparison between cohorts vaccinated
and unvaccinated against influenza and pneumococcal infection. Epidemiol
Infect 2002;129(3):515-24.

56\. Hak E, Nordin J, Wei F, et al. Influence of high-risk medical
conditions on the effectiveness of influenza vaccination among elderly
members of 3 large managed-care organizations. Clin Infect Dis
2002;35(4):370-7.

57\. Washington EL, Shen JJ, Bell R, et al. Patterns of hospital-based
pediatric care across diverse ethnicities: The case of pneumonia. J
Health Care Poor Underserved 2004;15(3):462-73.

58\. Braveman P, Schaaf VM, Egerter S, et al. Insurance-related
differences in the risk of ruptured appendix \[see comments\]. N Engl J
Med 1994;331(7):444-9.

59\. Bratton SL, Haberkern CM, Waldhausen JH. Acute appendicitis risks
of complications: Age and medicaid insurance. Pediatrics 2000;106(1 Pt
1):75-8.

60\. Buckley RG, Distefan J, Gubler KD, et al. The risk of appendiceal
rupture based on hospital admission source. Acad Emerg Med
1999;6(6):596-601.

61\. Guagliardo MF, Teach SJ, Huang ZJ, et al. Racial and ethnic
disparities in pediatric appendicitis rupture rate. Acad Emerg Med
2003;10(11):1218-27.

62\. Ponsky TA, Huang ZJ, Kittle K, et al. Hospital- and patient-level
characteristics and the risk of appendiceal rupture and negative
appendectomy in children. Jama 2004;292(16):1977-82.

63\. O\'Toole SJ, Karamanoukian HL, Allen JE, et al. Insurance-related
differences in the presentation of pediatric appendicitis. J Pediatr
Surg 1996;31(8):1032-4.

64\. Gadomski A, Jenkins P. Ruptured appendicitis among children as an
indicator of access to care. Health Serv Res 2001;36(1 Pt 1):129-42.

65\. Solomon RA, Mayer SA, Tarmey JJ. Relationship between the volume of
craniotomies for cerebral aneurysm performed at new york state hospitals
and in-hospital mortality. Stroke 1996;27(1):13-7.

66\. Taylor CL, Yuan Z, Selman WR, et al. Mortality rates, hospital
length of stay, and the cost of treating subarachnoid hemorrhage in
older patients: Institutional and geographical differences. J Neurosurg
1997;86(4):583-8.

67\. Whisnant JP, Sacco SE, O\'Fallon WM, et al. Referral bias in
aneurysmal subarachnoid hemorrhage. J Neurosurg 1993;78(5):726-32.

68\. Jenkins KJ, Newburger JW, Lock JE, et al. In-hospital mortality for
surgical repair of congenital heart defects: Preliminary observations of
variation by hospital caseload. Pediatrics 1995;95(3):323-30.

69\. Hannan EL, Racz M, Kavey RE, et al. Pediatric cardiac surgery: The
effect of hospital and surgeon volume on in-hospital mortality.
Pediatrics 1998;101(6):963-9.

70\. Sollano JA, Gelijns AC, Moskowitz AJ, et al. Volume-outcome
relationships in cardiovascular operations: New york state, 1990-1995. J
Thorac Cardiovasc Surg 1999;117(3):419-28.

71\. Cetta F, Feldt RH, O\'Leary PW, et al. Improved early morbidity and
mortality after fontan operation: The mayo clinic experience, 1987 to
1992. J Am Coll Cardiol 1996;28(2):480-6.

72\. Iannettoni MD, Bove EL, Mosca RS, et al. Improving results with
first-stage palliation for hypoplastic left heart syndrome. J Thorac
Cardiovasc Surg 1994;107(3):934-40.

73\. Gelatt M, Hamilton RM, McCrindle BW, et al. Arrhythmia and
mortality after the mustard procedure: A 30-year single-center
experience. J Am Coll Cardiol 1997;29(1):194-201.

74\. Knott-Craig CJ, Danielson GK, Schaff HV, et al. The modified fontan
operation. An analysis of risk factors for early postoperative death or
takedown in 702 consecutive patients from one institution. J Thorac
Cardiovasc Surg 1995;109(6):1237-43.

75\. Kaulitz R, Ziemer G, Luhmer I, et al. Modified fontan operation in
functionally univentricular hearts: Preoperative risk factors and
intermediate results. J Thorac Cardiovasc Surg 1996;112(3):658-64.

76\. Fontan F, Kirklin JW, Fernandez G, et al. Outcome after a
\"perfect\" fontan operation. Circulation 1990;81(5):1520-36.

77\. Gentles TL, Gauvreau K, Mayer JE, Jr., et al. Functional outcome
after the fontan operation: Factors influencing late morbidity. J Thorac
Cardiovasc Surg 1997;114(3):392-403; discussion 404-5.

78\. Kern JH, Hayes CJ, Michler RE, et al. Survival and risk factor
analysis for the norwood procedure for hypoplastic left heart syndrome.
Am J Cardiol 1997;80(2):170-4.