Safety outcomes in the United States: trends and challenges in measurement.

One of the key components of any effort to track national trends in patient safety involves measuring patient outcomes. Ultimately, outcomes at the clinical level can be understood as the occurrence of adverse events that cause harm to patients. Actually measuring these sorts of adverse events, however, poses major challenges (Institute of Medicine 2000, 2004; Pronovost et al. 2005). The provision of health care in the United States involves enormous complexity, across widely ranging clinical settings, procedures, conditions, and treatment interventions. There are many safety measures currently in use that apply to different aspects of this complex system, but the measures themselves vary considerably in the extent to which they have been validated, in the breadth and consistency of their use across facilities and regions, and in the purposes that they were designed to serve (Zhan et al. 2005). Drawing from this abundance of available measures to construct a big-picture view of patient safety in the United States is a difficult task. The task is nevertheless critical to understanding the impact of broad-based safety interventions, and to prioritizing resources and investment for patient safety in the future. The current study involved a review of national data on safety outcomes, undertaken as part of a larger, CIPP-based evaluation of AHRQ's patient safety initiative (see Farley and Battles 2008, in this issue). Because the AHRQ safety initiative was not expected to have observable effects on national measures of safety outcomes or practices within the 4-year lifespan of our evaluation effort, our assessment of outcomes took on a developmental approach, intended to help AHRQ prepare for continued monitoring of its own safety initiative over time. In part, our aim was to explore baseline trends in outcomes on some widely available safety measures during the early years of this decade, predating any possible impacts generated by AHRQ's activities. More importantly, our aim was also to explore difficulties involved in using available patient safety measures in this way, and to examine the methodological hurdles and interpretive gaps involved in doing so. Safety measurement and data capabilities represent a fundamental part of the health care infrastructure, and a key precursor to efforts to improve safety within the system, whether on a local or national scale. Diagnosing this measurement infrastructure and its limitations, and suggesting ways to improve and better leverage it, have been basic priorities in our evaluation work for AHRQ. It can be easy, in trying to look at national patterns in safety outcomes, either to become overwhelmed by methodological details or to reject any substantive findings outright, on grounds that the various measures are too flawed, too limited, or otherwise problematic. But for all of the legitimate criticisms of safety measures, we submit that this is an unsatisfactory answer. Patient safety is, and will remain, a major priority for health care reform and investment in the United States. Efforts are now ongoing throughout the country to try to improve safety performance in various ways, and AHRQ (and others) will face the challenge of gauging the ultimate success of these efforts. Stated another way, the challenge is to make best and responsible use of imperfect measurement resources in tracking national safety progress, while simultaneously using the same information to help bootstrap measurement capabilities to higher levels of performance. This study, as well as our previous years of work in investigating clinical safety outcomes for AHRQ (Farley et al. 2005, 2007a, b), is intended to help support that end.