Patient Safety Act: HHS Is in the Process of Implementing the Act, So Its Effectiveness Cannot Yet Be Evaluated
Published: 2010-01-29T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: The Institute of Medicine (IOM) estimated in 1999 that preventable medical errors cause as many as 98,000 deaths a year among hospital patients in the United States. Congress passed the Patient Safety and Quality Improvement Act of 2005 (the Patient Safety Act) to encourage health care providers to voluntarily report information on medical errors and other events--patient safety data--for analysis and to facilitate the development of improvements in patient safety using these data. The Patient Safety Act directed GAO to report on the law's effectiveness. This report describes progress by the Department of Health and Human Services, Agency for Healthcare Research and Quality (AHRQ) to implement the Patient Safety Act by (1) creating a list of Patient Safety Organizations (PSO) so that these entities are authorized under the Patient Safety Act to collect patient safety data from health care providers to develop improvements in patient safety, and (2) implementing the network of patient safety databases (NPSD) to collect and aggregate patient safety data. These actions are important to complete before the law's effectiveness can be evaluated. To do its work, GAO interviewed AHRQ officials and their contractors. GAO also conducted structured interviews with officials from a randomly selected sample of PSOs.
section_title: What GAO Found
paragraphs: AHRQ has made progress listing 65 PSOs as of July 2009. However, at the time of GAO's review, few of the 17 PSOs randomly selected for interviews had entered into contracts to work with providers or had begun to receive patient safety data. PSO officials told GAO that some PSOs were still establishing aspects of their operations; some were waiting for AHRQ to finalize a standardized way for PSOs to collect data from providers; and some PSOs were still engaged in educating providers about the confidentiality protections offered by the Patient Safety Act. AHRQ is in the process of developing the NPSD and its associated components--(1) the common formats PSOs and providers will be required to use when submitting patient safety data to the NPSD and (2) a method for making patient safety data non-identifiable, or removing all information which could be used to identify a patient, provider, or reporter of patient safety information. If each of these components is completed on schedule, AHRQ officials expect that the NPSD could begin receiving patient safety data from hospitals by February 2011. AHRQ officials could not provide a time frame for when they expect the NPSD to be able to receive patient safety data from other providers. AHRQ also has preliminary plans for how to allow the NPSD to serve as an interactive resource for providers and PSOs and for how AHRQ will analyze NPSD data to help meet certain reporting requirements established by the Patient Safety Act. According to AHRQ officials, plans for more detailed analyses that could be useful for identifying strategies to reduce medical errors will be developed once the NPSD begins to receive data.

section_title: Letter
paragraphs: Research has shown that serious injuries or deaths resulting from medical  care are both common and often preventable. In a frequently cited report,  the Institute of Medicine (IOM) estimated in 1999 that preventable medical  errors cause as many as 98,000 deaths a year among hospital patients in  the United States. The IOM identified several mechanisms for improving  patient safety, including the use of medical error reporting systems to  gather and analyze information on medical errors in order to prevent them  from occurring in the future. The IOM report noted, however, that health  care providers are often reluctant to report or disclose their medical errors  and to participate in related learning efforts out of fear of incurring legal  liability or professional sanctions. To address these concerns, the IOM  report recommended the expanded use of voluntary medical error  reporting systems that allow confidential reporting. The report also  recommended that Congress provide legal protections to prevent the  unauthorized disclosure of information collected and reported by  providers for the purpose of improving patient safety.
At the time of the IOM’s 1999 report, several states operated mandatory  systems for the reporting of serious medical errors–those resulting in  death or serious injury–but these reporting systems were primarily used to  hold providers accountable for their errors and often involved public  disclosure. In contrast, confidential, voluntary systems for reporting of  medical errors, designed primarily for developing improvements in patient  safety, were less common and less widely used. Partially in response to the  IOM report, Congress passed the Patient Safety and Quality Improvement  Act of 2005 (the Patient Safety Act) to encourage health care providers to  voluntarily report information on patient safety events and to facilitate the  development and adoption of interventions and solutions to improve  patient safety.
To achieve these goals, the Patient Safety Act directed the Department of  Health and Human Services (HHS) to create a list of public or private  organizations known as Patient Safety Organizations (PSO). Listing by  HHS indicates that PSOs are authorized to serve providers as independent  patient safety experts and to receive data regarding patient safety events  that will be considered privileged and confidential. The Patient Safety Act  prohibits the unauthorized disclosure of certain types of data regarding  patient safety events that providers send to listed PSOs. To facilitate the  development of improvements in patient safety, the Patient Safety Act also  requires PSOs to certify that they will analyze data regarding patient safety  events, provide feedback to providers, and develop and disseminate  information on ways providers can improve patient safety. By serving  multiple providers and aggregating data regarding patient safety events,  the Patient Safety Act intends for PSOs to help providers better  understand the underlying causes of patient safety events and develop  solutions to prevent or reduce the frequency of such events.
To support PSOs and providers in their efforts to develop and adopt  improvements in patient safety, the Patient Safety Act directed HHS to  create a network of patient safety databases (NPSD). The Patient Safety  Act specifies that the NPSD is to collect and aggregate non-identifiable  data regarding patient safety events voluntarily submitted to it by PSOs  and providers. The law does not require PSOs or providers to submit data  to the NPSD. The law also specifies that the NPSD should serve as a  resource that health care providers, PSOs, and others can use to develop  improvements in patient safety. Specifically, by facilitating the aggregation  and analysis of patient safety data from providers nationwide, the NPSD is  intended to assist PSOs and providers in identifying underlying patterns  and trends associated with patient safety events. In addition, the Patient  Safety Act also requires HHS to use data from the NPSD to analyze  national and regional statistics, including trends and patterns in patient  safety events, and to report on effective strategies for reducing patient  safety events and increasing patient safety. HHS has delegated  responsibility for listing PSOs, implementing and maintaining the NPSD,  and analyzing data submitted to the NPSD to the Agency for Healthcare  Research and Quality (AHRQ).
The Patient Safety Act requires GAO to report to Congress by February 1,  2010, on the law’s effectiveness in accomplishing its purpose. Although the  Patient Safety Act was enacted in July 2005, the law did not specify a time  frame for implementation. Final regulations to implement the act became  effective on January 19, 2009. In this report, we describe the progress  AHRQ has made in (1) listing PSOs, including PSO efforts to serve  providers, and (2) implementing the NPSD. These actions by AHRQ are  important to complete before the law’s effectiveness can be evaluated. In  addition, because participation—including submission of data on patient  safety events—by PSOs and providers is voluntary, their actions may also  contribute to the law’s effectiveness in accomplishing its purpose.
To describe the progress AHRQ has made in listing PSOs, we interviewed  AHRQ and other HHS officials and reviewed relevant documents,  including regulations, policies, and guidelines. We also conducted  structured interviews with officials from 17 randomly selected PSOs,  representing 26 percent of those listed as of July 2009, to obtain  information on the extent to which the PSOs have begun to collect and  analyze patient safety data and provide feedback to providers. Although  our sample was representative of listed PSOs, our findings from these  interviews cannot be generalized to all PSOs. Furthermore, we did not  speak directly with health care providers regarding their use, or potential  use, of PSOs. To describe the progress AHRQ has made in implementing  the NPSD, we interviewed officials at AHRQ and its contractors, and we  reviewed relevant documents including contracts, time lines, and progress  reports. In addition, we also obtained, for context, information on other  established patient safety reporting systems, as AHRQ’s efforts to list  PSOs and implement the NPSD are relatively new. Specifically, we  identified examples of how selected, established patient safety reporting  systems encourage reporting of patient safety event information by  providers and facilitate the development of improvements in patient  safety. We present these examples and related methodology in appendix I.
We conducted our work from March 2009 to January 2010 in accordance  with all sections of GAO’s Quality Assurance Framework that are relevant  to our objectives. The framework requires that we plan and perform the  engagement to obtain sufficient and appropriate evidence to meet our  stated objectives and to discuss any limitations in our work. We believe  that the information and data obtained, and the analysis conducted,  provide a reasonable basis for any findings and conclusions.
subsections: section_title: Background
paragraphs: AHRQ and the Office for Civil Rights (OCR) within HHS share  responsibility for implementing the Patient Safety Act. AHRQ is  responsible for listing PSOs, providing technical assistance to PSOs,  implementing and maintaining the NPSD, and analyzing the data submitted  to the NPSD. OCR has responsibility for interpreting, implementing, and  enforcing the confidentiality protections. To help implement the Patient  Safety Act, AHRQ and OCR developed the legislation’s implementing  regulations, which took effect January 19, 2009.
The Patient Safety Act establishes criteria that organizations must meet  and required patient safety activities that the organizations must perform  after being listed as PSOs. The criteria include an organizational mission to  improve patient safety and the quality of health care delivery; use of  collected data to provide direct feedback and assistance to providers to  minimize patient risk; staff who are qualified to perform analyses on  patient safety data; and adequate policies and procedures to ensure that  patient safety data are kept confidential. Required PSO activities include  activities such as efforts to improve patient safety and the quality of health  care delivery. (See app. II for the complete list of criteria and required PSO  activities as specified in the Patient Safety Act.) The criteria allow for  many types of organizations to apply to AHRQ to be listed as a PSO. These  organizations may include public and private entities, for-profit and not- for-profit organizations, and entities that are a component of another  organization, such as a hospital association or health system.
A PSO must attest for the initial listing period that it will comply with the  criteria and that it has policies and procedures in place that will allow it to  perform the required activities of a PSO. When reapplying for subsequent  3-year listing periods, a PSO must attest that it is complying with the  criteria and that it is in fact performing each of the required activities. The  regulations require AHRQ staff to review written PSO applications  documenting PSO attestations to each of the statutory criteria and  required activities. In the case of certain PSOs that are component  organizations, the regulations also require the applicant to complete an  additional set of attestations and disclosure statements detailing the  relationship between the component and parent organizations. The  regulations require that after AHRQ staff review the application materials  and related information, the applicant will be listed, conditionally listed, or  denied.
subsections: section_title: Legal Protections for Patient Safety Data
paragraphs: When a provider elects to use the services of a listed PSO, the Patient  Safety Act provides privilege and confidentiality protections for certain  types of data regarding patient safety events that providers collect for the  purposes of reporting to a PSO. In general, the Patient Safety Act  excludes the use of patient safety data in civil suits, such as those  involving malpractice claims, and in disciplinary proceedings against a  provider. While certain states have laws providing varying levels of  privilege and confidentiality protections for patient safety data, the Patient  Safety Act provides a minimum level of protection.
Regulations implementing the Patient Safety Act address the  circumstances under which patient safety data may be disclosed, such as  when used in criminal proceedings, authorized by identified providers, and  among PSOs or affiliated providers. OCR has the authority to conduct  reviews to ensure that PSOs, providers, and other entities are complying  with the confidentiality protections provided by the law. OCR also has the  authority to investigate complaints alleging that patient safety data has  been improperly disclosed and to impose a civil money penalty of up to  $11,000 per violation.
section_title: The NPSD and Requirements for Submitting Data to the NPSD
paragraphs: The Patient Safety Act requires HHS to create and maintain the NPSD as a  resource for PSOs, providers, and qualified researchers. The law  specifies that the NPSD must have the capacity to accept, aggregate, and  analyze non-identifiable patient safety data voluntarily submitted to the  NPSD by PSOs, providers, and other entities. Providers may submit non- identifiable data directly to the NPSD, or work with a PSO to submit  patient safety data. Neither PSOs nor providers are required by either the  Patient Safety Act or regulation to submit data to the NPSD. Figure 1  shows the intended flow of patient safety data and other information  among providers, PSOs, and the NPSD.
The Patient Safety Act authorizes HHS to develop common formats for  reporting patient safety data to the NPSD. According to the Patient Safety  Act, these formats may include the necessary data elements to be  collected and provide common and consistent definitions and a  standardized computer interface for processing the data. While most U.S.  hospitals have some type of internal reporting system for collecting data  on patient safety events, they often have varying ways of collecting and  organizing their data. This variation makes it difficult to accurately  compare patient safety events across systems and providers and can be a  barrier to developing solutions to improve patient safety. If providers or  PSOs choose to submit patient safety data to the NPSD, AHRQ requires  that these data be submitted using the common formats, because using the  common formats is necessary so that data in the NPSD can be aggregated  and analyzed. Aggregation and analysis of data is important for developing  the “lessons learned” or “best practices” across different institutions that  may help improve patient safety.
The Patient Safety Act and its implementing regulations provide additional  measures PSOs must follow whether or not they intend to submit the data  they collect to the NPSD. The Patient Safety Act regulations require PSOs  to collect patient safety data from providers in a standardized manner that  permits valid comparisons of similar cases among similar providers, to the  extent to which these measures are practical and appropriate. To meet this  requirement, the regulation specifies that PSOs must either (1) use the  common formats developed by AHRQ when collecting patient safety data  from providers, (2) utilize an alternative format that permits valid  comparisons among providers, or (3) explain to AHRQ why it would not  be practical or appropriate to do so. The Patient Safety Act also requires  any data regarding patient safety events that is submitted to the NPSD be  non-identifiable. According to the Patient Safety Act, users can access  non-identifiable patient safety data only in accordance with the  confidentiality protections established by the Patient Safety Act. The  Patient Safety Act’s regulations provide technical specifications for  making patient safety data non-identifiable.
Finally, the Patient Safety Act states AHRQ must analyze the data that are  submitted to the NPSD and include these analyses in publicly available  reports. Specifically, under the Patient Safety Act, AHRQ is required to  submit a draft report on strategies to improve patient safety to the IOM  within 18 months of the NPSD becoming operational and a final report to  Congress 1 year later. The Patient Safety Act requires this report to include  effective strategies for reducing medical errors and increasing patient  safety, as well any measures AHRQ determines are appropriate to  encourage providers to use the strategies, including use in any federally  funded programs. In addition, the Patient Safety Act states HHS must use  data in the NPSD to analyze national and regional statistics, including  trends and patterns of health care errors, and include any information  resulting from such analyses in its annual reports on health care quality.
section_title: AHRQ Has Listed PSOs, but Few PSOs We Interviewed Have Begun Serving Providers
paragraphs: AHRQ listed 65 PSOs from November 2008 to July 2009. However, few of  the 17 PSOs we randomly selected to interview had entered into contracts  or other business agreements with providers to serve as their PSO, and  only 3 PSOs reported having begun receiving patient safety data or  providing feedback to providers. PSO officials identified several reasons  why they have not yet engaged with providers. Some PSOs are still  establishing various aspects of their operations; some are waiting for the  common formats for collecting patient safety data to be finalized by  AHRQ; and some are still engaged in marketing their services and  educating providers about the federal confidentiality protections offered  by the Patient Safety Act.
subsections: section_title: As of July 2009, AHRQ Had Listed 65 PSOs Representing a Variety of Organizations
paragraphs: Although the regulations implementing the Patient Safety Act did not  become effective until January 19, 2009, AHRQ began listing PSOs  earlier, in November 2008. By July 2009, AHRQ had listed 65 PSOs in 26  states and the District of Columbia. AHRQ officials told us that in listing  PSOs they accepted PSOs’ attestations that the PSOs met the certification  requirements established in the Patient Safety Act—that is, to be a listed  PSO, an entity must have policies and procedures in place to perform the  required activities of a PSO and will comply with additional criteria for  listing. For continued listing beyond the initial period, PSOs must attest  that they have contracts with more than one provider and are in fact  performing each of the required activities.
The 65 PSOs AHRQ had listed represent a wide range of organizations,  including some that provided patient safety services for many years prior  to being listed as well as new organizations specifically established to  function as a PSO under the Patient Safety Act. AHRQ officials told us  that the organizations listed as PSOs include consulting firms that have  provided patient safety services for a range of providers and specialties, as  well as organizations with a focus on patient safety in a specific area such  as medical devices, hand hygiene, or pediatric anesthesia. The listed PSOs  also include vendors of patient safety reporting software and components  of state hospital associations.
AHRQ officials told us that the services PSOs deliver to individual  providers will likely vary, depending on the specific contractual or other  business agreements the PSOs establish with providers. For example, a  small hospital may want to contract with a PSO to provide all its internal  quality improvement services, while a large hospital may just contract with  a PSO to obtain the legal protections under the Patient Safety Act and to  contribute data to the NPSD. While officials of 13 of the 17 PSOs we  interviewed indicated they provided some patient safety services prior to  being listed, all 17 PSOs stated that the services they planned to make  available included the collection and analysis of patient safety data, the de- identification of patient safety data for submission to the NPSD, feedback,  and patient safety training.
section_title: Few Listed PSOs We Interviewed Have Begun to Serve Providers
paragraphs: While AHRQ has listed 65 PSOs, few PSOs we interviewed have entered  into contracts or other business agreements with providers to serve as  their PSO. Only 4 of the 17 listed PSOs we interviewed had any contracts  or other agreements with providers to serve as their PSO. Furthermore,  according to PSO officials, only 3 of these PSOs had begun to receive  patient safety data or provide feedback to providers. PSO officials  identified several reasons why they had yet to begin working with  providers and receiving patient safety data as of July 2009. These reasons  include the following:  The need to complete the development of various components of their  business operations. Some PSO officials we interviewed told us they still  need to determine various components of their operations. For example,  officials from some PSOs told us they have yet to determine their fee  structure for working with providers. Officials from 6 of 17 PSOs we  interviewed stated they were or would be contracting with other PSOs to  receive services, such as information technology systems support or data  security. Nine PSOs reported they had not yet determined whether they  would be contracting for some services. In addition, while officials from  most of the PSOs we interviewed indicated they planned to submit patient  safety data to the NPSD, 4 had not yet determined how they will make data  non-identifiable before sending it to the NPSD.
The need to obtain AHRQ’s final common formats for collecting data on  patient safety events. Officials from some PSOs we interviewed indicated  they needed the common formats to be finalized by AHRQ before  beginning to work with providers. While use of AHRQ’s common formats  to collect data from providers is not required under the regulations, most  PSOs we interviewed plan to use the common formats for collecting data  on patient safety events and submitting these data to the NPSD. Officials  from 7 of the 17 PSOs we interviewed said they plan to require providers  to submit data using the common formats, and 4 PSOs said they will not  require them of providers but will either convert the reports they receive  to the common formats or adapt their existing reporting system to include  the common formats.
The need to educate providers about the federal confidentiality  protections. Officials from several of the 17 PSOs we interviewed told us  they faced challenges in addressing provider concerns related to the scope  of the confidentiality protections and that these concerns needed to be  addressed before providers would be willing to engage the services of a  PSO. Some of these PSO officials described challenges in communicating  details of the confidentiality protections. According to AHRQ officials, the  rules for when, where, and how patient safety data are protected from  disclosure are both complex and interrelated with the privacy rules for  protected health information under HIPAA. AHRQ officials acknowledged  the need to work with PSOs to clarify the rules governing the  confidentiality of patient safety data so PSOs can better communicate  these to providers. AHRQ officials indicated they would address these  issues in upcoming quarterly conference calls they hold with PSO  representatives. (See appendix I for examples of ways established patient  safety reporting systems communicate legal protections for providers and  the data they submit.)
section_title: AHRQ Is in the Process of Implementing the NPSD and Has Developed Preliminary Plans for Using NPSD Data
paragraphs: AHRQ is in the process of implementing the NPSD and developing its  associated components that are necessary before the NPSD can receive  patient safety data—(1) the common formats PSOs and providers will be  required to use if submitting patient safety data to the NPSD and (2) a  method for making these data non-identifiable. If each of these  components is completed on schedule, AHRQ officials expect that the  NPSD could begin receiving patient safety data from hospitals in February  2011. AHRQ officials could not provide a time frame for when they expect  the NPSD to be able to receive patient safety data from other providers.  AHRQ also has preliminary plans for how to allow the NPSD to serve as an  interactive resource for providers and PSOs and for how AHRQ will  analyze NPSD data to help meet its reporting requirements under the  Patient Safety Act.
subsections: section_title: AHRQ Is in the Process of Developing the NPSD, the Common Formats for Hospitals to Submit Data to the NPSD, and a Method for Making Data Non- identifiable
paragraphs: AHRQ is in the process of developing the NPSD, and AHRQ officials  expect that the NPSD could begin receiving patient safety data from  hospitals by February 2011. Specifically, AHRQ established a 3-year  contract with Westat effective September of 2007 to develop the NPSD,  which is being set up as a database that AHRQ officials stated is essential  for meeting the requirements of the act. AHRQ and Westat officials told us  that completion of the NPSD depends on both the development of the  common formats that will be used to submit patient safety data to the  NPSD and on the development of a method for making the data non- identifiable. If each of these components is completed on schedule, AHRQ  officials expect that the NPSD could begin to receive patient safety data  from hospitals by February 2011.
AHRQ is finalizing the common formats that PSOs and hospitals will be  required to use if submitting patient safety data to the NPSD. AHRQ  officials expect that the common formats could be available for hospitals  to use in submitting data electronically to the NPSD by September 2010.  AHRQ began developing the common formats for hospitals in 2005 by  reviewing the data collection methods of existing patient safety systems.  In 2007, AHRQ contracted with the National Quality Forum (NQF) to assist  with the collection and assessment of public comments on a preliminary  version of the common formats that was released in August 2008. These  common format forms are used to collect information on patient safety  events, including information about when and where an event occurred, a  description of the event, and patient demographic information. AHRQ  issued the common formats for hospitals in paper form in September 2009,  and is in the process of making electronic versions available for hospitals  and PSOs to use when submitting data to the NPSD. Specifically, AHRQ  officials told us that they are in the process of developing technical  specifications that private software companies and others can use to  develop electronic versions of the common formats. According to AHRQ  officials, hospitals and PSOs will need these electronic versions of the  common formats in order to submit data to the NPSD. Their current  project plan indicates that the technical specifications will be completed  by March 2010. AHRQ officials estimate that electronic versions of the  common formats could be available to hospitals and PSOs by September  2010.
AHRQ officials stated that they expect eventually to develop common  formats for providers in other health care settings, such as nursing homes  and ambulatory surgical centers. Furthermore, AHRQ officials told us that  that they plan on developing future versions of the common formats  capable of collecting data from the results of root cause analyses that  providers may conduct. However, AHRQ officials were unable to provide  an estimate for when the common formats for other providers will be  available or when the capability to collect information from root cause  analyses will be available.
The Patient Safety Act also requires that data submitted to the NPSD be  made non-identifiable by removing information that could be used to  identify individual patients, providers, or facilities. To help PSOs and  providers meet this requirement, AHRQ contracted with the Iowa  Foundation for Medical Care (IFMC) to operate a PSO Privacy Protection  Center (PPC) that will develop a method for making patient safety data  non-identifiable and assist PSOs and providers by removing any  identifiable patient or provider information from the data before  submission to the NPSD. Current AHRQ and PPC project plans indicate  that the PPC should be ready to receive and make patient safety data non- identifiable beginning in September 2010. AHRQ officials told us that this  process involves not only removing information from each record that  could be used to identify patients, providers, or reporters of patient safety  information, but also determining whether identities could be determined  from other available information and using appropriate methods to  prevent this type of identification from occurring. AHRQ officials told us  that PPC officials are working with experts to develop the PPC’s method  for making data non-identifiable.
AHRQ officials stated that their rationale for establishing the PPC was to  determine a method for making data non-identifiable, provide a cost  savings for PSOs, encourage data submission to the NPSD, and create  consistency in the non-identifiable data that are submitted to the NPSD.  According to AHRQ officials, the PPC will provide its services to PSOs at  no charge and will submit non-identifiable patient safety data on behalf of  PSOs to the NPSD. However, PSOs are not required to use the PPC and  may choose to make their patient safety data non-identifiable internally or  with the help of a contractor of their choice.
AHRQ project plans indicate that the PPC will be able to submit data to  the NPSD beginning in February 2011, approximately 5 months after the  PPC begins receiving data from hospitals. AHRQ officials stated that this  time period is necessary, in part, because the PPC needs to begin receiving  data before it can determine if its method for rendering data non- identifiable is appropriate or needs to be adjusted. For example, if the PPC  receives a sufficient volume of data, then officials expect to be able to  submit data on individual patient safety events and have it remain non- identifiable. If the volume of data is too low, however, PPC officials expect  to have to aggregate data from individual events so that it remains non- identifiable once submitted to the NPSD, in which case AHRQ officials  stated they may delay submission of data to the NPSD until a sufficient  volume is received. AHRQ officials noted that it is impossible to determine  in advance the volume of data that will be submitted to the PPC due to the  voluntary nature of submissions. As a result, the level of detail that will  exist in the NPSD data cannot be determined in advance of data being  received and processed by the PPC. Figure 2 summarizes key dates in  AHRQ’s efforts to develop the NPSD and its related components.
section_title: AHRQ Has Preliminary Plans For How to Meet Requirements for Use of NPSD Data, Though AHRQ Officials Have Identified Limitations to the Types of Analyses That Will Be Conducted
paragraphs: The Patient Safety Act requires that the NPSD serve as an interactive  resource for providers and PSOs, allowing them to conduct their own  analyses of patient safety data. To meet this requirement, AHRQ has  developed plans to allow providers to query the NPSD to obtain  information on patient safety events, including information on the  frequencies and trends of such events. AHRQ’s contract with Westat to  construct the NPSD includes a series of tasks for developing, testing,   and implementing this interactive capability of the NPSD. The contract  specifies that these interactive capabilities will be available within   12 months of the NPSD beginning to receive patient safety information.  Based on AHRQ’s estimate that the NPSD may be operational by February  2011, the interactive capabilities of the NPSD could be available by  February 2012. However, AHRQ officials indicated that they had not yet  determined the specific types of information that will be available to PSOs  and providers as this will depend, in part, on the level of detail that is  included in the NPSD data after the data are made non-identifiable.
The Patient Safety Act also states that HHS must use the information  reported into the NPSD to analyze national and regional statistics,  including trends and patterns of health care errors, and to identify and  issue reports on strategies for reducing medical errors and increasing  patient safety after the NPSD becomes operational. To do this, AHRQ has  developed preliminary plans for analyzing the data that will be submitted  to the NPSD. According to AHRQ officials, these plans specify how the  agency will analyze NPSD data to determine trends and patterns, such as  the frequency with which certain types of adverse events happen across  providers based on the data they may submit to the NPSD. However,  AHRQ has yet to develop plans for more detailed analyses of NPSD data  that could be useful for identifying strategies to reduce medical errors.  Officials explained that these plans will not be developed until the NPSD  begins receiving data and they are able to determine the level of detail in  the data and what analyses it will support.
Despite the potential for standardization provided by the common formats,  AHRQ officials have identified important limitations in the types of  analyses that can be performed with the data submitted to the NPSD. For  example, AHRQ officials explained that because submissions to the NPSD  are voluntary, the trends and patterns produced from the NPSD will not be  nationally representative and, therefore, any analyses conducted cannot  be used to generate data that are generalizable to the entire U.S.  population. In addition, officials stated that the results from some analyses  may be unreliable because there is no way to control for duplicate entries  into the NPSD, which could occur if a provider submits a single patient  safety event report to more than one PSO. Finally, AHRQ officials noted  that it will be difficult to determine the prevalence or incidence of adverse  events in specific populations. They told us that determining prevalence or  incidence rates requires information on the total number of people at risk  for such events, and that the patient safety data submitted to the NPSD  will not include this information. (See appendix I for more information  about the ways established patient safety reporting systems analyze data  to develop solutions that improve patient safety.)
section_title: Concluding Observations
paragraphs: AHRQ is still in the early stages of listing PSOs and developing plans for  how it will analyze NPSD data and report on effective strategies for  improving patient safety, as required under the Patient Safety Act. As a  result, we cannot assess whether, or to what extent, the law has been  effective in encouraging providers to voluntarily report data on patient  safety events and to facilitate the development and adoption of  improvements in patient safety. In addition, because improvements to  patient safety depend on the voluntary participation of providers and  PSOs, it remains uncertain whether the goals of the Patient Safety Act will  be accomplished even after AHRQ completes its implementation. For  example, providers will have to decide whether to work with a PSO and  the extent to which they will report patient safety data to both the PSO  and the NPSD. Whether the process results in specific recommendations  for improving patient safety will depend on the volume and quality of the  data submitted and on the quality of the analyses conducted by both PSOs  and by AHRQ. Finally, if these recommendations are to lead to patient  safety improvements, providers must recognize their value and take  actions to implement them.
section_title: Agency Comments
paragraphs: The Department of Health and Human Services reviewed a draft of this  report and provided technical comments, which we have incorporated as  appropriate.
We will send copies of this report to the Secretary of Health and Human  Services and other interested parties. In addition, the report is available at  no charge on the GAO Web site at http://www.gao.gov.
If you or your staff have questions about this report, please contact me at  (202) 512-7114 or kohnl@gao.gov. Contact points for our Office of  Congressional Relations and Public Affairs may be found on the last page  of this report. GAO staff members who made key contributions to this  report are listed in appendix III.
section_title: Appendix I: Examples from Established Patient Safety Reporting Systems
paragraphs: Because the Agency for Healthcare Research and Quality’s (AHRQ) efforts  to list Patient Safety Organizations and implement the Network of Patient  Safety Databases are relatively new but some other patient safety  reporting systems are already established, we identified examples of how  selected established patient safety reporting systems encourage reporting  of patient safety event information by providers and facilitate the  development of improvements in patient safety. We judgmentally selected  five established patient safety reporting systems from a list of such  systems compiled by AHRQ. We selected systems that collected data for  learning purposes and that appeared in a literature review we conducted  of 45 relevant articles in peer-reviewed, trade, or scholarly publications  published since January 2000. After selecting the systems, we conducted  structured interviews with representatives of these systems to identify  examples of ways that these systems encouraged providers to submit  patient safety data for analysis and used the data collected by their  systems to help develop improvements in patient safety. The system  representatives we interviewed provided common examples that we have  grouped into four areas:    Practices that encourage providers to learn from patient safety data, rather    Communication intended to clearly explain legal protections for providers  and the data they submit;   Data collection tools intended to standardize the data providers submit;    Data analyses that produce actionable feedback.
Practices that encourage providers to learn from patient safety  data, rather than blame individuals. Representatives from all five  patient safety reporting systems we reviewed said their systems encourage  providers to learn from patient safety data as a way to improve patient  safety, and not blame individuals for an event. According to system  representatives, one way they did this was to emphasize the value of the  data collected by the system for learning ways to reduce the risk that a  certain event will recur. For example, representatives from one system  said they created posters to hang in health care facilities from which the  system collected patient safety data. Representatives from this system  explained that the posters described a patient safety event about which  the system received data as well as the solutions the system developed to  improve patient safety. Another practice representatives said they used is  allowing providers to submit data anonymously. Four out of five system  representatives said their systems offered providers a way to submit data  anonymously.
Communication intended to clearly explain legal protections for  providers and the data they submit. Many of the representatives we  interviewed from patient safety reporting systems told us that their  systems communicate information intended to clearly explain the legal  protections afforded providers and the patient safety data they submit. For  example, one system in our review provided guidance for providers on  how to clearly label data to invoke the confidentiality protections  associated with patient safety data under a law that protects data in this  system. Representatives from another patient safety reporting system told  us that communicating information about available legal protections can  be particularly important for systems that collect data from providers in  multiple states, because the legal protections for providers and patient  safety data vary from state to state. For example, representatives from two  patient safety reporting systems with users in multiple states said their  systems provided customized legal information for providers based on the  state confidentiality laws that applied to each provider’s location. A  representative from one of these systems also said that the legal  information the system offered helped providers understand what types of  data to submit and encouraged them to submit it.
Data collection tools intended to standardize the data providers  submit. Representatives from all five systems told us they had developed  tools intended to standardize the data providers submit to their patient  safety databases. For some systems these tools include common formats  and computer systems. Some of the representatives explained that  standardizing the information providers submit helps ensure that patient  safety events, especially events involving clinical terms, are classified in  the same way. Some representatives also said that if a system did not  define clinical terms for providers, providers may define events differently,  which can limit the system’s ability to analyze submitted patient safety  data. Furthermore, the representatives said, standardizing terms increased  the value of the data as it is aggregated, as well as any resulting analyses.  Representatives from all five systems said the ability to collect and  aggregate standardized patient safety data allowed them to identify  patterns in patient safety events, which they believed enabled their  systems to suggest ways to improve patient safety.
Some system representatives said that standardizing the way providers  submit patient safety data allowed them to streamline the data collection  process for providers. Some representatives said they designed their data  collection protocols to allow providers to fulfill additional reporting  requirements related to accreditation or quality improvement functions,  such as submitting data regarding certain patient safety events to the Joint  Commission. Representatives from one system said that their systems did  this to make collecting and submitting patient safety data more efficient  for providers and thereby increase the likelihood that providers would  submit such data to the patient safety reporting system. In another  example, one system built a feature into its computer program that  allowed providers to transfer data directly from providers’ in-house  databases to the patient safety data collection system, a data collection  method system representatives said accounted for approximately   40 percent of all data received from providers.
Data analyses that produce actionable feedback. Representatives  from all five patient safety reporting systems told us that their systems  analyzed submitted data to develop actionable steps providers could  implement to improve patient safety. According to the representatives,  their systems aggregated data from provider submissions and used these  data for both quantitative analyses, such as trend or frequency analyses,  and qualitative analyses, which examine narrative data to determine  whether there were any common themes across events. Representatives  from all five systems said they used both qualitative and quantitative  analyses because neither method alone was completely sufficient to  develop improvements to patient safety. For example, one system’s  representatives said they conducted qualitative analyses such as using a  computer program to analyze and group the narrative data providers  submitted to learn about the factors that contributed to patient safety  events. The same representatives explained that their system also  conducted quantitative analyses such as trend analyses on events to see  how often they occur.
Representatives from all the systems said they used various methods to  encourage providers to implement the improvements to patient safety the  systems helped develop. Examples of methods they used included sending  an e-mail from the system when new content was published on the  system’s Web site, hosting Web conferences, and publishing analyses in  trade or scholarly publications. All the representatives said their systems  collaborated with other organizations to increase the likelihood that the  improvements they developed were implemented. For example, one  system worked with a statewide coalition of organizations in the quality  improvement field to encourage providers to implement the patient safety  improvements the system developed.
section_title: Appendix II: Selected Statutory Requirements for Listing of Patient Safety Organizations
subsections: section_title: GAO Contact:
section_title: Staff Acknowledgments:
paragraphs: In addition to the contact named above, William Simerl, Assistant Director;  Eric R. Anderson; Eleanor M. Cambridge; Krister Friday; Kevin Milne; and  Andrea E. Richardson made key contributions to this report.