Clinical Lab Quality: CMS and Survey Organization Oversight Should Be Strengthened
Published: 2006-06-16T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: The Clinical Laboratory Improvement Amendments of 1988 (CLIA) strengthened and extended quality requirements for labs that perform tests to diagnose or treat disease. About 36,000 labs that perform certain complex tests must be surveyed biennially by either a state or one of six private accrediting organizations. CMS oversees implementation of CLIA requirements and the activities of survey organizations. GAO was asked to examine (1) the quality of lab testing; (2) the effectiveness of surveys, complaint investigations, and enforcement actions in detecting and addressing lab problems; and (3) the adequacy of CMS's CLIA oversight.
section_title: What GAO Found
paragraphs: Because of limited comparable data from CMS and survey organizations, too little is known about the quality of lab testing. For example, a standardized assessment of lab quality across survey organizations is not possible because of different definitions of what constitutes a serious quality problem. One survey organization had no systematic way of identifying the problematic labs it inspects. However, GAO's analysis of an indicator that measures a lab's ability to consistently produce accurate test results suggests that lab quality may not have improved at hospital labs in recent years. Based on an analysis of available data and interviews with CMS and survey organizations, real and potential lab quality problems are masked by survey, complaint, and enforcement weaknesses. Because most survey organizations announce the timing of biennial surveys, allowing labs to prepare for inspections, surveys may not provide a realistic picture of lab quality. Although two survey organizations that generally inspect hospital labs plan to begin unannounced surveys in 2006, they may not be possible at physician office labs that have irregular hours. Survey organizations that typically inspect such labs, however, provide more advance notice about upcoming inspections than CMS allows states to provide. Several other factors suggest that surveys and complaints do not present a realistic picture of lab quality. Interviews with officials from a sample of states confirmed that some survey organizations do not cite all serious deficiencies, as evidenced by variability in the limited available lab survey data. Officials said that surveyors may be reluctant to cite deficiencies because they view their role as educational, not regulatory; moreover, CMS has instructed state surveyors not to cite some deficiencies for over 2 years after implementing new lab requirements. Finally, lab workers may file complaints infrequently because of concern about retaliation and a lack of understanding about how to file a complaint. CMS rarely imposes sanctions, even for labs with the same repeat deficiencies, a reflection of the educational focus of the CLIA program. CMS does not require labs to participate in a key quality assurance test as frequently as CLIA requires. Although funded by lab fees, CMS officials indicated that the program has not been allowed to hire sufficient staff to carry out the agency's oversight responsibilities. Moreover, CMS's principal oversight tool, intended to determine if all serious deficiencies were identified, lacks independence because many oversight reviews are conducted simultaneously with survey organizations. CMS's presence may make surveyors more attentive to survey tasks than when they are not being observed. Compared to independent reviews, simultaneous reviews rarely identify missed deficiencies. Furthermore, CMS does not collect and analyze data on serious deficiencies identified by each survey organization and thus, is unable to assess whether lab quality is improving or declining. Nor does CMS effectively analyze other key data such as the use of sanctions. To improve oversight, CMS is establishing a nationwide complaints database. CMS is also instituting annual survey organization performance reviews.

section_title: Letter
paragraphs: Clinical lab tests are one of the most frequently billed Medicare  procedures and, according to the American Clinical Laboratory  Association, affect an estimated 70 percent of medical decisions. To  improve oversight of clinical labs, Congress passed legislation in 1967;  renewed concerns about quality, including errors in Pap smear tests used  to diagnose cervical cancer, resulted in enactment of the Clinical  Laboratory Improvement Amendments of 1988 (CLIA). In recent years,  despite CLIA, lab quality problems in several states have raised questions  about the adequacy of lab oversight. Lab oversight is critical because  inaccurate or unreliable lab tests may lead to improper treatment,  unnecessary mental and physical anguish for patients, and higher health  care costs.
The Centers for Medicare & Medicaid Services (CMS) is responsible for  overseeing compliance with CLIA requirements. As of December 2005,  there were approximately 193,000 labs nationwide, ranging from very  small physician office labs that conduct fewer than 2,000 tests annually to  hospital labs that conduct millions of tests each year. Most clinical labs  regulated under CLIA must obtain a certificate from CMS but only about  19 percent—those that conduct moderate- to high-complexity tests— undergo biennial inspections, which are also referred to as surveys. The  surveys assess lab compliance with mandated personnel and testing  standards. In addition, surveyed labs must participate in proficiency  testing, a program that requires them to test samples with unknown  characteristics that are then graded by an external party. Labs with serious  deficiencies may be sanctioned, e.g., required to cease testing. Labs have a  choice of being surveyed by (1) their state survey agency, under contract  with CMS; (2) their state CLIA-exempt program for labs in New York and  Washington; or (3) one of six private accrediting organizations. State  survey agency inspections use CLIA requirements that are intended to help  ensure valid and reliable lab tests; the two state CLIA-exempt programs  and six accrediting organizations survey labs using their own requirements  that CMS has determined to be at least equivalent to CLIA’s. Each survey  organization is also responsible for investigating complaints about lab  quality. Because of the critical importance of accurate lab test results and  oversight, you asked us to conduct a nationwide assessment of (1) the  quality of lab testing; (2) the effectiveness of surveys, complaint  investigations, and enforcement actions in detecting problems and  ensuring compliance; and (3) the adequacy of CMS oversight of the CLIA  program.
To determine what is known about the quality of lab testing, we analyzed  data on serious deficiencies identified during surveys by state survey  agencies using CMS’s On-Line Survey, Certification, and Reporting system  (OSCAR). The CLIA program inspection requirements are classified as  either “standard-” or “condition-” level. Similarly, deficiencies are also  characterized as standard- or condition-level, based on the requirement in  which the deficiency occurs. Because condition-level requirements  generally consist of one or more standard-level requirements, a deficiency  at the condition-level denotes a serious or systematic problem. We  requested comparable data on serious deficiencies from state CLIA- exempt programs and the three largest accrediting organizations—the  College of American Pathologists (CAP), COLA, and the Joint Commission  on Accreditation of Healthcare Organizations (JCAHO)—which together  survey about 97 percent of accredited labs. CAP, COLA, JCAHO, and  exempt-state programs each maintain their own separate databases. We  also analyzed proficiency testing data—another indicator of a lab’s ability  to produce accurate test results. CMS officials generally recognize OSCAR  and proficiency testing data to be reliable. We discussed the OSCAR  database with CMS officials and tailored our analysis to ensure the  accuracy of our findings. Because exempt states and accrediting  organizations survey labs using their own requirements, we worked with  them to develop data comparable to OSCAR deficiency data. We discussed  our analyses with CMS and each of the survey organizations to ensure that  we had interpreted the data correctly. Based on discussions with officials  from the three accrediting organizations, we determined that they take  appropriate steps to ensure the reliability of their data. Because it was not  practical to independently test the reliability of accrediting organization  data, we present these data as reported by those organizations.
To assess the effectiveness of lab surveys, complaint investigations, and  enforcement mechanisms in detecting problems and securing compliance,  we reviewed the processes used to ensure the quality of clinical lab testing  and analyzed available data related to these issues. We also conducted  structured interviews with officials from (1) CMS, (2) three CMS regional   offices, (3) 10 state survey agencies, (4) the New York and Washington  CLIA-exempt programs, and (5) the three accrediting organizations. We  judgmentally selected the 10 state survey agencies to include a mixture of  states whose lab inspections identified a range of serious deficiencies from  few to many. We also discussed the quality problems discovered at a  Maryland hospital lab with a Maryland state survey agency official and  interviewed 9 of the 36 CAP surveyors who participated in surveys of this  lab from 1999 through 2003 to obtain a firsthand perspective on the CAP  survey process. Based on our review and discussions with CMS and  survey organization officials, we focused on several key issues, including  the rationale for announced surveys, the ability of lab surveys to identify  serious deficiencies, the balance struck between the regulatory and  educational goals of lab surveys, the implications of CAP’s use of  volunteer surveyors from neighboring labs to conduct inspections, how  survey organizations facilitate the filing of complaints, and the use of  sanctions to encourage compliance. We analyzed data on the number of  complaints received by each survey organization from 2002 through 2004  and discussed CAP’s initiatives to encourage the filing of complaints. In  addition, we determined the extent to which labs had the same serious  problems on consecutive surveys and discussed with CMS the steps the  agency had taken to deter an inconsistent pattern of compliance.
To assess the effectiveness of CMS oversight of the CLIA program, we  analyzed the laws and regulations that define CMS’s role and authority. We  also reviewed CMS’s process for determining that the survey requirements  and procedures of state CLIA-exempt programs and accrediting  organizations are at least equivalent to those of CLIA. We analyzed the  results of validation reviews that federal surveyors from CMS regional  offices conducted for state survey agency lab inspections and that state  survey agency staff conducted for accrediting organization inspections  from 1999 through 2003. We also examined other mechanisms that CMS  uses to hold survey organizations accountable for their performance:   (1) the collection and analysis of data on surveys, complaints, and  enforcement actions, including steps taken to address communication and  coordination issues that became evident during a complaint investigation  at a Maryland hospital lab; and (2) recently developed annual reviews that  measure state survey agency compliance with CLIA program  requirements, such as the timeliness of surveys. We conducted our work  from January 2005 through May 2006 in accordance with generally  accepted government auditing standards.
subsections: section_title: Background
paragraphs: A clinical lab is generally defined as a facility that examines specimens  derived from humans for the purpose of disease diagnosis, prevention, and  treatment, or health assessment of individuals. While hospital and  interstate labs were previously subject to regulation, CLIA strengthened  federal requirements and extended them to most other clinical labs,  including physician office labs. For example, CLIA strengthened personnel  requirements for lab workers, strengthened proficiency testing that  evaluates the accuracy of lab testing between surveys, and created a range  of sanctions to enforce compliance.
Most clinical labs regulated under CLIA must obtain a certificate from  CMS and pay fees every 2 years to cover the costs of administering the  CLIA program, including surveys and other oversight activities. The fees  vary based on the complexity and volume of testing performed. Lab tests  are categorized as waived, moderate, or high complexity. Approximately  81 percent of all labs (about 157,000) are not subject to routine biennial  surveys because they perform (1) “waived” tests, which are examinations  and procedures that have an insignificant risk of erroneous results,  including those approved for home use or determined to employ  methodologies so simple or accurate that the likelihood of erroneous  results is negligible; or (2) tests performed during the course of a patient  visit with a microscope on specimens that are not easily transportable.
subsections: section_title: Surveyed Labs
paragraphs: CLIA establishes more stringent requirements for the 19 percent (about  36,000) of labs performing moderate- or high-complexity testing, including  the requirement for a survey and participation in routine proficiency  testing. Since the early 1990s, the number and proportion of labs subject to  surveys have declined, while the number and proportion conducting  waived tests have increased. Surveys examine lab compliance with CLIA  program requirements in several areas including: personnel qualifications,  proficiency testing, quality control, quality assurance, and recordkeeping.
Personnel: CLIA sets minimum qualifications for all persons performing or  supervising moderate- or high-complexity lab tests and specifies  responsibilities for each position.
Proficiency testing: Surveyed labs must participate in an approved  external proficiency testing program, which evaluates the accuracy of  laboratory testing. Under this requirement, labs purchase samples with  unknown characteristics several times each year from an approved  proficiency testing provider. The lab is required to test the samples with  its routine patient testing, and the results are returned to the testing  provider to be graded. A proficiency testing failure is defined as  unsatisfactory performance on two consecutive or two out of three testing  events. The results of proficiency testing for all inspected labs are  transmitted to CMS and maintained in a database.
Quality control: Labs must have a process for routinely monitoring  personnel, testing equipment, and the testing environment to ensure  proper operation and accurate results.
Quality assurance: Labs must follow their plan to monitor the overall  operation of the laboratory on an ongoing basis and must resolve  identified problems that affect the quality of their testing.
Recordkeeping: Labs must maintain an audit trail of testing that  documents specimen integrity and test performance for all phases of the  test process from the test order to the test report.
section_title: Survey Organizations
paragraphs: In general, labs have a choice of who conducts their surveys—state survey  agencies using CLIA inspection requirements or other survey  organizations that use requirements CMS has determined to be at least  equivalent to CLIA’s. CMS contracts with state survey agencies in most  states to inspect labs against CLIA requirements. CLIA established an  approval process to allow states and private accrediting organizations to  use their own requirements to survey labs. As noted earlier, New York  and Washington operate CLIA-exempt programs and CMS has approved  six private, nonprofit accrediting organizations to survey labs—the  American Association of Blood Banks (AABB), the American Osteopathic  Association (AOA), the American Society of Histocompatibility and  Immunogenetics (ASHI), CAP, COLA, and JCAHO. The requirements of  both state CLIA-exempt programs and accrediting organizations must be  reviewed by CMS at least every 6 years to ensure CLIA equivalency, but  may be more stringent than those of CLIA. For example, when inspecting  labs engaged in moderate- and high-complexity testing, New York and  some accrediting organizations also look at the labs’ procedures for  conducting “waived” tests, which is not required under CLIA. Figure 1 lists  the three types of survey organizations and indicates whether they survey  labs under CLIA requirements, or use their own CLIA-equivalent  requirements. It also shows the percentage of labs performing moderate-  to high-complexity testing surveyed by each type of organization. In  general, state survey agencies, COLA, and Washington’s CLIA-exempt  program survey physician office labs, while New York’s CLIA-exempt  program, CAP, and JCAHO survey hospital labs.
section_title: Surveys and Complaint Investigations
paragraphs: Survey organizations (1) conduct surveys and complaint investigations,  and (2) monitor proficiency test results submitted by surveyed labs three  times a year. Surveys are typically conducted by former or current lab  workers, who assess lab compliance with CLIA or CLIA-equivalent  requirements. Most lab inspections are announced, that is, the lab has  advance notice of when the survey will occur. Generally, surveyors verify  that lab personnel are appropriately qualified to conduct testing, evaluate  proficiency test records, check equipment and calibration to ensure that  appropriate quality control measures are in place, and determine whether  the lab has a quality assurance plan and uses it to, among other things,  appropriately identify and resolve problems affecting testing quality.  Surveys also include an educational component to assist labs in  understanding how to comply with CLIA requirements. The duration of a  survey generally depends on the size—in terms of the number of tests  conducted—and complexity of a lab as well as the number of surveyors.  Thus, a survey conducted at a small lab may only take a few hours to  complete, while a survey at a large hospital lab may take a survey team a  full week or more.
In addition to inspections, survey organizations are responsible for  determining the seriousness of and investigating all complaints. For those  complaints that are determined to pose immediate jeopardy—an imminent  and serious threat to patient health and a significant hazard to public  health—CMS requires that the investigation be initiated within 2 working  days. Complaints may be investigated on-site or through communications  between the survey organization and the lab. Complaint investigations for  all survey organizations are unannounced.
Lab survey requirements are classified as either “standard-” or “condition-”  level. Generally, condition-level requirements are made up of one or more  related standard-level requirements. For example, the condition-level  requirement on enrollment and testing of samples through a proficiency  testing program has two related standard-level requirements:   (1) enrollment, which includes requirements for the lab to provide the  name of the program it has enrolled in to the Department of Health and  Human Services and authorize the release of testing data to the  department; and (2) testing, which specifies that the samples must be  tested in the same manner as any specimen and prohibits referring the test  samples to another lab for analysis.
Deficiencies are also characterized as standard- or condition-level based  on the requirement in which the deficiency occurs. Deficiencies in  standard-level requirements, that is, standard-level deficiencies, denote  problems that generally are not serious, while condition-level deficiencies  are cited when the problems are serious or systemic in nature. A serious  problem is defined as an inadequacy in a lab’s quality of services that  adversely affects, or has the potential to adversely affect, the accuracy and  reliability of patient test results. When deficiencies are found during  surveys or complaint investigations, labs are required to submit a plan of  correction, detailing how and when they will address the deficiencies.  Additionally, CMS can impose principal or alternative sanctions, or both.  Principal sanctions include revocation of a CLIA certificate, cancellation  of the right to receive Medicare payments, or limits on testing. Revocation  of a CLIA certificate is equivalent to termination from the CLIA program.  Alternative sanctions are less severe and include civil money penalties or  on-site monitoring. For condition-level deficiencies that do not involve  immediate jeopardy, labs have an opportunity to correct the deficiencies,  which we refer to as a grace period, before the sanctions are imposed. If a  lab is unable to correct a deficiency during this grace period, CMS  determines whether to impose a sanction and the type of sanction.
section_title: CMS Oversight
paragraphs: CMS, including its 10 regional offices, oversees state and accrediting  organization survey activities. CMS reviews and approves initial and  subsequent applications from exempt-state programs and accrediting  organizations to ensure CLIA equivalency. Validation reviews are one of  CMS’s primary oversight tools. Federal surveyors in CMS regional offices  are responsible for conducting validation reviews of state survey agency  and exempt-state program inspections, but state survey agency staff  conduct the validation reviews of accrediting organization inspections.  An objective of these reviews is to determine if all condition-level  deficiencies were identified. These reviews are conducted within 60 days  of a state’s or 90 days of an accrediting organization’s survey of a lab.  Starting in 1999, CMS required that at least one validation review be  conducted simultaneously with an accrediting organization’s survey, a step  intended to encourage an exchange of ideas and approaches among  surveyors. CMS also encourages the use of simultaneous reviews of state  survey agency inspections. By law, the number of labs selected for  validation reviews must be sufficient to allow a reasonable estimate of the  performance of each accrediting organization being assessed. CMS  requires fewer validation reviews of state survey agency lab surveys   (1 percent) than for those of exempt-state programs or accrediting  organizations (5 percent).
Beginning in 2003, CMS regional offices began reviewing the activities of  state survey agencies against a set of 13 performance standards. The  standards cover areas such as the timeliness of lab inspections, surveyor  personnel qualifications and training, CLIA data management, and the  handling of complaints. CMS’s goal is to evaluate each state survey  agency’s ability to carry out its CLIA responsibilities and to make  improvements. CMS is also developing performance standards for other  survey organizations that inspect labs using their own CLIA equivalent  requirements.
section_title: Insufficient Data Exist to Identify Extent of Serious Lab Quality Problems
paragraphs: The extent of serious quality problems at labs is unclear because CMS has  incomplete data on condition-level deficiencies identified by state survey  agencies prior to 2004. We also found that the lack of a straightforward  linkage between CLIA requirements and the CLIA-equivalent requirements  of some survey organizations makes it virtually impossible to assess lab  quality in a standardized manner, such as identifying the proportion of labs  with condition-level deficiencies. Such deficiencies indicate serious or  systemic quality problems. Proficiency testing results—the one available  data source that can be used to uniformly compare lab quality across  survey organizations—raise questions about whether lab quality has  improved in recent years.
subsections: section_title: Limited Data Are Available on the Quality of Labs Inspected by State Survey Agencies
paragraphs: CMS’s OSCAR database contains limited data on the quality of labs  inspected by state survey agencies and, as a result, it is not possible to  analyze changes in the quality of lab testing over time. In January 2004,  CMS implemented revised CLIA survey requirements and modified the  existing OSCAR data—state survey agency findings—to reflect the  changes. The revisions affected approximately two-thirds of the CLIA  condition-level requirements. As a result of the data modifications, the  findings for surveys conducted prior to 2004 no longer reflect all key  condition-level requirements in effect at the time of those surveys. Based  on the available 2004 OSCAR data (which represent about one half of all  labs surveyed by state survey agencies), we found that 6.3 percent of labs  had condition-level deficiencies (see app. II for data on all state survey  agencies, including the District of Columbia). As will be discussed below,  similar data are not available for labs surveyed by other survey  organizations.
section_title: Quality of Labs Inspected by Other Survey Organizations Is Very Difficult to Measure in a Standardized Manner
paragraphs: Differences between the inspection requirements that state survey  agencies use to measure lab quality and those of exempt-state programs  and accrediting organizations make it virtually impossible to measure lab  quality in a standardized manner. Because exempt-state programs and  accrediting organizations do not classify inspection requirements and  related deficiencies as either standard- or condition-level, they cannot  easily identify the number of CLIA condition-level deficiencies cited at the  labs they survey or the proportion of surveyed labs with condition-level  deficiencies.
We asked exempt-state programs and accrediting organizations what  percentage of their requirements, and any deficiencies cited for failure to  meet those requirements, indicated serious problems that were equivalent  to CLIA condition-level deficiencies. While only 8 percent of CLIA  requirements used by state survey agencies are classified as condition- level and therefore serious, the proportion of requirements that exempt- state programs and accrediting organizations classify as serious ranged  from 20 percent up to 100 percent (see table 1).
CAP and COLA crosswalked their recent survey findings to CLIA  condition-level requirements. Although their analysis suggested that from  about 56 to 68 percent of labs surveyed during 2004 had a deficiency in at  least one condition-level requirement, they acknowledged that these  proportions overstated the subset of labs with serious problems. JCAHO  did not crosswalk its inspection requirements to those of CLIA because  staff would have had to manually review each survey report to determine  which deficiencies were equivalent to deficiencies in CLIA condition-level  requirements. However, JCAHO did tell us that in 2004, about 90 percent  of the labs it surveyed had a deficiency in at least one requirement and, as  previously noted, JCAHO classifies all its requirements as serious.
Despite the difficulty of identifying CLIA equivalent condition-level  deficiencies, two of the three accrediting organizations we reviewed have  systems to identify labs they survey that have serious quality problems.  COLA estimated that about 9 percent of labs it surveyed in 2004 were  subject to closer scrutiny because of the seriousness of the problems  identified. According to JCAHO, about 5 percent of the labs it surveyed in  2004 were not in compliance with a significant number of requirements.  The third accrediting organization, CAP, has criteria for identifying labs  that warrant greater scrutiny, but CAP officials told us that identifying  such labs had to be accomplished on a case-by-case basis rather than  through a database inquiry. As a result, CAP plans to spend in excess of   $9 million during 2006 and 2007 to develop an integrated data system that  pulls together multiple factors—survey results, complaints, proficiency  testing, findings of other inspection bodies, and changes in lab directors— to enable it to readily identify problem labs. According to CAP officials,  such labs will be targeted for greater monitoring, and CMS and other  survey organizations will be notified about CAP’s actions.
section_title: Proficiency Testing Results Suggest that Quality Has Not Improved at Hospital Labs in Recent Years
paragraphs: Although CMS noted that proficiency testing trend data show a decrease in  failures for labs as a whole, the data suggest that lab quality may not have  improved at hospital labs for the period 1999 through 2003. Proficiency  testing is an important oversight tool for survey organizations because it is  an objective indicator of a lab’s ability to consistently produce accurate  test results and is conducted more frequently than surveys—three times a  year versus once every 2 years. In the absence of comparable survey data,  proficiency testing results provide a uniform way to assess the quality of  lab testing across survey organizations.
Our analysis of CMS proficiency testing data for 1999 through 2003  suggests that there has been an increase in proficiency testing failures for  labs inspected by CAP and JCAHO, which generally inspect hospital labs,  and a decrease in such failures for labs surveyed by state survey agencies  and COLA, which tend to inspect physician office labs (see fig. 2). CMS  defines failures as unsatisfactory performance in two consecutive or two  out of three proficiency testing events. For example, the percentage of  labs with proficiency testing failures surveyed by CAP and JCAHO from  1999 through 2003 increased from 4.1 percent to 6.8 percent and from   6.6 percent to 7.8 percent, respectively. It is unclear, however, whether the  decrease in failures for physician office labs represents an actual  improvement in lab quality or reflects the fact that some problematic labs  are no longer surveyed. Specifically, many physician office labs now  perform waived tests and therefore are no longer surveyed or participate  in proficiency testing. Between 1998 and 2005, the percentage of labs  subject to surveys and proficiency testing decreased from about   30 percent to about 19 percent.
section_title: Oversight Weaknesses Mask Quality Problems
paragraphs: Weaknesses in surveys, complaint processes, and enforcement mask real  and potential quality problems at labs. Survey weaknesses include:   (1) inspections that most organizations announce ahead of the visit, which  allows labs to prepare for their inspections and portray themselves in a  manner that may not accurately reflect their day-to-day quality assurance  processes; (2) variability in the proportion of labs with condition-level  deficiencies in 2004, which suggests surveys are not conducted in a  consistent manner; and (3) the goal of educating lab workers during  surveys taking precedence over, or precluding, the identification and  reporting of deficiencies. Furthermore, the significant increase in  complaints since CAP took steps to help ensure that lab workers know  how to file a compliant suggests that some quality problems at labs  inspected by other survey organizations may not be reported. Finally,  sanctions are not being used effectively as an enforcement tool to promote  labs’ compliance with CLIA requirements, as evidenced by the relatively  few labs with repeat condition-level deficiencies on consecutive surveys  from 1998 through 2004 that had sanctions imposed.
subsections: section_title: Announced Surveys May Result in Unrealistic Picture of Lab Quality
paragraphs: Because labs can and do prepare for surveys, CMS regional office officials  and most of the state survey agencies acknowledged that announced  surveys may not always provide a realistic picture of lab quality. As  shown in table 2, the amount of advance notice for surveys varies from as  little as 2 weeks up to 12 weeks; until recently, only the New York CLIA- exempt program conducted unannounced surveys. Survey agency officials  in two states told us that surveyors had inspected labs where records  documenting the implementation of periodic quality control procedures  were completed in the same handwriting using the same colored pen. This  degree of uniformity raises a concern about whether the quality control  occurred at all, or as frequently as the records suggested. A CAP surveyor  told us that the pathologist at one lab had cleaned up, and signed off on,  about 3-months worth of quality control records the night before the  survey. In hearings on the questionable test results at a Maryland hospital  lab, a worker testified that lab staff prepared frantically for their  announced inspections.
In 2006, both CAP and JCAHO began conducting unannounced inspections  at most of the hospital labs they survey. Both CAP and JCAHO officials  told us that the unannounced surveys will occur as early as 6 months prior  to the anniversary of a lab’s prior survey. CMS and survey organizations  that inspect physician office labs provided several justifications for  continuing to announce inspections at such labs, including (1) ensuring  that the lab is open and that appropriate personnel are available to answer  surveyor’s questions, and (2) minimizing disruptions to patient care. These  justifications appear to be reasonable because they reflect the operating  tempo at physician office labs. It may not be appropriate, however, to  provide such labs with 4 to 12 weeks advance notice, given that CMS  currently limits the advance notice provided by state survey agencies to no  more than 2 weeks.
section_title: Variability in Reported Survey Deficiencies Suggests that Labs Are Not Surveyed Consistently
paragraphs: Variability in OSCAR data for state survey agency inspections conducted  in 2004 suggests that labs are not surveyed in a consistent manner, and  interviews with CMS and state survey agency officials confirmed this  hypothesis. As a result, available data likely understate the extent of  serious quality problems at labs. In 2004, the percentage of labs that state  survey agencies reported with condition-level deficiencies varied  considerably by state, ranging from none in 6 states to about 25 percent of  labs in South Carolina. These data only included findings for about one- half of the labs surveyed by state survey agencies. Of the 33 states that  survey more than 100 labs, 16 found condition-level deficiencies at fewer  than 5 percent of labs, while 6 states identified such serious deficiencies in  more than 10 percent of the labs they surveyed (see app. II).
Based on interviews with CMS and 10 state survey agencies, it appears  that at least some of this variability is due to differences in states’  approaches to surveys as opposed to true differences in lab quality. For  example, CMS told us that, because there is not a prescriptive checklist to  guide the survey process, the reliance on state surveyor judgment will  result in variations in the citing of deficiencies. To compensate for the  unstructured nature of the state survey process, officials we interviewed  from 2 state survey agencies told us that they created checklists to help  ensure that surveyors looked at all of the critical elements during lab  surveys. Furthermore, while some of the state survey agencies we spoke  with told us that their surveyors always cite condition-level deficiencies  that are identified during surveys, officials in other states said that there  are circumstances under which condition-level deficiencies would not be  cited. For example, according to officials from a state survey agency we  interviewed, surveyors prefer not to cite condition-level deficiencies.  Rather, surveyors in this state prefer to cite multiple standard-level  deficiencies instead of a condition-level deficiency because it allows the  imposition of state law sanctions, avoiding what was characterized as a  less efficient federal sanctions process. Additionally, officials from   2 other state survey agencies explained that surveyors consider a lab’s  compliance history when determining what deficiencies to cite, while  officials from a third state told us that surveyors will educate lab workers,  particularly new lab workers, about the CLIA requirements rather than  citing CLIA condition-level deficiencies.
section_title: Balance Between Educational and Regulatory Roles by CMS and Survey Organizations Appears to Be Inappropriate
paragraphs: The goal of educating lab workers sometimes takes precedence over, or  precludes, the identification and reporting of deficiencies that affect the  quality of lab testing. As a result, data on the quality of lab testing and  trends in quality over time may be misleading. Although CLIA neither  requires nor precludes an educational role for surveyors, the preamble to  CMS’s implementing regulation noted that surveys are intended, in part, to  provide an opportunity for on-site education regarding accepted  laboratory procedures. In addition, CMS guidance and training encourage  state surveyors to play an educational role. Many state survey agency  officials we interviewed also told us that their surveyors play a major  educational role. As noted earlier, surveyors from one state survey agency  do not cite condition-level deficiencies when lab workers are new but  prefer to educate the new staff. Because CMS revised its OSCAR database  in 2004, it is not possible to identify states that have consistently not cited  condition-level deficiencies, data that would help to quantify the extent to  which an educational role is substituting for appropriate regulation of  labs.
An inappropriate balance between the educational and regulatory role is  also evident in some accrediting organization practices. One of the CAP  surveyors we interviewed with over 30 years of lab experience estimated  that the majority of pathologists—individuals who generally serve as CAP  survey team leaders—view surveys as educational, rather than as  assessments of compliance with lab requirements. Another surveyor told  us that CAP’s survey process focuses heavily on education, and that some  survey team leaders emphasize education more than others. For COLA, the  process of educating labs begins even prior to a survey. For example,  COLA encourages labs to submit a self-assessment for review prior to the  scheduled survey so that the labs can identify COLA requirements with  which they are not in compliance. About 20 percent of all labs surveyed  during 2004 submitted a self-assessment (616 labs) and, compared to labs  that did not submit a self-assessment, fewer deficiencies were identified at  these labs during on-site surveys.
CMS appears to be inappropriately stressing education over regulation in  its implementation of (1) 2003 lab quality control requirements for the  CLIA program and (2) proficiency testing for lab technicians who interpret  Pap smears, a test for cervical cancer. When state surveyors began  assessing compliance with new lab quality control requirements in January  2004, they were instructed to note deficiencies on a cover letter to labs  rather than on the survey report itself for a period of 2 years. Thus, such  deficiencies are not recorded in the OSCAR database. In part because of a  lack of lab “buy-in” for some of the new policies and procedures, CMS  officials have extended the educational period for about another 2 years.  CMS has taken a similar educational approach to Pap smear proficiency  testing, which began in 2005. CMS will not cite deficiencies or impose  sanctions against labs in which staff fail the new Pap smear proficiency  testing in 2005 or 2006, as long as the labs and individuals involved  complete such testing, including following the regulatory protocol for  subsequent testing in the case of an initial failure. According to CMS, this  educational focus allows labs and their staff to become familiar with the  proficiency testing program and to prepare themselves for such testing,  since there was about a 13-year time lag between the 1992 regulations that  implemented CLIA and the 2005 implementation of Pap smear proficiency  testing. This educational approach seems questionable given CMS’s  concern about some of the high initial proficiency test failure rates.
section_title: Use of Volunteer Surveyors by CAP Raises Concerns
paragraphs: Although state survey agencies, exempt-state programs, COLA, and  JCAHO employ dedicated staff surveyors, CAP relies primarily on  volunteer teams consisting of lab workers from other CAP-inspected labs  to conduct surveys. In contrast to the mandatory training and continuing  education programs in place for the staff surveyors of other survey  organizations, training for CAP’s volunteer surveyors is currently optional.  CAP plans to establish a mandatory training program beginning in mid- 2006. As a condition of accreditation, labs inspected by CAP must survey  another CAP-accredited lab of similar size and composition at least once  every 2 years. According to data provided by CAP, two-thirds of volunteer  surveyors who had recently participated in a survey had no formal training  in the 3 to 5 years preceding the survey. Two CAP surveyors we  interviewed told us that they had not completed any training because it  was optional. Two other surveyors told us that they had never been  notified about the existence of optional CAP training. While full-time  surveyors employed by other survey organizations conduct from 30 to  about 200 surveys per year, CAP volunteer surveyors have much less  experience conducting surveys because they only survey about one lab  each year.
Three of the nine CAP surveyors we interviewed stated that they believed  that mandatory training was important, but some surveyors wondered  when lab workers would have time to complete the courses because of  their demanding work schedules. According to CAP officials, however, the  required training will take only 1 to 2 days and surveyors will have a  choice of live seminars and workshops or e-learning completed at their  own computers. For ongoing training requirements, CAP plans to give  surveyors a choice of taking additional training or passing a competency  evaluation. CAP will track compliance with its new training requirements  to ensure that surveyors successfully complete training and demonstrate  competency within 2 years of participating in a survey. CAP’s required  training is less extensive than that required by other survey organizations.  For example, state survey agency inspectors must complete 5 days of  basic training and periodic advanced courses afterwards while COLA staff  inspectors participate in a 5-week orientation program and an annual   20 hours of continuing education.
The use of volunteer inspectors by CAP also raises concerns about the  appearance of a conflict of interest. These concerns arise because of the  way CAP survey teams are structured. CAP’s Commission on Laboratory  Accreditation policy manual specifies that the inspection team leader is  the individual responsible for the conduct of an ongoing site inspection,  and must not be in a business, professional, or personal relationship that  would preclude an objective inspection. Furthermore, the manual states  that the inspection team leader is usually responsible for determining the  size of, and assembling, the inspection team. However, until April 2006,  CAP policy did not preclude competing labs from surveying one another or  lab survey team members from soliciting business, such as referrals, from  a lab at the conclusion of the survey. The policy was also silent about  survey team members’ business, professional, or personal relationships  that could cloud their independence. Typically, inspection team leaders  are pathologists who direct other labs in the community, and the  inspection team is comprised of several employees from the team leader’s  lab.
We believe that the use of volunteers, including those from nearby labs,  and the personal and professional relationships that may exist among lab  staff and survey team members, creates the appearance of a conflict of  interest and could undermine the integrity of the survey process.  Comments from some CAP surveyors we interviewed raise a concern  about having survey team leaders who are also the day-to-day supervisors  of team members. For example, lack of agreement about the seriousness  of a deficiency could result in the team leader instructing the team to  downgrade the deficiency to a recommendation, a less serious finding that  does not appear in the inspection report. Team members who are  subordinates to the team leader may feel that they have no other recourse  than to follow the team leader’s instructions. Recognizing that team  members’ objectivity may be compromised in this situation, CAP’s revised  conflict of interest policy instructs all parties to be cautious to retain  objectivity in fact finding throughout the inspection process.
In discussing these findings with CAP officials, they told us that they plan  to institute a number of initiatives to help ensure survey objectivity,  including (1) resurveying the same lab by an independent team to assess  the consistency of inspections, (2) centralizing survey team assignments  performed by CAP staff, (3) not announcing surveys, and (4) not notifying  labs of the survey team composition prior to the survey.
section_title: Lab Workers Who File Complaints About Quality Problems in Lab Testing Not Afforded Whistle- blower Protections
paragraphs: Some lab workers may not be filing complaints about quality problems at  their labs because of anonymity concerns or because they may not be  familiar with filing procedures. Complaints about labs can come from a  variety of sources, including lab workers. Complaints are an important  tool in detecting quality problems between lab surveys. For example,  complaints about testing at a hospital lab were crucial because  information had been concealed, complicating the detection of quality  problems during the lab’s surveys. As a result of a complaint, surveyors  were able to substantiate inadequate calibration of testing equipment that  could adversely affect patient care.
Based on OSCAR data and data obtained from exempt-state programs and  accrediting organizations for 2002 through 2004, few complaints were  received about lab testing relative to the number of labs—significantly less  than one complaint per lab per year. The low volume of lab complaints  may be related to complainants’ concerns about anonymity and fear of  retaliation for filing a complaint. It may be easy for a lab to determine the  source of a complaint filed by a lab worker. For example, in some cases,  either the nature of the complaint or the piece of testing equipment in  question could narrow the list of possible complainants. Two CAP  surveyors we interviewed commented that, in their opinion, it would be  easy to determine the identity of a complainant. During congressional  hearings in 2004, a Maryland hospital lab worker testified that she and her  colleagues feared losing their jobs because of the complaints they filed.
Because of the difficulty of protecting the anonymity of lab workers who  file complaints, whistle-blower protections for such individuals are  particularly important. Two of the three accrediting organizations we  interviewed have whistle-blower protections—CAP and JCAHO. For  example, CAP implemented a comprehensive whistle-blower protection  policy in July 2004 that includes revocation of accreditation or other  appropriate action for any lab that directly or indirectly threatens,  intimidates, or retaliates against a lab worker. While officials from New  York and Washington’s exempt-state programs told us that whistle-blower  laws in their states provide some protection for lab workers who file  complaints, officials in most of the other 10 states we interviewed told us  that they did not have any whistle-blower protections or were unable to  identify specific protections that applied to lab workers in their state.  Currently, there are no federal whistle-blower protections specifically for  workers in labs covered by CLIA. In 2005, legislation was introduced to  provide whistle-blower protections to workers in labs covered by CLIA.
We also found that lab workers may not know how to file a complaint.  CAP experienced a significant increase in the number of complaints it  received since October 2004, when it began requiring CAP-inspected labs  to display posters on how to file complaints. Specifically, from October  through December 2004, CAP received an average of 22 complaints per  month, compared to an average of 11 complaints per month in the   9 months preceding the poster requirement. As a result, the number of  complaints about the quality of lab testing more than doubled in 2004 and  the number substantiated increased by more than 40 percent—even  though the poster was only displayed for the last 3 months of 2004 (see  table 3). In September 2005, COLA also began requiring labs to display a  complaints poster similar to CAP’s. It is too early, however, to determine  the impact of COLA’s new complaints poster on the number, type, and  substantiation rate of complaints. Neither CMS nor JCAHO plans to  require a similar complaints poster.
section_title: Lab Sanctions Are Rarely Imposed
paragraphs: Few labs were sanctioned by CMS from 1998 through 2004—even those  with the same condition-level deficiencies on consecutive surveys— because many proposed sanctions are never imposed. Our analysis of CMS  enforcement data from 1998 through 2004 found that 501 labs were  sanctioned, which equates to less than 3 percent of labs inspected by state  survey agencies. The most common were principal sanctions, which may  result in suspension or limitation of testing or termination from the CLIA  program; few labs were subjected to alternative sanctions, such as  directed plans of correction or civil monetary penalties (see table 4).  Appendix III shows the number of labs surveyed by state survey agencies  and the number of sanctioned labs from 1998 through 2004.
Although few labs were sanctioned from 1998 through 2004, over 9,000  labs had sanctions proposed during that same time period. Before  sanctions go into effect, labs are given a grace period to correct condition- level deficiencies, unless the deficiencies involve immediate jeopardy, that  is, an imminent threat to patient health and significant hazard to public  health. Most labs correct the deficiencies within the grace period. CMS  officials told us that it was appropriate to give labs an opportunity to  correct such deficiencies within a prescribed time frame and thus avoid  sanctions. However, a principal objective of the enforcement process— one reflected in CMS guidance—is to motivate labs to comply with CLIA  requirements, thereby helping to ensure the provision of accurate and  reliable test results. Based on the large number of labs with proposed  sanctions that were never imposed, it is unclear how effective the  enforcement process is at motivating labs to consistently comply with  CLIA requirements.
The number of labs with the same repeat condition-level deficiencies from  one survey to the next also raises questions about the overall effectiveness  of the CLIA enforcement process. From 1998 through 2004, 274 labs  surveyed by state survey agencies had the same condition-level deficiency  cited on consecutive surveys and 24 of these labs had the same condition- level deficiency cited on more than two surveys. This analysis may  understate the percentage of labs with repeat condition-level deficiencies  because OSCAR data prior to 2004 no longer reflect about two-thirds of  condition-level requirements and associated deficiencies at the time of  those surveys. We found that only 30 of the 274 labs with repeat condition- level deficiencies had sanctions imposed—either principal, alternative, or  both. According to the CLIA legislative history, congressional concern  about labs with repeat deficiencies led to alternative sanctions to provide  an enforcement option short of principal sanctions to encourage  compliance.
From 1998 through 2004, less than 1 percent of accredited labs (81) lost  their accreditation; few of these labs were subsequently sanctioned by  CMS and many still participate in the CLIA program. Our analysis of CMS  reports on sanctioned labs found that only 9 of the 81 labs had either  principal and/or alternative sanctions imposed and that 1 of the 9 still  performs moderate- to high-complexity testing. Based on a review of its  CLIA certificate database, CMS officials told us that about half of the 81  labs still perform moderate- to high-complexity testing but could not  describe the actions taken by CMS regional offices in response to the loss  of accreditation. We contacted state survey agencies or CMS regional  office officials to determine why 3 labs that COLA concluded had cheated  on proficiency testing by referring the samples to another lab to be tested  had no sanctions imposed. The purpose of proficiency testing is to provide  an objective, external evaluation of the accuracy of a lab’s test results,  which is negated when another lab analyzes the sample. By statute, the  intentional referral of samples to another lab for proficiency testing is a  serious deficiency that should result in automatic revocation of a lab’s  CLIA certificate for at least 1 year. Based on our interviews, we found  that the 3 labs were allowed to continue testing because they had initiated  corrective actions; in effect, these labs were given an opportunity to  correct a deficiency that appears to have required a loss of their CLIA  certificate for at least 1 year. A fourth lab was ultimately sanctioned for  proficiency testing cheating by CMS but was allowed to continue testing  for almost 2 years after having its accreditation revoked.
We also attempted to analyze data on other actions, short of revoking  accreditation, used by accrediting organizations to encourage lab  compliance and, in particular, how they respond to labs with serious  repeat deficiencies. According to CMS, this information is dispersed  across CMS regional offices. CAP officials told us that they could initiate  four intermediate actions including probation (lab is closely watched to  ensure correction of problems), accreditation with conditions (nonroutine  inspection to be scheduled), suspension of a lab section, and cessation of  a specific type of testing; suspension and probation were instituted in  2004. According to CAP, in 2005, 28 labs were on probation, 106 labs were  accredited with conditions, 1 lab was suspended, and 7 labs were required  to cease a specific type of test. In 2004, JCAHO awarded conditional  accreditation to 3 percent of the labs it inspects because they were not in  substantial compliance with its survey requirements, as evidenced by the  number of requirements not met; JCAHO conducts an on-site follow-up  survey at such labs. From 2002 through 2004, COLA required about 30 labs  per year to cease testing due to issues identified during surveys and about  217 labs per year to cease testing certain tests or specialties due to  unsuccessful proficiency testing.
section_title: CMS Oversight of CLIA Is Inadequate
paragraphs: CMS’s oversight is not adequate to help ensure that labs meet CLIA  requirements. While CLIA requires proficiency testing quarterly, CMS only  requires such testing three times each year. In addition, the agency is not  meeting its responsibility to determine that accrediting organization and  exempt-state requirements and processes continue to be at least  equivalent to CLIA’s. CMS attributed the delay in making equivalency  determinations to having too few staff. Further, ongoing CMS validation  reviews do not provide an independent assessment of the extent to which  surveys identify all condition-level deficiencies—primarily due to their  timing. Finally, CMS does not adequately use data, such as the results of  surveys, to monitor survey organization activities and processes. Realizing  that its existing oversight activities need to be strengthened, CMS has  begun instituting performance reviews to measure survey organization  compliance with its standards and is developing protocols to ensure  improved communication among survey organizations concerning  complaints about lab quality.
subsections: section_title: CMS’s Implementation of Proficiency Testing Is Inconsistent with CLIA
paragraphs: CMS’s decision to require proficiency testing for almost all laboratory tests  only three times a year is inconsistent with the statutory requirement.  CLIA requires that proficiency testing be conducted “on a quarterly basis,  except where the Secretary determines for technical and scientific reasons  that a particular examination or procedure may be tested less frequently  (but not less often than twice per year).” The committee report on the bill  that forms the basis for much of CLIA indicated that “proficiency testing  should be the central element in determining a laboratory’s competence,  since it purports to measure actual test outcomes rather than merely  gauging the potential for accurate outcomes.”
In CMS’s 1992 rule implementing CLIA, the agency provided a rationale for  reducing the frequency of proficiency testing, but did not provide a  technical and scientific basis for reducing the frequency for particular  procedures or tests. According to CMS’s justification, experts were  divided on the appropriate frequency of proficiency testing generally. In  addition, requiring fewer events of proficiency testing would give  laboratories more time to analyze the causes of test failures before the  next event of proficiency testing and also enhance proficiency testing’s  value as an educational tool. Because CLIA increased the number of labs  that were required to undergo proficiency testing, CMS believed that the  number of organizations that provided proficiency testing services would  not have been able to meet the anticipated increase in demand for testing  services. To help avoid anticipated delays in completing proficiency  testing and reporting requirements, CMS reduced the frequency of testing  events to three times per year.
CMS’s requirement for proficiency testing does not meet the conditions  specified in the statute that must be satisfied in order to require testing  less frequently than quarterly. The language of the statute, as well as  relevant legislative history, indicate that a decision to reduce the  frequency of testing should be in the nature of an exception made with  regard to a particular test, not the norm for all tests, and must be based on  “technical and scientific” considerations related to that particular test.  The reasons that CMS gave for requiring only three events per year were  not based on scientific and technical considerations relevant to particular  tests. Instead, CMS’s decision was based on concerns of an administrative  and logistical nature that CMS wanted to alleviate by reducing the  frequency of testing events.
section_title: CMS Is Late in Ensuring CLIA Equivalency of Exempt States’ and Accrediting Organizations’ Inspection Requirements and Processes
paragraphs: We found that CMS has been late in determining that exempt states’ and  accrediting organizations’ inspection requirements and processes are at  least equivalent to CLIA’s. CMS must verify their equivalency and, by  regulation, CMS requires such survey organizations to seek reapproval at  least once every 6 years, or more frequently if deemed necessary. CMS  establishes the time frames for when the next reapproval should occur,  which have ranged from about 15 months to about 6 years. However, CMS  has not completed its equivalency reviews within these time frames and  accrediting organizations and exempt state programs have continued to  operate without proper approval. Equivalency reviews for CAP, COLA,  JCAHO, and Washington due to be completed between November 1, 1997,  and April 30, 2001, were an average of about 40 months late. In August  1995, CMS determined that New York’s next equivalency review should be  completed by June 30, 2001, but was over 4 years past due as of December  2005. Similarly, COLA’s equivalency review was about 3 years past due.
Because accrediting organizations and exempt-state programs may choose  to make changes to their inspection requirements between periodic  equivalency reviews (1) accrediting organizations are required to submit  changes to their inspection requirements and policies 30 days prior to  changing their standards and (2) exempt-state programs are required to  provide notice when they change their licensure or inspection  requirements. Although federal regulations require CMS to review  equivalency when an accrediting organization or exempt-state program  adopts new requirements, a CMS official told us that the agency is not  required to review such changes before their implementation to ensure  equivalency. As a result, such survey organizations may introduce  changes that are inconsistent with CLIA requirements. For example,  JCAHO made a significant change to its inspection requirements in  January 2004 but did not receive CMS approval until 6 months later; CMS  did not begin an in-depth review of JCAHO’s revised requirements until  early 2005—over a year after they were implemented by JCAHO.  According to CMS, their review has identified several critical areas where  JCAHO standards are less stringent than those of CLIA. JCAHO  acknowledged the need to make some adjustments to its revised  requirements.
CMS officials attributed delays in making equivalency determinations and  reviewing interim changes to having too few staff. The CLIA program,  located in CMS’s Center for Medicaid and State Operations (CMSO),  currently has approximately 21 full-time-equivalent positions compared to  a peak of 29 such positions several years ago. The reduction occurred over  time through attrition. As required by statute, the CLIA program is funded  by lab fees and since its inception the program’s fees have exceeded  expenses. As of September 30, 2005, the CLIA program had a carryover  balance of about $70 million—far more than required to hire an additional  six to seven staff members. However, CMS officials told us that because  the CLIA program staff are part of CMSO, they are subject to the personnel  limits established for CMSO, regardless of whether or not the program has  sufficient funds to hire more staff. Although CMSO is at its authorized  personnel allocation, the CLIA program could hire additional staff with  approval from the Administrator. We were told that CMSO has not  requested such approval.
We also noted issues that raise a question about the thoroughness of CMS  equivalency reviews because some survey organizations’ procedures or  policies appear to be less stringent than those required by CMS for the  CLIA program. For example:    Accrediting organizations provide labs more advance notice about  upcoming surveys than CMS allows state survey agencies to give to the  labs they inspect.
JCAHO surveyors focus their review of lab testing on the 12 months prior  to the survey. CMS requires that state surveyors review the entire   24 months of testing since the last survey.
While CMS requires initial and advanced surveyor training, CAP  encourages but does not require its volunteer surveyors to participate in  surveyor training.
As of August 2005, CAP’s policy manual indicates that complaint  investigations may be announced or unannounced. CMS guidance requires  that complaint investigations be unannounced.
Prior to 2005, CMS’s equivalency determination reviews focused on the  inspection requirements themselves, and not the procedures and policies  used by accrediting organizations and exempt-state programs in carrying  out oversight of labs; this focus on inspection requirements may explain  the divergence from the policies and procedures CMS requires for state  survey agencies. During 2006, CMS is simultaneously reviewing the  equivalency of COLA and JCAHO inspection requirements and, for the first  time, incorporating on-site observations of accrediting organization  policies and systems into the review and approval process. For example,  CMS is checking to ensure that accrediting organizations have adequate  systems in place to track such things as (1) complaints, (2) correction of  deficiencies, and (3) proficiency testing.
section_title: Many CMS Validation Reviews Lack Independence and Reviews Skip Some State Survey Agencies
paragraphs: CMS validation reviews that are intended to evaluate lab surveys  conducted by both states and accrediting organizations do not provide  CMS with an independent assessment of the extent to which surveys  identify all serious—that is, condition-level or condition-level equivalent— deficiencies. CMS requires its regional offices to conduct validation  reviews of 1 percent of labs inspected by state survey agencies in a year. In  contrast, validation reviews of 5 percent of labs inspected by accrediting  organizations during a year are conducted by state survey agency  personnel. CMS does not specifically require that validations occur in each  state and some states do not have validation reviews each year.  Furthermore, many validation reviews occur at the same time a survey  organization conducts its inspection and, in our view, the collaboration  among the two teams during these simultaneous surveys prevents an  independent evaluation.
subsections: section_title: Validation of State Survey Agency Lab Surveys
paragraphs: The requirement to validate 1 percent of labs surveyed by state survey  agencies in a year—roughly 100 validation reviews each year—does not  ensure sufficient oversight of state survey agencies. The validation review  requirement, which is included in CMS regional office annual budget  memorandums, does not specify how many validation reviews must be  conducted in each state. While the 10 CMS regional offices generally  validated 1 percent of the state survey agency inspections within their  region, they often did not validate 1 percent of inspections within each  state and, in fact, performed none in some states. From 1999 through 2003,  federal surveyors:    validated less than 1 percent of labs surveyed by state survey agencies in  an average of about 25 percent of states, ranging from 7 states in 2002 to  17 states in 2003; and    did not conduct any validation reviews in an average of 16 percent of  states per year, ranging from 3 states in 2002 to 12 states in 1999.
In 11 states, no validation reviews were conducted in multiple years. For  example, no validation reviews were conducted in Michigan and  Washington, D.C. during 4 of 5 years from 1999 through 2003. Without  validating at least some surveys in each state, CMS is unable to determine  if the states are appropriately identifying deficiencies.
Seventy-five percent of validations of state lab surveys were conducted  simultaneously from fiscal years 1999 through 2003. According to CMS  officials, the large proportion of simultaneous validation reviews provides  an opportunity for federal surveyors to share information with state  surveyors, monitor their conformance with CLIA inspection requirements,  and identify training and technical assistance needs. However, we found  that such reviews do not provide an accurate assessment of state  surveyors’ ability to identify condition-level deficiencies. Of the 13  validation reviews that identified missed condition-level deficiencies, only  1 was a simultaneous review (see table 5). Validations of state surveys  typically utilize one federal surveyor for either independent or  simultaneous validation reviews; therefore, increasing the proportion of  independent validation reviews to strengthen CMS oversight likely would  not require additional federal surveyors. Moreover, conducting  independent validation reviews eliminates the extra effort required to  coordinate schedules to ensure that the validation reviews occur at the  same time as the state survey.
According to CMS guidance, at least one validation review of an  accrediting organization’s survey of labs should be conducted  simultaneously each year, but not all validation reviews should be  simultaneous because a combination of simultaneous and independent  reviews provides a balanced view of surveyor performance. CMS officials  were unable to tell us exactly how many of the roughly 275 validation  reviews conducted each year from fiscal year 1999 through fiscal year 2003  were simultaneous. However, one of the three accrediting organizations  we reviewed told us that a significant proportion of their validation  reviews are conducted simultaneously. JCAHO estimated that 33 percent  of its validation reviews were conducted simultaneously. COLA estimated  that 9 percent of validation reviews conducted in 2004 and 2005 were  simultaneous. Finally, CAP officials told us that, from 2002 through 2004,  11 percent of validation reviews of CAP-accredited labs were conducted  simultaneously.
Given the limitations of simultaneous reviews, conducting independent  validation reviews are a more effective way of ensuring the equivalency of  accrediting organization inspection requirements and processes between  the equivalency determinations. CMS officials told us that the agency’s  intent in instituting simultaneous reviews was for state and accrediting  organization surveyors to share best practices, to promote understanding  of each other’s programs, and to foster accrediting organization  improvement. They indicated that they considered it a learning experience  both if an accrediting organization surveyor added a deficiency noted by a  state surveyor to a survey report and vice versa. However, most of the  state survey agency officials we interviewed told us that simultaneous  validation reviews do not provide a realistic evaluation of the adequacy of  accrediting organizations’ inspection process. In fact, CMS guidance  encourages surveyors to discuss the survey findings prior to concurrent  conferences with lab personnel to review their findings.
From fiscal years 1999 through 2003, state survey agency surveyors found  condition-level deficiencies missed by accrediting organization surveyors  on 64 validation reviews, but only 6 of these validation reviews were  simultaneous. In contrast, 58 (91 percent) of the validation reviews that  identified serious deficiencies missed by accrediting organizations were  independent validation reviews. (See table 6.)
section_title: CMS Use of Data for Oversight of CLIA Program Is Limited
paragraphs: CMS does not routinely collect and analyze data essential for effective  oversight of the CLIA program but has initiatives to automate some  available data to make them more accessible for analysis. Using data to  analyze activities across survey organizations can be a powerful tool in  improving CMS oversight of the CLIA program. Such analyses include  identifying and addressing inconsistencies in how surveys are conducted.  Although CMS tracks the most frequently cited deficiencies at labs in an  effort to improve quality, it does not routinely track the proportion of labs,  by state, in which state survey agencies identify condition-level  deficiencies—those that denote serious or systemic problems. According  to a CMS official, the agency has not evaluated variability in such  deficiencies since 1999. As noted earlier in this report, variability in  survey findings suggests inconsistencies in how surveys are conducted.  CMS also does not require exempt-state programs and accrediting  organizations to routinely submit data on serious deficiencies identified at  the labs they inspect, unless the deficiencies pose immediate jeopardy to  the public or an individual’s health. As noted earlier, the lack of a  common vocabulary on what constitutes a serious deficiency would make  it virtually impossible for CMS to analyze such data.
We also found that CMS does not effectively use available data to assess  clinical lab quality in areas such as proficiency testing, sanctions, and  complaints. For example, CMS’s analysis of proficiency testing data for all  labs showed improvements over time. As reported earlier, proficiency  testing failures have increased for labs surveyed by CAP and JCAHO.  Comprehensive analysis of the proficiency testing database is particularly  valuable because it provides a uniform way to assess the quality of lab  testing across survey organizations, which is not currently available for  survey results. CMS is now in the process of automating the annual  registry of sanctioned labs, which should help it identify important trends,  such as the infrequent use of alternative sanctions. Automating the  registry, however, will not address the lack of data on (1) steps taken by  state survey agencies and regional offices when labs have their  accreditation revoked or (2) interim steps, short of revocation of  accreditation, that accrediting organizations take to help encourage lab  compliance. CMS also lacks a complaints database, and therefore was  unable to assess the impact of CAP’s decision to require labs to  prominently display a poster on how to file a complaint. CMS is  developing, and plans to launch, a complaints database in March 2006.
section_title: CMS Implementing Performance Reviews for Survey Organizations
paragraphs: CMS has implemented performance reviews for state survey agencies and  is in the process of developing such reviews for accrediting organizations.  First implemented in 2004, the annual CLIA state performance reviews  evaluate each survey agency’s ability to accomplish its lab oversight  responsibilities. The reviews, conducted on-site by CMS regional office  staff, measure performance in 13 areas, such as the timely conduct of  surveys and the appropriate documentation of any deficiencies identified.  According to CMS, the reviews are based on the performance- improvement model that characterizes much of the administration of the  CLIA program. Consequently, the primary role of regional offices in  conducting the reviews is to provide education and support for state  survey agency improvement.  For the 2004 reviews, 38 states were  required to submit corrective action plans to their respective CMS regional  offices in at least 1 of the 13 areas examined. Three areas required the  most corrective action plans: principles of documentation, proficiency  testing desk reviews, and survey time frames.
Principles of documentation. CMS found that some state survey agencies  lacked the supervisory personnel to conduct internal reviews intended to  ensure the appropriate documentation of deficiencies. It also found that  some state survey agencies did not follow the protocol instructions on  how to quantify such reviews.
Proficiency testing desk reviews. Because of personnel shortages, some  state survey agencies were unable to perform proficiency testing desk  reviews between surveys, waiting instead until the next on-site survey to  address unsuccessful proficiency testing. CMS plans to provide additional  surveyor training on desk review requirements.
Survey time frames. CMS regional office staff were inconsistent in scoring  whether state survey agencies met the established time frames for initial  surveys. While some regions were lenient if a state missed the time frame  by just 1 day or provided a reasonable explanation—such as staff turnover  or illness—other regions were more stringent in scoring states against the  standard.
However, it is not yet clear to what extent the 2004 scores represent state  survey agency shortcomings or a learning curve for the states in  understanding the performance review protocols.
In partnership with the accrediting organizations, CMS is developing  performance standards comparable to, but different from, those  implemented in 2004 for state survey agencies. For example, both the state  survey agency review protocols and those proposed for accrediting  organizations measure the timeliness of the surveys, but those proposed  for the latter would also focus on several areas that are unique to  accrediting organizations. The performance standards would include   (1) timely and consistent information sharing and (2) alerting CMS about  decisions to limit or remove accreditation in a timely manner. According  to a CMS official, the agency plans to phase in the performance standards,  starting with a standard on complaints. For example, if the CLIA  complaints database is activated in March 2006, CMS could begin to  monitor accrediting organization responsiveness to, and outcomes of,  complaints. Because the database will contain national lab complaint data,  CMS will be able to compare the volume and outcome of complaints  across survey organizations. According to CMS, implementation of the  accrediting organization performance standards will be a central—not  regional—office responsibility.
section_title: Conclusions
paragraphs: Clinical labs play a pivotal role in the nation’s health care system by  diagnosing many diseases, including potentially life-threatening diseases,  so that individuals receive appropriate medical care. Given this important  role, lab tests must be accurate and reliable. CMS and survey organization  oversight is intended to ensure that labs produce reliable test results, a key  objective of CLIA. Our work demonstrated that the oversight of clinical  labs needs to be strengthened in several areas.
Determining the quality of lab testing is difficult because it is virtually  impossible to crosswalk inspection requirements across survey  organizations. Without standardized survey findings across all survey  organizations, CMS cannot tell whether the quality of lab testing has  improved or worsened over time or whether deficiencies are being  appropriately identified.
Lab oversight has weaknesses that make it difficult to determine the  quality of lab testing because they mask quality problems. To help surveys  provide a realistic picture of day-to-day operations, CAP and JCAHO began  unannounced surveys of the labs they survey—generally hospital labs—in  2006. While unannounced surveys at physician office labs may not be  practical, Washington’s exempt program and COLA currently give such  labs more advance notice than the 2 weeks CMS prescribes for labs  inspected by state survey agencies. Similarly, the greater weight that CMS  and survey organizations sometimes place on their educational, as  opposed to their regulatory role may lead to survey findings that do not  accurately reflect lab quality. Educating labs to ensure high-quality testing  should complement but not replace the enforcement of CLIA inspection  requirements. The low number of lab complaints may be the result of a  lack of information about how to file a complaint and lab workers’ fear of  retaliation. Because protecting the anonymity of lab workers who file  complaints is difficult, whistle-blower protections for such individuals are  particularly important. Finally, labs with the same serious deficiencies on  consecutive surveys often escape sanctions, even though Congress  authorized alternative sanctions to give CMS more flexibility to achieve  lab compliance. Without the threat of real consequences, labs may not be  sufficiently motivated to comply with CLIA inspection requirements.
CMS’s oversight is not adequate to enforce CLIA requirements. The agency  is not requiring labs to participate in proficiency testing on a quarterly  basis, as required by CLIA. Furthermore CMS is not conducting CLIA- equivalency determinations within the time frames it established for such  reviews, nor has it always reviewed changes to exempt-state and  accrediting organizations’ inspection requirements before their  implementation, even though it requires their submission to ensure  continued CLIA equivalency of their requirements. Although the CLIA  program has generated funds, CMS agencywide staffing limitations have  prevented the program from hiring sufficient staff to complete equivalency  reviews in a timely manner. Many validation reviews are conducted at the  same time a survey organization conducts its survey, and such  simultaneous reviews may not provide a true assessment of surveyor  performance. Independent validation reviews of accrediting organization  surveys are critical because CMS has not conducted equivalency reviews  within the time frames it established. We also found that few validation  reviews of state survey agency lab inspections are conducted each year  and that none occurred in some states. Because state surveyors conduct  validation reviews of accrediting organizations to ensure the continuing  CLIA equivalency of their inspection requirements, conducting an  appropriate number of validation reviews of state survey agency lab  inspections is critical. CMS also has not yet taken the lead in ensuring the  availability and use of data from survey organizations to help it monitor  their performance—particularly the consistency with which surveys are  conducted. CMS is creating a new complaint database, but its plan to  automate the existing sanctions registry will not address the lack of data  on enforcement actions taken by state survey agencies and regional offices  when labs have their accreditation revoked.
section_title: Recommendations for Executive Action
paragraphs: To enable CMS to track the nature and extent of lab quality problems  across survey organizations, we recommend that the CMS Administrator  take the following action:    Work with exempt-state programs and accrediting organizations to  standardize their categorization and reporting of survey findings in a way  that tracks to CLIA inspection requirements and allows for meaningful  comparisons across organizations, such as the analysis of trends in the  citation of condition-level deficiencies.
To ensure consistency in the oversight of labs by survey organizations, we  recommend that the CMS Administrator take the following four actions:    Ensure that the advance notice of upcoming surveys provided to physician  office labs is consistent with CMS’s policy for advance notice provided by  state survey agencies.
Ensure that regulation of labs is the primary goal of survey organizations  and that education to improve lab quality does not preclude the  identification and reporting of deficiencies that affect lab testing quality.  Impose appropriate sanctions on labs with consecutive condition-level  deficiencies in the same requirements.
Require all survey organizations to develop, and require labs to  prominently display, posters instructing lab workers on how to file  anonymous complaints.
To improve oversight of labs and survey organizations, we recommend  that the CMS Administrator take the following eight actions:    Consistent with CLIA, require quarterly proficiency testing, except when  technical and scientific considerations suggest that less frequent testing is  appropriate for particular examinations or procedures.
Ensure that evaluations of exempt-state and accrediting organization  inspection requirements take place prior to expiration of the period for  which they are approved in order to ensure the continued equivalency of  their requirements with CLIA’s.
Ensure that changes to the inspection requirements of exempt states and  accrediting organizations be reviewed prior to implementation, as required  by regulation, to ensure that individual changes do not affect the overall  CLIA equivalency of each organization.
Allow the CLIA program to utilize revenues generated by the program to  hire sufficient staff to fulfill its statutory responsibilities.
Ensure that federal surveyors validate a sufficient number of inspections  conducted by each state survey agency to allow a reasonable estimate of  their performance, including a minimum of one independent validation  review for each state survey agency surveyor.
Require that almost all validation reviews of each accrediting  organizations’ surveys be an independent assessment of performance.
Collect and routinely review standardized survey findings and other  available information for all survey organizations to help ensure that CLIA  requirements are being enforced and to monitor the performance of each  organization.
Establish an enforcement database to monitor actions taken by state  survey agencies and regional offices on labs that lose their accreditation.
section_title: Agency and Accrediting Organization Comments and Our Evaluation
paragraphs: We provided a draft of this report to CMS, and to CAP, COLA, and  JCAHO—the three laboratory accrediting organizations included in our  review. CMS strongly endorsed our overall conclusion that quality  assurance for the nation’s clinical labs should be strengthened and noted  that the report provided insights into areas where it can improve, augment,  and reinforce oversight of both labs and accrediting organizations to  ensure quality testing. Overall, CMS concurred with 11 of our 13  recommendations. Despite this endorsement, however, CMS (1) provided  an alternative assessment of lab quality, (2) disagreed that the phase-in of  certain CLIA requirements inappropriately stressed education as opposed  to regulation, (3) expressed concern about how to identify and sanction  labs with repeat condition level deficiencies, (4) disagreed with our  recommendation regarding the frequency of proficiency testing, and   (5) stated that it was already meeting our recommendation to conduct  almost all validation reviews of each accrediting organization  independently. We continue to believe that implementation of these  recommendations is necessary for the effective oversight of labs. (CMS’s  comments are reproduced in app. IV.) CAP indicated that it took seriously  our findings and recommendations and intended to determine if there  were additional measures it could take to strengthen its own oversight.  COLA said that our recommendations to improve CMS oversight of survey  organizations had merit. Nonetheless, CAP, COLA, and JCAHO disagreed  with some of our findings and recommendations to CMS. (CAP, COLA, and  JCAHO’s comments are reproduced in app. V, VI, and VII, respectively.)  Our evaluation first responds to CMS’s and related accrediting  organizations’ comments and then addresses additional comments by  accrediting organizations.
subsections: section_title: Assessment of Lab Quality
paragraphs: CMS and COLA commented that lab performance has improved since the  enactment of CLIA. In particular, CMS pointed to the substantial decline— from about 80 percent to about 42 percent—in the percentage of labs  nationwide with deficiencies between 1994 and 2004. It is important to  note that CMS’s data (1) do not distinguish between serious condition- level deficiencies and less serious standard-level deficiencies, (2) include  the early start-up period when physician office labs were first regulated,  and (3) exclude deficiency data on the substantial number of labs  surveyed by accrediting organizations and state CLIA-exempt programs.  Due to these shortcomings, we do not believe that CMS’s data provide an  accurate assessment of lab quality nationwide.
Based on the limited data available on state survey agency inspections of  labs since 1998 and the lack of any comparable data on accrediting  organization and exempt-state program survey findings, we concluded that  insufficient data existed to identify the extent of serious quality problems  at labs. CMS did not retain backup files of pre-2004 data on deficiencies  identified by state survey agencies. Although CMS has determined that  accrediting organization and exempt-state program lab requirements are at  least equivalent to CLIA’s, there is no agreement across survey  organizations on how to distinguish serious from less serious deficiencies.  While CMS concurred with our recommendation to standardize the  categorization and reporting of survey findings in a way that tracks to  CLIA and allows meaningful comparisons across survey organizations, it  noted that a straightforward linkage of requirements is limited by CMS’s  authority under the statute—that is, survey organizations are permitted by  statute to have different requirements—and that it will approach  implementation of our recommendation cautiously. JCAHO said that it  agreed with the need for a common, agreed upon, taxonomy that could be  used by all survey organizations to track serious deficiencies, but  commented that it thought CMS’s implementation of our recommendation  would require a revamping of JCAHO’s accreditation system. That was not  the intent of our recommendation and it is clear from CMS’s comments  that its implementation of our recommendation would not require an  overhaul of accrediting organizations’ systems. CAP acknowledged the  complexity and inherent challenges in measuring the quality of lab testing,  but noted that it is committed to working to develop better systems to  detect labs with serious quality problems—those that impact patient care.
The statutory authority that permits standards different from CMS’s  (provided they are at least as stringent) does not impair the ability to  develop a crosswalk that allows for meaningful comparisons across survey  organizations—such as an analysis of trends in the citation of condition- level deficiencies. In fact, CMS regulations already require accrediting  organizations and exempt-state programs to submit a crosswalk—detailed  comparisons of their individual accreditation or licensure approval  requirements with comparable CLIA condition-level requirements—when  they apply and reapply for approval from CMS. Such a comparison is  possible because CMS already identifies instances when accrediting  organizations have missed condition-level requirements during validation  reviews. For example, CMS should require survey organizations to   (1) indicate which of their requirements relate to each CLIA condition- level requirement, and (2) explain which deficiencies in their  requirements, if cited, should be considered equivalent to CLIA condition- level deficiencies.
CMS also pointed to the steady increase in successful proficiency testing  across all labs as an indication of improvements in lab quality. Our  analysis of proficiency testing results suggested that lab quality had not  improved at hospital labs in recent years. CMS correctly noted that the  overall proportion of labs with no test failures increased from about   88 percent in 1998 to about 93 percent in 2003—that is, fewer labs failed  proficiency testing. However, by focusing on overall proficiency testing  results, CMS data mask trends in failure rates for subsets of labs such as  hospital labs. For example, from 1999 through 2003, the percentage of  CAP-surveyed labs with proficiency testing failures increased from   4.1 percent to 6.8 percent; CAP generally inspects hospital labs. CMS also  commented that the overall improvement cannot be dismissed as a result  of some labs being granted waived status because the more dramatic  improvements predated the recent increase in the number of waived labs.  It further commented that removing waived labs from the data would not  result in improved performance rates. First, the number of waived labs— those performing waived tests or provider-performed microscopy— increased by about 26,600 from 1993 though 1998 and then increased by  another approximately 33,700 labs from 1998 through 2004. Second, CMS’s  comment suggested that it had conducted an analysis of the impact of  removing waived labs from the proficiency testing data. However, it did  not provide any data analysis when we subsequently asked to see the  evidence behind its assertion. COLA also addressed this issue, and did not  challenge our conclusion that the decrease in proficiency testing failures  for physician office labs might not represent an actual improvement in lab  quality, but instead could reflect the fact that some problematic labs are  no longer surveyed.
section_title: Educational Focus of CLIA
paragraphs: CMS agreed that it was important to maintain an appropriate balance  between its regulatory and educational approaches to CLIA  implementation. While CMS noted that objective review and feedback are  the bedrock of education, it emphasized that the educational approach  does not preclude surveyors from identifying lab deficiencies. CAP and  COLA offered similar comments. However, we found evidence that the  goal of educating lab workers sometimes takes precedence over, or  precludes, the identification and reporting of deficiencies and  recommended that CMS take steps to ensure that regulation remains the  primary goal of surveys. To address this problem, CMS stated that it will  provide additional state agency surveyor training, improve guidance,  develop an action plan to promote greater consistency among surveyors,  and institute periodic performance and consistency reviews. CMS’s  comments did not address evidence we presented that an educational  emphasis may also prevent fulfillment of regulatory responsibilities by  some accrediting organizations.
CMS disagreed that the extended phase-in periods for new quality control  requirements and proficiency testing for lab technicians who interpret Pap  smears were inappropriate. COLA noted that federal requirements in many  regulated industries are phased in to allow them time to understand and  effectively implement the requirements. CMS reaffirmed that, in the case  of significant new requirements and for the time period specified by CMS,  the educational approach may result in identified deficiencies being  communicated to laboratories without a concomitant citation, as is the  case with quality control and Pap smear testing requirements. As  discussed in the report, we believe that CMS’s educational phase-in  periods are excessive. We found that the phase-in period for new quality  control requirements was extended from 2 years to about 4 years, in part  because of the lack of lab “buy-in” for some of the new policies and  procedures. Similarly, the phase-in period for Pap smear proficiency  testing is 2 years, despite (1) CMS’s concern about some of the high initial  test failure rates, (2) the consequences of inaccurate test results on  patients’ diagnoses and treatment, and (3) the approximately 13-year time  lag between the 1992 implementation of the CLIA regulations and the  commencement of Pap smear proficiency testing.
section_title: Sanctioning Labs with Serious, Repeat Deficiencies
paragraphs: In commenting on our recommendation to appropriately sanction labs  with repeat condition-level deficiencies, CMS acknowledged the need to  carefully monitor repeat deficiencies but expressed concern that focusing  on the condition cited may not indicate a true repeat deficiency because  the underlying failures could have been different in the two consecutive  surveys for those labs. CMS’s assertion is inconsistent with its own policy  on serious, repeat deficiencies for other providers, such as nursing homes.  In general, immediate sanctions must be imposed on nursing homes with  consecutive serious deficiencies, regardless of whether the deficiencies  are in the same care area. As we have previously reported, allowing  providers to avoid sanctions by correcting serious deficiencies contributes  to an up-and-down pattern of compliance and undermines the deterrent  effect of sanctions. According to the CLIA legislative history,  congressional concern about labs with repeat deficiencies led to the  introduction of alternative sanctions such as civil money penalties as a  substitute for more severe principal sanctions, which include termination  from the CLIA program.
section_title: Proficiency Testing Frequency
paragraphs: CMS disagreed with our recommendation that it require quarterly  proficiency testing except when technical and scientific considerations  indicate that less frequent testing is justified for particular tests; CMS  insisted that proficiency testing three times a year was “appropriate.” CMS  stated that CMS and the Centers for Disease Control and Prevention had  together determined that the reduced frequency was based on technical  and scientific grounds. We asked for a record of the agencies’ deliberation  supporting that decision. CMS supplied a brief, undated narrative, which it  attributed to the Centers for Disease Control and Prevention. It was not  clear to us that this narrative was contemporaneous with the decision to  reduce the frequency of proficiency testing. Moreover, the narrative  focused on the relative costs and benefits of proficiency testing at various  intervals. There was no analysis of technical and scientific considerations  with regard to particular tests that presented a basis for reducing the  frequency.
Based on CMS’s response, we maintain that CMS’s decision to require  proficiency testing three times a year is not authorized by CLIA. CMS did  not dispute that, according to CLIA, it must base the decision to reduce the  frequency of proficiency testing on scientific and technical considerations  relevant to particular tests, and that the decision to reduce the frequency  is in the nature of an exception to the norm of quarterly testing. CMS also  acknowledged that the “public explanation” contained in the preamble to  the rule setting the proficiency testing requirement at three times a year  referred only to general concerns about the perceived burden associated  with quarterly testing. For example, CMS stated in its final rule that the  prospect of reduced frequency would provide a “needed respite” to both  laboratories and proficiency test providers. In sum, CMS has not adhered  to the conditions set out in the statute for reducing the frequency of  proficiency testing and has implemented a policy that is not supported by  statutory authority.
section_title: Equivalency of Accrediting Organization and Exempt State Programs
paragraphs: CMS acknowledged the need to complete timely equivalency reviews of  accrediting organization and exempt-state requirements, which were an  average of about 40 months late for the 3-1/2-year period we examined.  Regarding interim changes made between periodic equivalency reviews,  CMS agreed with our recommendation and stated it would review such  interim changes for both accrediting organizations and exempt-state  programs prior to implementation, as required by regulation.
Furthermore, CMS indicated that changes to accrediting organization  requirements did not necessarily impact CLIA equivalency determinations  because accrediting organizations may have more stringent requirements  than CLIA’s. While possibly true, CMS must review the changes to  determine whether CLIA equivalency is affected. For example, in 2005,  when it reviewed JCAHO’s revised standards a year after they were  implemented, CMS identified several critical areas where JCAHO’s  standards were less stringent than those of CLIA.
CMS acknowledged that a significant increase in workload and the decline  in CLIA program staff were factors which contributed to delays in making  equivalency determinations and reviewing interim changes. Although CMS  stated that it reserved the right to manage the work within available  resources and its assessment of priorities, it also made a commitment to  explore our recommendation to utilize revenues generated by the CLIA  program to hire sufficient staff to fulfill its responsibilities. We believe that  additional staff would not only improve the timeliness of equivalency  reviews, but also their thoroughness.
section_title: Accrediting Organization Validation Reviews
paragraphs: CMS stated that, consistent with our recommendation, 88 percent of  accrediting organization validation reviews were conducted independently  in calendar year 2005. However, our recommendation was to require that  almost all validation reviews of each accrediting organizations’ surveys be  conducted independently. CMS’s comments do not indicate the proportion  of independent validation reviews conducted for each accrediting  organization. Because CMS did not begin collecting data on the number of  simultaneous accrediting organization validation reviews performed until  August 2003, we relied on estimates from accrediting organizations. CMS  did not challenge JCAHO’s estimate that 33 percent of its validation  reviews were simultaneous, compared to about 10 percent for CAP and  COLA. We do not believe that performing an estimated 33 percent of  JCAHO’s validation reviews simultaneously is consistent with our  recommendation.
COLA commented that simultaneous validation reviews are useful in  assuring consistency and in providing an understanding of processes  across survey organizations. It also questioned the accuracy of most of the  missed condition-level deficiencies identified by CMS during independent  validation reviews. We did not assess the process CMS uses to identify  such missed deficiencies but based on a discussion with CMS officials it  appears that the process is thorough and time consuming. JCAHO  commented that we had misinterpreted the results of simultaneous and  independent validation reviews of accrediting organizations because, by  JCAHO’s estimate, the proportion of missed condition-level deficiencies is  roughly equivalent for both types of surveys. We did not find the  assumptions behind JCAHO’s estimate convincing, given the lack of data  on the actual number of simultaneous versus independent validation  reviews conducted for each accrediting organization. Furthermore, most  of the state survey agency officials we interviewed, whose inspectors  conduct accrediting organization validation reviews, told us that  simultaneous validation reviews do not provide a realistic evaluation of  the adequacy of accrediting organizations’ inspection processes.
section_title: Additional Comments by Accrediting Organizations
paragraphs: Additional CAP comments. CAP commented that we underestimated  the value of using lab professionals in the inspection process and that we  provided no factual evidence that their use was less effective than other  models. In contrast to CAP, other survey organizations employ dedicated  staff surveyors who have mandatory and continuing education  requirements. In addition, such dedicated surveyors conduct from 30 to  about 200 surveys per year compared to CAP’s lab professionals who  volunteer to perform about 1 survey per year. CAP partially addressed our  concern about the lack of mandatory training for its volunteer surveyors.  It plans to begin requiring training for survey team leaders in July 2006 and  for survey team members in 2007. However, CAP’s proposed new  mandatory training is much less extensive than that required by other  survey organizations.
Moreover, we reported that some CAP surveyors we interviewed raised a  concern about having survey team leaders who are also the day-to-day  supervisors of team members. For example, lack of agreement about the  seriousness of a deficiency could result in the team leader instructing the  team to downgrade the deficiency to a recommendation, a less serious  finding that does not appear in the inspection report. Team members who  are subordinates to the team leader may feel that they have no other  recourse than to follow the team leader’s instructions. CAP recently  revised its conflict-of-interest policy which now instructs all parties to be  cautious to retain objectivity in fact finding throughout the inspection  process. We do not believe that this change in CAP’s conflict-of-interest  policy addresses the concerns raised by the CAP surveyors we  interviewed. In its comments, CAP indicated that it would continue to  closely monitor this issue to determine if further actions were necessary.
Additional COLA comments. COLA disagreed with our assertion that  announced surveys may result in an unrealistic picture of lab quality—a  conclusion supported by CMS regional office staff and most state survey  agency officials we interviewed. We acknowledged that unannounced  surveys of the physician office labs typically surveyed by COLA and state  survey agencies were not practical given the unpredictable operating  hours of such labs and need to minimize disruptions to patient care.  However, we recommended that the advanced notice be limited to the   2 weeks permitted by CMS for state survey agencies. COLA currently  provides up to 12 weeks advanced notice. COLA contends that providing  up to 6 months of advanced notice before a survey would only improve the  lab’s operation more quickly if the lab took that opportunity to review  COLA’s self assessment questions and correct any missing or incorrect  processes or documentation. We believe that COLA’s example  underscores the importance of our recommendation; such actions should  be an ongoing process at labs—not a reaction to an upcoming inspections.
Additional JCAHO comments. JCAHO said that our recommendation  that all survey organizations develop and require labs to prominently  display posters that instruct lab workers on how to file anonymous  complaints was too narrow and prescriptive and may inadvertently limit  organizations from using other, more effective ways to educate lab  workers on this topic. JCAHO did not explain how implementing our  recommendation would limit other initiatives. In fact, CMS’s comments  identified a number of promising approaches that it believed could  supplement posters. JCAHO also said that our analysis of the increase in  CAP complaints after it required posters in the labs it inspects failed to  recognize a broad national trend. JCAHO indicated that it also experienced  a dramatic increase in lab complaints between 2004 and 2005 without the  use of posters. This increase may be related to JCAHO’s July 2005  requirement for labs to educate staff on how to report concerns. CAP told  us that during the 3 months after they required a poster to be displayed  they observed an immediate increase in the number of complaints. Thus,  CAP lab complaints increased by over 100 percent in 2004 compared to  2003 and by another approximately 71 percent in 2005. We continue to  believe that CAP’s experience suggests that complaint posters can be an  important way to encourage lab workers to communicate their concerns.
CMS, CAP, COLA, and JCAHO also provided technical comments which  we incorporated as appropriate.
As arranged with your offices, unless you publicly announce its contents  earlier, we plan no further distribution of this report until 30 days after its  issue date. At that time, we will send copies to the Administrator of the  Centers for Medicare & Medicaid Services and appropriate congressional  committees. We will also make copies available to others upon request. In  addition, the report will be available at no charge on the GAO Web site at  http://www.gao.gov.
If you or your staff have any question s about this report, please contact  me at (312) 220-7600 or aronovitzl@gao.gov. Contact points for our Offices  of Congressional Relations and Public Affairs may be found on the last  page of this report. GAO staff who made major contributions to this report  are listed in appendix VIII.
section_title: Appendix I: Effects of Lab Errors on Patient Health
paragraphs: This appendix contains examples of lab errors and their consequences,  illustrating the importance of the quality of lab testing and the effects of  lab errors on patient health. The examples in table 7 are summarized from  case studies in the journal Laboratory Errors and Patient Safety.
section_title: Appendix II: Labs Surveyed by State Survey Agencies and the Percentage with Condition- Level Deficiencies, by State in 2004
section_title: Appendix III: Number of Labs Subject to Surveys by State Survey Agencies in 2005 and Number of Labs with Sanctions, 1998 to 2004
paragraphs: Number of labs (2005)
Number of labs (2005)
section_title: Appendix IV: Comments from the Centers for Medicare & Medicaid Services
section_title: Appendix V: Comments from the College of American Pathologists
section_title: Appendix VI: Comments from COLA
section_title: Appendix VII: Comments from the Joint Commission on Accreditation of Healthcare Organizations
section_title: Appendix VIII: GAO Contact and Staff Acknowledgments
subsections: section_title: Acknowledgments
paragraphs: In addition to the contact named above, Walter Ochinko, Assistant  Director; Lucia P. Fort; Dan Lee; Kevin Milne; Dean Mohs; Elizabeth T.  Morrison; Michelle Rosenberg; and Elizabeth Scherer made key  contributions to this report.