Nursing Homes: More Reliable Data and Consistent Guidance Would Improve CMS Oversight of State Complaint Investigations
Published: 2011-04-07T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: CMS, the agency within HHS that manages Medicare and Medicaid, contracts with state survey agencies to investigate complaints about nursing homes from residents, family members, and others. CMS helps assure the adequacy of state complaint processes by issuing guidance, monitoring data that state survey agencies enter into CMS's database, and annually assessing performance against specific standards. Concerns have been raised about the timeliness and adequacy of complaint investigations and CMS's oversight. GAO examined (1) complaints received, investigated, and substantiated by state survey agencies; (2) whether those agencies were meeting CMS performance standards and other requirements; and (3) the effectiveness of CMS's oversight. In addition to analyzing CMS data on complaints and performance reviews, GAO examined CMS guidance and conducted interviews with officials from three high- and three low-performing state survey agencies and their CMS regional offices. GAO addressed data reliability concerns by reporting only data we determined to be reliable.
section_title: What GAO Found
paragraphs: CMS's complaints data showed that state survey agencies received 53,313 complaints about nursing homes in 2009. The number and types of complaints varied among states. For example, 11 states received 15 or fewer complaints per 1,000 nursing home residents while 14 states received more than 45. State survey agencies assess the severity of a complaint and assign a priority level, which dictates if and when an investigation must be initiated. About 10 percent of complaints were prioritized as immediate jeopardy, requiring investigation within 2 working days of receipt, while 45 percent were prioritized as actual harm-high, requiring investigation within 10 working days of prioritization. State survey agencies investigated all but 102 complaints that required an investigation. Among investigated complaints, 19 percent were substantiated and resulted in the citation of at least one federal deficiency. The percentage of immediate jeopardy and actual harm-high complaints that were substantiated with at least one federal deficiency cited was higher if the investigation was initiated on time. In CMS's performance assessment for fiscal year 2009, many state survey agencies had difficulty meeting some of CMS's nursing home complaint standards, most of which also assess performance with regard to incidents--specific care issues that nursing homes are required to report. In particular, 19 state survey agencies had difficulty investigating actual harm-high complaints and incidents within the required time frame. However, most states were able to meet other CMS standards--timely investigation of immediate jeopardy complaints and incidents and appropriate prioritization of complaints and incidents. Although CMS's performance assessment does not review state survey agencies' communication with complainants, CMS does expect the agencies to convey investigation findings according to CMS guidelines. GAO found state survey agencies had varied interpretations of those guidelines, and some provided limited information to complainants. CMS's oversight of state survey agencies' complaint investigation processes, through its performance standards system and complaints database, is hampered by data reliability issues. While CMS's performance standards are consistent with certain key criteria for performance measures identified by GAO and other audit agencies, performance scores are not always reliable, due in part to inadequate sample sizes and inconsistent interpretation of some standards by CMS reviewers. In addition, CMS has not made full use of the information it collects. For example, in part because of data reliability concerns, CMS does not routinely use data from the complaints database to calculate certain measures that could enhance its understanding of agencies' performance. Although CMS requires state survey agencies that fail performance standards to develop corrective action plans, states' plans do not necessarily address the underlying causes of performance issues, such as staffing shortages.
section_title: What GAO Recommends
paragraphs: GAO recommends that the CMS Administrator take several steps to strengthen oversight of complaint investigations, such as improving the reliability of its complaints database and clarifying guidance for its state performance standards to assure more consistent interpretation. HHS generally agreed with our recommendations.

section_title: Letter
paragraphs: The 1.4 million elderly and disabled residents living in nursing homes are  considered a highly vulnerable population. They frequently depend on  others for assistance with basic activities of daily living such as dressing,  eating, or toileting, and some require skilled nursing or rehabilitative care.  The vast majority of nursing homes that care for these residents  participate in Medicare and Medicaid, and in 2009, nursing homes received  about $89 billion in payments from these programs. Ensuring quality of  care in these nursing homes is a joint responsibility of the Centers for  Medicare & Medicaid Services (CMS), within the U.S. Department of  Health and Human Services (HHS), and state survey agencies. Congress  and CMS set federal requirements, and CMS contracts with state survey  agencies to perform both routine inspections of nursing homes, known as  standard surveys, and complaint investigations, among other activities.
Complaint investigations offer a unique opportunity to identify and correct  potential care problems. They can provide more timely alerts of potential  problems than standard surveys and target specific areas identified by  residents, their families, nursing home staff, and others. In 2009, half of all  violations of federal requirements that resulted in some level of harm to  nursing home residents were cited during complaint investigations. State  survey agencies generally develop their own investigation procedures but  must follow certain federal procedures and time frames for complaints  that allege a violation of federal requirements. State survey agencies also  must provide certain information about their complaint investigations to  CMS through its national complaints database. CMS oversees state survey  agencies in part by assessing their performance on four standards that  pertain to nursing home complaints. These standards are part of a broader  CMS State Performance Standards System.
Members of Congress and others have raised concerns about the  timeliness and adequacy of nursing home complaint investigations, as well  as the manner in which the findings are communicated to complainants.  Concerns have focused not only on state survey agencies’ nursing home  complaint investigations, but also on CMS’s oversight. You expressed  interest in learning more about these issues. Specifically, we examined   (1) the number and types of complaints CMS’s database showed as  received, investigated, and substantiated by state survey agencies;   (2) whether state survey agencies were meeting CMS’s performance  standards and complainant communication requirements and steps taken  by the agencies to meet them; and (3) the effectiveness of CMS’s oversight  of state survey agencies’ complaint investigation processes.
To describe the number and types of nursing home complaints received,  investigated, and substantiated by state survey agencies, we analyzed  CMS’s national complaints data for calendar years 2004 through 2009 for  all 50 states and the District of Columbia. Because concerns have focused  primarily on complaints, we included only complaints in our analysis for  this objective and excluded facility-reported incidents, which nursing  homes are required to self-report to state survey agencies. In addition, we  included only complaints that alleged a violation of federal requirements.  To assess the reliability of the complaints data we received from CMS, we  interviewed officials from CMS and state survey agencies about the quality  of the data, reviewed relevant documentation, and examined the data for  reasonableness and internal consistency. In the course of this assessment,  we found some data limitations. Specifically, CMS officials told us that  they have concerns that some state survey agencies may not have entered  all of the complaints they received into CMS’s national database. We  therefore consider the number of complaints in CMS’s national data to be  a conservative estimate of the total number of complaints received by  state survey agencies. In addition, we analyzed only those variables that  we found to be reliable. We learned that in some cases, data are missing  for certain variables that state survey agencies are not required to enter  into the database—such as the date on which the state survey agency  acknowledged the complaint—and that state survey agencies interpret  certain variables differently from one another. For example, state survey  agencies have differing interpretations of what it means to substantiate a  complaint. Some state survey agencies limit use of the term to complaints  where at least one deficiency is cited while others consider complaints to  be substantiated if they are confirmed, even if no deficiencies are cited. In  this report, we chose to report data about complaints that were  substantiated with at least one federal deficiency cited, as we believe these  data should be more consistent across states than data on all complaints  reported to be substantiated. In addition, the citation of a federal  deficiency demonstrates that the nursing home has failed to meet federal  requirements. After reviewing the possible limitations of the complaints  data, we determined that the data we report were sufficiently reliable for  the purposes of our report.
To determine whether state survey agencies are meeting CMS’s standards  and complainant communication requirements and to describe steps they  have taken to meet the requirements, we analyzed scores for two of the  four nursing home complaint performance standards in CMS’s State  Performance Standards System for fiscal years 2006 through 2009 for all   50 states and the District of Columbia. For our analysis, we reviewed  performance on the two standards we considered the most reliable:   (1) prioritization of complaints based on the severity of the allegations and  (2) timeliness of investigations. Although these standards assess state  survey agencies’ performance with respect to incidents as well as  complaints, we used scores on these standards as measures of  performance with respect to complaints alone. CMS does not calculate  separate scores for complaints. Moreover, on a national level, complaints  considerably outnumber incidents in CMS’s database, and state survey  agencies’ scores on the standards are therefore likely to primarily reflect  their performance with respect to complaints. We analyzed scores from  fiscal years 2006 through 2009 because CMS reorganized its performance  system in 2006, and the most recent data available at the time of our study  were from fiscal year 2009. Because of changes made in the standards’  requirements and scoring during this time period, we have presented trend  data only when scores were comparable over time. We also analyzed  CMS’s national complaints data on the length of time taken by state survey  agencies to investigate complaints in calendar year 2009. In addition, we  reviewed the guidance CMS provided to state survey agencies and its own  regional offices, which are responsible for evaluating state survey  agencies’ nursing home complaint processes. We also conducted  structured telephone interviews with CMS regional office officials in  Atlanta, Chicago, and Dallas and state survey agency officials in Arkansas,  Florida, Michigan, Tennessee, Texas, and Wisconsin. We gathered  additional perspectives on CMS’s requirements at a membership meeting  of the Association of Health Facility Survey Agencies (AHFSA), the  organization that represents state survey agencies. Finally, we reviewed  both templates and samples of actual letters to complainants provided by  the six state survey agencies in our sample.
To assess the effectiveness of CMS’s oversight of state survey agencies’  complaint investigation processes, we drew on information from our data  analyses and interviews, including interviews with officials at CMS  headquarters. We also evaluated the four nursing home complaint  performance standards using key criteria for performance measures  identified by GAO and other audit agencies. These criteria include  whether the standards are comprehensive, limited in number and overlap,  practical, balanced, comparable over time, and reliable. Additionally, to  examine the extent to which CMS has used performance information to  promote improvements in state survey agencies’ nursing home complaint  investigation processes, we reviewed information from our interviews and  data analyses. We also reviewed corrective action plans that state survey  agencies in our sample were required to submit for any performance  standards they failed between fiscal years 2006 through 2009.
We conducted our review from January 2010 through April 2011 in  accordance with generally accepted government auditing standards. Those  standards require that we plan and perform the audit to obtain sufficient,  appropriate evidence to provide a reasonable basis for our findings and  conclusions based on our audit objectives. We believe that the evidence  obtained provides a reasonable basis for our findings and conclusions  based on our audit objectives.
subsections: section_title: Background
paragraphs: When investigating complaints about nursing homes, state survey agencies  follow state policies and procedures based on CMS instructions. To  oversee state survey agencies’ complaint investigation processes, CMS  uses data from its complaints database and State Performance Standards  System.
subsections: section_title: Complaint Investigation Policies and Procedures
paragraphs: CMS’s State Operations Manual outlines procedures for state survey  agencies’ investigation of nursing home complaints. This manual is based  on requirements in statutes and regulations and includes a detailed  protocol for handling complaints and incidents, such as directions for key  parts of the complaints process—intake, prioritization, investigation, and  reporting of results.
Intake. State survey agencies receive complaints via phone calls, e-mails,  or letters. At intake, staff review the information provided by the  complainant and, because each complaint can have more than one  allegation, determine the type(s) of allegations involved, such as resident  abuse or poor quality of care.
Prioritization. Based on the nature of the allegations, staff assign a  priority level to the complaint, which determines if an onsite investigation  is required. Four of the eight priority levels require an onsite investigation.  (See table 1.) For example, investigations for complaints that allege  “immediate jeopardy” to a resident’s health, safety, or life must be started  within 2 working days of receipt, while investigations for complaints that  allege a high level of actual harm (“actual harm-high”) to a resident must  be started within 10 working days of prioritization.
Investigation. During the unannounced investigation, state agency  surveyors may conduct a document review and observe nursing home  conditions. Additionally, surveyors interview witnesses, including the  resident about whose care the complaint was filed and other residents  with similar care needs, being careful to protect the anonymity of those  involved in the complaint. Surveyors determine whether the allegations  are substantiated and whether the nursing home should be cited for any  deficiencies (failure to meet federal or state quality standards), which may  be related or unrelated to the complaint allegations. Deficiencies are  categorized according to scope and severity. Scope refers to the number of  residents potentially or actually affected and has three levels—isolated,  pattern, or widespread. Severity refers to the degree of relative harm and  has four levels—immediate jeopardy (actual or potential for death or  serious injury), actual harm, potential for more than minimal harm, or  potential for minimal harm.
Reporting of Results. After the complaint investigation is completed, the  state survey agency notifies the complainant and the nursing home of the  outcome of the investigation, following guidelines specified in the State  Operations Manual.
section_title: CMS Oversight of State Survey Agencies’ Complaint Investigation Processes
paragraphs: CMS oversees state survey agencies’ complaint investigation processes  using its complaints data and State Performance Standards System.
CMS’s Complaints Data. As of January 1, 2004, state survey agencies  were required to enter data about all complaints and incidents into the  ACTS—Automated Survey Processing Environment (ASPEN)  Complaints/Incidents Tracking System—database according to guidance  provided by CMS. Officials in CMS’s headquarters and regional offices  can access all information in ACTS, though the information is stored on  individual state servers. CMS provides guidance to state survey agencies  regarding ACTS database procedures, including what complaint  information states are required to enter. The information is then uploaded  into CMS’s national complaints database, which contains a variety of  information about complaints, such as the date of the alleged event, the  name of the nursing home involved, and the source of the complaint. (See  table 2.)
State Performance Standards System. CMS’s 10 regional offices are  responsible for annually evaluating state survey agencies’ nursing home  complaint investigations using four performance standards. (See table 3.)  CMS developed the State Performance Standards System in fiscal year  2001 to assess whether state survey agencies were meeting the  requirements for the survey and certification program and to identify areas  for improvement. In fiscal year 2006, CMS reorganized the performance  standards system, and in the following years made several revisions to the  four nursing home complaint performance standards. None of the  standards focus exclusively on nursing home complaints. For some  standards, the scope of review includes incidents as well as complaints,  facilities other than nursing homes, or standard surveys as well as  complaint investigations. For all except the timeliness standard, the  review is based on samples rather than the universe of complaints and  incidents.
Upon completion of the performance evaluation, CMS regional offices  share the results with each respective state survey agency and CMS  headquarters, which in turns shares each state’s scores with all of the  other states. State survey agencies that fail performance standards must  submit corrective action plans to their CMS regional offices, which the  regional offices can accept or reject, depending on whether they believe  the state has outlined appropriate steps to address poor performance. The  regional offices use these plans to follow up with state survey agencies as  part of their monitoring activities.
section_title: CMS 2009 Data Show that States Received Over 50,000 Nursing Home Complaints and Substantiated the Complaint and Cited Federal Deficiencies in 19 Percent of Investigations
paragraphs: CMS’s national complaints data show that state survey agencies received  over 50,000 complaints about nursing homes in calendar year 2009. The  number and types of complaints varied among states. State survey  agencies investigated all but 102 of the complaints that required an  investigation. Among complaints that were investigated and uploaded to  CMS’s national database for 2009, 19 percent were substantiated with at  least one federal deficiency cited.
subsections: section_title: According to CMS’s National Data, State Survey Agencies Received 53,313 Nursing Home Complaints in 2009
paragraphs: State survey agencies reported receiving 53,313 complaints about nursing  homes in 2009. In 2009, 9 states received fewer than 100 complaints while  17 states received more than 1,000. Six states—Illinois, Missouri, New  York, Ohio, Texas, and Washington—accounted for roughly half of all 2009  complaints in CMS’s database. Although the number of nursing home  residents has remained relatively stable, the number of complaints  received generally increased by about 1,000 complaints a year from 2004 to  2008. In 2009, the number of complaints dropped by about 5,000.
Complaint Rate. Nationally, in 2009, CMS’s database showed a complaint  rate of roughly 38 complaints per 1,000 nursing home residents. The  complaint rate ranged from less than 1 (0.77) in South Dakota to about 137  in Washington. Additionally, 11 states received 15 or fewer complaints per  1,000 nursing home residents, while 14 states received more than 45. (See  fig. 1.)
Submission of Complaints and Sources. CMS data show that state  survey agencies received three-quarters of complaints in 2009 by phone.  Complaints also were submitted through other means, such as in writing,  through e-mail, or in person. In 2009, complaints were typically submitted  by family members (47 percent), anonymously (19 percent), or by  residents (10 percent). Complaints were also submitted by current nursing  home staff or other sources.
Prioritization of Complaints. In 2009, among the complaints in CMS’s  national data, state survey agencies prioritized most as either actual harm- high (45 percent) or actual harm-medium (33 percent). Roughly 10 percent  of complaints were prioritized as immediate jeopardy and about 4 percent  were prioritized as actual harm-low. Approximately 8 percent of  complaints were prioritized at the four lowest levels and did not require an  onsite investigation. State survey agencies varied in the percentage of  complaints they prioritized at different levels. For example, 23 state survey  agencies prioritized more than 50 percent of complaints as immediate  jeopardy or actual harm-high, while 7 state survey agencies prioritized  fewer than 10 percent of complaints they received at these two levels.
Allegations. Allegations are specific charges within complaints; each  complaint can have multiple allegations. In 2009, according to CMS’s  national data, the average number of allegations per complaint was 2.3.  Allegations that focused on quality of care or treatment accounted for  about 40 percent of all allegations in 2009. (See table 4.)
section_title: CMS National Data Show States Investigated Nearly All Complaints that Required an Investigation and Cited Deficiencies in 19 Percent of the Investigations
paragraphs: CMS data show that in 2009 about 48,900 of the approximately 53,300  complaints received required an investigation and that state survey  agencies investigated all but 102 of those complaints. Among those   102 complaints, 25 percent were prioritized as either immediate jeopardy  or actual harm-high (6 and 19 percent respectively). The remaining   75 percent were complaints prioritized as actual harm-medium or actual  harm-low. The percentage of complaints investigated from 2004 through  2009 remained relatively stable even as the number of complaints  increased in all years except 2009.
In 2009, an investigation was initiated within CMS’s required time frames  for most complaints prioritized as either immediate jeopardy or actual  harm-high. Among immediate jeopardy complaints, an investigation was  initiated within 2 working days of receiving the complaint for 88 percent of  complaints. Among complaints prioritized as actual harm-high, an  investigation was initiated within 10 working days of prioritization for   72 percent of complaints.
Roughly 19 percent of the complaints that were investigated and uploaded  into CMS’s complaints database for 2009 were substantiated with at least  one deficiency cited. However, there was considerable variation across  states. In 19 states, more than 30 percent of the complaints investigated  were substantiated with at least one deficiency cited, while in 5 states, the  proportion was less than 10 percent. Of the approximately 16,000 nursing  homes nationwide, about 2,800 had one substantiated complaint where at  least one deficiency was cited. In addition, about 1,100 nursing homes had  two such complaints.
The percentage of immediate jeopardy complaints that were substantiated  with at least one deficiency cited was higher than for complaints  prioritized at lower levels in 2009. According to CMS’s complaints  database, roughly 26 percent of the immediate jeopardy complaints that  were investigated were substantiated with at least one deficiency cited.  Among complaints prioritized at lower levels, the percentage was around  21 percent for actual harm-high complaints, 17 percent for actual harm- medium complaints, and 12 percent for actual harm-low complaints.
In 2009, among the complaints prioritized as immediate jeopardy or actual  harm-high, the percentage substantiated with at least one deficiency was  higher if the investigation was initiated within required time frames than if  it was not. For example, among actual harm-high complaints that were  investigated within 10 working days of prioritization, 22 percent were  substantiated with at least one federal deficiency cited. (See table 5.) In  contrast, among actual harm-high complaints that were investigated late,  the proportion was 17 percent. (App. I contains state-level data on  complaints received, investigated, and substantiated by state survey  agencies, according to CMS data.)
section_title: Many State Survey Agencies Had Difficulty Meeting Certain Performance Standards for Nursing Home Complaint Investigations, but Reported Taking Steps Intended to Improve Performance
paragraphs: Many state survey agencies did not meet some of CMS’s performance  standards for nursing home complaints in fiscal year 2009. In particular,  19 state survey agencies had difficulty investigating complaints and  incidents prioritized as actual harm-high within the required time frame.  State survey agencies reported that they have taken or plan to take steps  in four key areas—staffing, agency restructuring, training and guidance,  and monitoring—to meet CMS’s nursing home complaint standards.  Although the standards do not assess state survey agencies’  communication with complainants, CMS does expect agencies to convey  investigation findings to complainants in accordance with CMS’s State  Operations Manual. We found that agencies varied in their interpretations  of the manual’s instructions, and some provided limited information to  complainants.
subsections: section_title: Many State Survey Agencies Had Difficulty Meeting Certain Nursing Home Complaint Standards, Particularly for Timely Investigation of Actual Harm-High Complaints
paragraphs: More than half of state survey agencies had difficulty meeting certain CMS  performance standards pertaining to nursing home complaints.  According to CMS’s assessment for fiscal year 2009, 28 state survey  agencies failed the timeliness of investigations standard for either  immediate jeopardy or actual harm-high complaints, the prioritization of  complaints standard, or both.
Timeliness of Investigations Standard. CMS’s assessment of state  survey agencies’ performance found that some had difficulty meeting the  timeliness of investigations standard, which evaluates: (1) whether an  investigation was initiated within 10 working days of prioritization for  actual harm-high complaints and incidents for nursing homes, and   (2) whether an investigation was initiated within 2 working days of receipt  for immediate jeopardy complaints and incidents for nursing homes and  other facilities.
State survey agencies must begin investigating at least 95 percent of  complaints and incidents within required time frames.
For actual harm-high complaints and incidents, CMS evaluates  performance for nursing homes separately from that of other  facilities.
For immediate jeopardy complaints and incidents, CMS evaluates  performance for both nursing homes and other types of facilities.
CMS found that in fiscal year 2009, 19 state survey agencies failed to meet  the timeliness of investigations standard for complaints and incidents  prioritized as actual harm-high. This marked an improvement from fiscal  year 2008, when 25 states failed. States’ fiscal year 2009 scores varied  widely. For example, among states failing this standard, Louisiana nearly  passed with 94.4 percent of actual harm-high complaints and incidents  investigated within the required time frame, while Michigan’s score was  17.3 percent. (For information on all state survey agencies’ performance  on this standard, see app. II.) According to CMS’s national data for  calendar year 2009, the 19 states that failed this standard in fiscal year  2009 accounted for more than half (52 percent) of all actual harm-high  complaints received nationally. In these 19 states, at least 43 percent of  actual harm-high complaint investigations were initiated late, and at least  33 percent were initiated more than 11 working days late.
Officials from the three state survey agencies in our sample that failed to  meet the timeliness standard for actual harm-high complaints cited long- standing workload and staffing issues as reasons. More specifically,  officials with the Michigan and Texas survey agencies said they had  difficulty because of staffing shortages and because the volume of  complaints and incidents increased. Tennessee officials noted that the  state has tried to hire the additional staff needed to investigate the state’s  backlog of complaints, but has been hampered by low salaries for  surveyor positions as well as a cumbersome state hiring process.
Nationwide, state survey agencies generally performed better on CMS’s  timeliness standard for immediate jeopardy complaints and incidents than  they did for actual harm-high complaints and incidents. In CMS’s  assessment for fiscal year 2009, all but nine state survey agencies passed  this standard by initiating investigations within 2 working days of receipt  for at least 95 percent of the immediate jeopardy complaints and incidents  they received about nursing homes and other facilities. Among the nine  state survey agencies that failed this standard, four had scores at or below  50 percent. As with actual harm-high complaints and incidents, the two  state survey agencies in our sample that failed the timeliness standard for  immediate jeopardy complaints and incidents—Michigan and Tennessee— cited staffing shortages or increases in the number of complaints and  incidents as key reasons.
Fourteen state survey agencies that met CMS’s timeliness standard for  immediate jeopardy complaints and incidents did not meet the timeliness  standard for actual harm-high complaints and incidents. An official in one  CMS regional office noted that immediate jeopardy complaints are the  highest priority and therefore rightly received the most attention.
Prioritization of Complaints Standard. CMS’s assessment of state  survey agencies’ performance found that most agencies (32) consistently  passed this standard for the past four years.
State survey agencies must appropriately prioritize at least 90 percent  of complaints and incidents. CMS evaluates performance for nursing  homes separately from that of other facilities.
In CMS’s assessment for fiscal year 2009, all but nine state survey agencies  passed this performance standard. Among the nine state survey agencies  that failed this standard in fiscal year 2009, most had scores between   70 percent and 88 percent. (See app. II for information on all state survey  agencies’ performance on this standard.)
All but one of the six state survey agencies in our sample passed the  prioritization standard in fiscal year 2009. Officials from Tennessee said  that the agency had difficulty meeting this standard because of personnel  changes and because it took time for new management to fully understand  how the agency operates. Officials from the five state survey agencies in  our sample that passed this standard generally attributed their agencies’  performance on the prioritization standard to staff skills and experience,  training, and processes for quality control. For example, officials from two  state survey agencies—Arkansas and Texas—attributed their states’  success, in part, to a supervisor’s or quality assurance specialist’s review  of the priority levels assigned by the staff members who received the  complaint.
section_title: State Survey Agencies Reported Taking Steps Intended to Improve or Maintain Performance on CMS’s Standards
paragraphs: State survey agencies reported that they have taken or plan to take steps  in four key areas—staffing, agency restructuring, training and guidance,  and monitoring—to either improve or maintain performance on CMS’s  nursing home complaint standards.
Staffing. Officials from three of the state survey agencies in our sample  indicated that because staff shortages affected their ability to meet CMS  standards, they had taken steps to increase staffing. For example, officials  of the Michigan survey agency, which repeatedly failed the timeliness of  investigations standard between 2006 and 2009, reported that beginning in  fiscal year 2009, the agency was able to hire additional surveyors and as of  June 1, 2010, had eliminated its backlog of complaints. Tennessee officials  indicated that the agency received state legislature approval in February  2009 to hire additional surveyors to fill vacant positions. Texas officials  also hired additional surveyors to conduct complaint investigations.
Officials of state survey agencies in our sample that met all or most of  CMS’s nursing home complaint standards credited, among other factors,  experienced agency staff. For example, Wisconsin officials indicated that  the agency’s ability to meet CMS’s standards was partly due to the quality  of the staff hired by the agency—specifically, some staff members’  experience in the regulatory process, as both health care providers and  regulators.
Agency Restructuring. Some state survey agencies restructured  complaint investigation operations to address performance issues, either  consolidating regional offices or creating separate units to investigate  complaints. For example, to provide better statewide coverage with  available staff, the Tennessee survey agency downsized from three  regional offices to two. Arkansas and Texas both established separate  complaint investigation units—in Arkansas’s case, more than 10 years  ago—in an effort to better manage large volumes of complaints.
Officials of state survey agencies that have separate complaint  investigation units cited several advantages to dividing complaint  investigation functions from standard survey functions, including greater  efficiency and flexibility. For example, some officials said that staff  assigned to the complaints unit are able to build experience and familiarity  with the process and thus conduct more efficient investigations and  prepare more accurate reports; likewise, staff that focus on standard  surveys are able to conduct these inspections more efficiently because  they do not have to investigate complaints at the same time. One official  also said that a separate complaint investigation unit affords managers  more flexibility—for example, by allowing them to more easily change  staff members’ assignments from day to day to respond to high priority  complaints.
Training and Guidance. Officials of some state survey agencies  attributed their agencies’ successful performance on the prioritization of  complaints standard partly to staff training. State survey agencies also  issued guidance, including policy manuals and standardized forms or  templates, to guide staff through the complaint investigation process. For  example, Florida provides staff with a 44-page manual, with chapters on  intake, prioritization, and investigation of complaints, and created an  automated complaint investigation form that captures information about  each allegation in a complaint, as well as the evidence collected and  findings reached with respect to each.
Monitoring. Among the state survey agencies in our sample that failed to  meet some of CMS’s standards, officials indicated that their agencies had  implemented or planned to implement additional monitoring efforts. For  example, Texas officials indicated that the agency conducts reviews  throughout the complaint process. For example, after a complaint has  been prioritized, a quality assurance specialist reviews the information to  ensure that the prioritization was appropriate. Similarly, officials from  Tennessee’s survey agency indicated that the agency planned to increase  monitoring. In particular, the officials indicated that each of the state’s  regional offices would track and report quarterly on the timeliness of  investigations for all immediate jeopardy and actual harm-high complaints.  Tennessee officials indicated that surveyors in the state’s regional offices  would be immediately alerted when they are assigned an immediate  jeopardy complaint to investigate, something not always done in the past.
State survey agencies in our sample that generally passed CMS’s  performance standards indicated that monitoring programs contributed to  the agencies’ success. For example, a Florida official indicated that a  supervisor reviews a sample of complaints received on the previous day to  determine whether they were prioritized appropriately.
section_title: Some State Survey Agencies Provide Limited Information to Complainants about Investigation Findings
paragraphs: Although the CMS performance standards do not assess whether state  survey agencies are providing sufficient information to complainants  about investigation results, CMS’s State Operations Manual indicates that  state survey agencies should provide a written report to complainants in  accordance with certain guidelines specified in the manual. The manual  specifies that the state agency should acknowledge the complainant’s  concerns, identify the agency’s regulatory authority to investigate, provide  a summary of investigation methods and the date of the investigation,  summarize the investigation findings, and identify any follow-up action to  be taken.
The six state survey agencies in our sample varied in their interpretations  of the manual, particularly the instruction to provide a summary of the  investigation findings. Two of the six agencies consistently provided  detailed information that specifically addressed complainants’ allegations.  For example, one sample letter we received from the Wisconsin survey  agency lists four specific allegations made by the complainant and then  describes the agency’s finding with respect to each, including whether a  deficiency was cited. (See fig. 2 for an excerpt from this letter.) The other  state survey agency that provided detailed information (Michigan) did so  by enclosing the investigation report with the letter, along with the  statement of deficiencies, if any were cited. A Michigan survey agency  official said that staff also make at least one attempt to contact a  complainant by telephone to explain the findings.
In contrast, four of the state survey agencies sent complainants only  boilerplate descriptions of the complaint investigation, typically sending  one type of form letter if surveyors cited deficiencies and another if they  did not. For example, in the sample letter we received from Florida, the  survey agency varied the middle paragraph of its three-paragraph letter  depending on whether deficiencies were cited (see fig. 3). An official of  this agency said the letter was intended to let complainants know that the  point of an investigation is to determine a nursing home’s compliance with  regulations.
Of the four state survey agencies that provided boilerplate descriptions of  their investigation findings, two told complainants how to obtain a more  detailed report. For example, a sample letter from the Arkansas state  survey agency noted that the agency’s report on the deficiencies cited and  the nursing home’s plan of correction should be posted in the nursing  home. An Arkansas survey agency official said that complainants could  also request a copy of the investigation report, but that it might be heavily  redacted to protect medical and identifying information.
section_title: CMS’s Oversight of State Survey Agencies’ Complaint Investigation Processes Is Hampered by Data Reliability Issues, Due in Part to Inconsistent Interpretation of Performance Standards Among CMS Reviewers
paragraphs: CMS’s oversight of state survey agencies’ complaint investigation  processes, through its performance standards system and complaints  database, is hampered by data reliability issues. While the four  performance standards CMS uses to assess state survey agencies’  processes for investigating nursing home complaints are consistent with  certain key criteria for performance measures identified by GAO and other  audit agencies, the standards have weaknesses in areas related to other  key criteria, particularly data reliability, due in part to inadequate sample  sizes and inconsistent interpretation of some standards by CMS reviewers.  In addition, CMS has not made full use of the information it collects about  state survey agencies’ complaint investigation processes. For example, in  part because of data reliability concerns, CMS does not routinely use data  from the complaints database to calculate certain measures that could  enhance its understanding of state survey agencies’ performance.  Although CMS requires state survey agencies that fail performance  standards to develop corrective action plans, these plans do not  necessarily address the underlying causes of performance issues, such as  staffing shortages.
subsections: section_title: CMS’s Performance Standards Are Comprehensive and Limited in Number and Overlap, but Performance Scores Are Not Always Reliable
paragraphs: CMS’s four nursing home complaint performance standards—  (1) prioritization of complaints, (2) timeliness of investigations, (3) quality  of investigations, and (4) documentation of deficiencies—are consistent  with some, but not all, of the key criteria for performance measures  identified by GAO and other audit agencies. Specific weaknesses we  identified include a lack of comparability over time in the performance  scores and thus an inability to assess trends; a lack of balance among  some standards; and, most critically, a lack of data reliability, due in part  to inadequate sample sizes and varying interpretations of the standards.
Consistent with key criteria for performance measures, CMS’s  performance standards are comprehensive and limited in number  and overlap. Officials of all of the state survey agencies and CMS regional  offices in our sample indicated that they considered the four nursing home  complaint standards comprehensive. Although the performance standards  system does not include standards for certain steps in the complaint  investigation process, such as intake, officials indicated that the standards  cover key steps, which include prioritizing complaints, scheduling and  conducting investigations, and documenting any deficiencies identified.  The standards are also limited in number and overlap, with each focused  on different aspects of the nursing home complaint process than the  others.
Performance trends cannot be easily assessed because scores are  not comparable over time. Because CMS changed the scoring  methodologies for three of the four nursing home complaint standards  during the past 4 years, it is not readily apparent from scores on these  standards whether state survey agencies’ performance improved or  worsened over that time period. CMS officials generally felt that the  changes had enhanced the standards—in the case of the documentation of  deficiencies and quality of investigations standards, by holding state  survey agencies accountable for meeting all of the underlying  requirements or by highlighting specific areas in need of improvement.  Further, they did not identify the lack of trend data as a major concern.  Officials noted that CMS judges state survey agencies’ performance for a  given year, not in relation to prior years, and does not count scores on a  standard in the first year after a significant change in methodology.
However, a lack of consistent trend data makes it more difficult for CMS  to assess whether the steps that it and the states are taking to improve  performance on the nursing home complaint standards are having the  desired effect.
The balance among standards may be undermined by how the  prioritization standard is scored. In general, the standards are  balanced, so that the incentives created by one standard are  counterbalanced by the incentives created by other standards. However,  because the prioritization standard requires only that complaints be  assigned a priority level at or above the level assigned by CMS reviewers,  this standard may create an incentive for state survey agencies to assign  higher priority levels than are warranted—which may jeopardize the  timeliness of investigations. As one state survey agency official pointed  out, the staff members who prioritize complaints may not be responsible  for conducting investigations; consequently, these staff may be more  focused on the agency’s meeting the prioritization standard than the  timeliness standard and thus err on the side of caution in prioritizing  complaints. According to CMS headquarters officials, the prioritization  standard is scored this way because the agency was most concerned about  complaints being prioritized at too low a level and did not want to fault  state survey agencies for investigating complaints sooner than necessary.  However, officials of two CMS regional offices noted that assigning  complaints too high a priority level can cause misallocation of resources,  as state survey agencies that prioritize complaints at higher levels than are  warranted must investigate these complaints within shorter time frames  than they otherwise would.
Some performance scores are unreliable because of inadequate  sample sizes and varying interpretations of standards among CMS  reviewers. For three of the four CMS performance standards, the samples  specified by CMS are in some cases too small to yield reliable data. Scores  on the prioritization of complaints, quality of investigations, and  documentation of deficiencies standards were generally based on a sample  of 10 to 40 cases (10 percent, up to a maximum of 40). With samples this  small, the margin of error around states’ scores on the prioritization of  complaints standard, for example, was as much as 19 percentage points in  fiscal year 2009. Accordingly, at least some of the states that received  passing marks on this standard may actually have failed, and at least five  of the nine states that received failing marks may actually have passed.  Although the small sample sizes CMS requires make the reviews involved  in certain standards more practical, by reducing the documentation CMS  reviewers must examine, the trade-off is a lack of precision in the scores  for these standards.
Moreover, interpretation of some standards has varied among CMS  reviewers—in terms of both the materials reviewed to assess performance  and how certain requirements were construed by reviewers.
Materials reviewed. To assess the quality of investigations, some CMS  regional offices reviewed only information surveyors entered into the  complaints database, while other CMS regional offices reviewed more  extensive hard-copy notes from complaint investigations. CMS  headquarters officials indicated that relying solely on the information in  the complaints database to assess the quality of investigations was not  consistent with federal guidance, stating that regional office officials  should follow the guidance for the standard, which calls for reviewers to  examine a variety of documents, including surveyor worksheets and  investigation notes. They also noted that the investigation notes are not  required data elements in the complaints database. Some state survey  agency officials said that their scores on this standard have suffered  because the investigation notes in the database do not always provide a  complete picture of the agency’s complaint investigations.
How requirements were construed. State survey agency officials we  interviewed also noted differences in how CMS reviewers understood  certain requirements in the standards, particularly in the documentation of  deficiencies standard. For example, officials described differences in  reviewers’ interpretations of what it means to quantify the extent of a  deficient practice, one of the requirements in that standard. One state  survey agency official said that his agency’s scores on the standards  improved from one half of the year to the next simply because the CMS  staff conducting the review changed. Officials in one of the CMS regions  where all state survey agencies failed the documentation of deficiencies  standard acknowledged the 100 percent failure rate was at least partially  due to a change in the regional office’s review—specifically, regional  managers having issued more explicit instructions to staff about how to  assess states’ performance on particular requirements. The clustering of  failing scores on this standard within certain CMS regions also suggests  regional variation in interpretation; in three regions, all of the state survey  agencies failed the documentation of deficiencies standard in fiscal year  2009, while in the other seven regions, half or fewer of the state survey  agencies failed.
Although some CMS regional offices have tried to ensure consistent  interpretation of the standards within their own regions—for example, by  requiring that multiple reviewers concur on any failing marks given to  state survey agencies and encouraging ongoing dialogue about the  standards—some officials we interviewed believe CMS should do more to  ensure consistency across regions. CMS headquarters officials told us that  the agency has issued additional guidance when officials became aware of  a need for clarification, but some CMS regional office officials said that  parts of the guidance need enhancement and that CMS headquarters  should have more staff dedicated to developing guidance and answering  questions from regional office staff. In addition, some state survey agency  officials suggested that CMS regional offices should have less autonomy in  the performance review process. One official suggested that CMS  headquarters should exert more control over the regional offices with  respect to the review process, and others indicated a need for more  “review of the reviewers”—for example, by having the performance  reviews conducted by each regional office validated by another. Officials  of one state survey agency, noting that state survey agencies can appeal  their performance scores only to the same regional office that conducted  their performance review, suggested that a second regional office should  at least be involved in the appeals process.
section_title: CMS Has Not Made Full Use of Performance Information on State Survey Agencies’ Complaint Investigations
paragraphs: CMS has not made full use of the information it collects about state survey  agencies’ complaint investigation processes through its complaints  database and performance standards system. For example, CMS does not  routinely use data from its complaints database to calculate certain  measures that could enhance its understanding of state survey agencies’  performance investigating complaints and has not publicly reported state  survey agencies’ scores on the performance standards.
CMS has not made full use of data in the complaints database to  monitor performance. In part because of data reliability concerns, CMS  does not routinely calculate certain measures that could shed additional  light on state survey agencies’ performance—such as substantiation rates  or additional measures of the timeliness of investigations.
Substantiation rates, if interpreted by state survey agencies in a consistent  manner, could provide insight into the quality of complaint investigations.  Given the many factors that influence these rates, including whether the  complaints have a basis in fact, it would not be appropriate to require state  survey agencies to achieve a particular rate. However, substantial  variation in rates, either among states or over time, could signal issues  with complaint investigations and prompt further inquiry by CMS. A CMS  headquarters official told us that because some state survey agencies may  consider a complaint to be substantiated even if no federal deficiencies are  cited, CMS headquarters does not systematically monitor substantiation  rates and most CMS regional offices probably do not do so either. The  Patient Protection and Affordable Care Act (PPACA), enacted March 23,  2010, requires HHS to post on the Nursing Home Compare Web site  summary information on substantiated complaints, including their  number, type, severity and outcome, by March 23, 2011. Accordingly, a  CMS official told us that CMS headquarters will issue guidance to ensure  that state survey agencies interpret substantiation in a consistent manner.
Additional measures of timeliness—such as the number of days by which  state survey agencies miss the deadlines for some complaint  investigations—could provide CMS with a more comprehensive picture of  performance in this area. We found that some state survey agencies with  similar scores on CMS’s timeliness standard for actual harm-high  complaints in fiscal year 2009 had very different backlogs of complaint  investigations. For example, looking at two state survey agencies with  performance scores of 82 and 85 percent—which indicates, respectively,  that 18 and 15 percent of their investigations were late—we found that 51  percent of one agency’s late investigations were initiated more than 30  days late in calendar year 2009, compared with 4 percent for the other  agency. Currently, the reliability of timeliness measures such as this is  uncertain because state survey agencies do not necessarily enter all  complaints into CMS’s database or prioritize complaints in the same way.
Responsibility for training to address performance issues has  generally been left to CMS regional offices. The CMS regional offices  in our sample have used information from the performance standards  system to identify performance issues, but training designed to address  these issues has generally been undertaken by individual CMS regional  offices and, as a result, has varied in content and scope. Complaint  investigation training at the national level has been limited and was not  designed to address specific performance issues identified during  reviews. Officials of most of the state survey agencies in our sample  indicated that CMS’s training and guidance was sufficient, but officials of  two state survey agencies noted that their agencies provide any training  above the basic level. One state survey agency official said that CMS  should offer more comprehensive training, including more material on  complaint investigations, so that states are not “sinking or swimming” on  their own and are able to conduct investigations in a more consistent  manner.
PPACA directed HHS to enter into a contract to establish a National  Training Institute to help surveyors develop complaint investigation  skills. However, as of March 2011, funds had not yet been appropriated to  implement this provision of the act, and CMS estimates that it would cost  about $12 million to establish the institute. As a start, CMS has redirected  about $1 million from other projects to initiate a project which will  provide instruction on all aspects of complaint surveys for all facility  types, including nursing homes.
Corrective action plans are not timely and may not address the  underlying causes of performance issues. CMS requires state survey  agencies that fail performance standards to submit plans to improve their  performance, but CMS does not require these plans to be submitted until  halfway through the next performance cycle, which allows little time for  corrective actions to take effect before the next performance review. (See  fig. 4.) Moreover, despite CMS regional office input, the plans do not  necessarily address the underlying causes of state survey agencies’ failure  to meet performance standards. For example, all three of the state survey  agencies in our sample that failed the timeliness of investigations standard  for immediate jeopardy complaints, actual harm-high complaints, or both  in all 4 fiscal years from 2006 through 2009 cited staff shortages as a  reason, but two of the three submitted at least one corrective action plan  during that period that did not propose hiring the additional staff needed.
CMS regional office officials indicated that they had accepted such  corrective action plans because the steps the state survey agencies did  propose—such as developing a graphic analysis tool to track performance  or implementing additional central oversight of regional offices—were  likely to improve performance to some extent, and because CMS does not  have the authority to require state survey agencies to hire or reallocate  staff. Only one of the CMS regional offices in our sample reported ever  having rejected a corrective action plan, and officials of one CMS regional  office told us they preferred that a corrective action plan provide a  realistic account of what a state survey agency was going to try to achieve  rather than propose actions that the agency could not carry out.
Some CMS officials view the penalties the agency might impose for  failure to meet nursing home complaint standards as  counterproductive or unrealistic. CMS’s regulations provide for  penalties to be imposed on a state survey agency for failure to follow  procedures specified by CMS for complaint investigations, such as  reducing funding or terminating the contract under which the state survey  agency conducts standard surveys and complaint investigations. CMS  headquarters officials noted that while CMS has reduced funding to state  survey agencies for failure to meet requirements for standard surveys,  such as statutory time frames, the agency has not done the same for  complaint investigations. One official said that CMS has not done so  partly because of concerns about the fairness of penalizing states for  failure to meet standards that may vary from year to year, as well as  concerns that reducing states’ funding might make it even more difficult  for them to meet the standards. Some CMS regional office officials said  that reducing state survey agencies’ funding for failure to complete  complaint investigations on time made sense, but others said that taking  resources away from the agencies could be counterproductive, further  hampering their ability to carry out investigations. Although CMS could  terminate its contract with a state survey agency, CMS officials we  interviewed indicated that this was not a realistic option.
CMS has not publicly reported state survey agencies’ performance  scores. Public reporting of performance information has been advocated  by GAO and other auditors as a critical step in performance management  because it provides policymakers and the public with information needed  to assess progress and may also serve to motivate agency managers and  staff. While CMS has shared state survey agencies’ scores on the  performance standards with all of the other state survey agencies, it has  not made the scores available to other stakeholders, such as residents,  family members, or advocates. According to a CMS headquarters official,  some state survey agencies have made their own scores publicly available,  but CMS has not yet issued any guidance to the states on public disclosure  of scores. This official told us that CMS plans to issue a policy memo  affirming state survey agencies’ right to disclose their own scores and is  also considering making all of the scores publicly available, possibly on  CMS’s Web site. Although some CMS regional office officials questioned  whether performance reports might too easily be misconstrued by the  public and necessarily gloss over details that would provide a more  nuanced picture of performance, GAO’s prior work on performance  management suggests reports can be structured to avoid these potential  pitfalls—for example, by explaining the limitations of the data and using  clearly defined terms and readily understood tables and graphs to convey  information.
section_title: Conclusions
paragraphs: In the past decade, CMS has made several efforts to improve the intake  and investigation of nursing home complaints by state survey agencies,  including (1) implementation of a database that not only helps state survey  agencies track complaints but also helps CMS monitor the state survey  agencies’ performance and (2) establishment of and refinements to its  performance standards related to nursing home complaints. However, our  review indicates that challenges remain.
CMS’s complaint data have limitations. We found that the lack of  consistency in state surveys agencies’ use of the database—particularly in  terms of which complaints are entered and how certain fields are  interpreted—undermines the reliability of some of the data and limits the  usefulness of the database as a monitoring tool. CMS does not routinely  use the data to calculate measures such as substantiation rates that could  enhance its understanding of complaint investigations partly because of  concerns about the reliability of the data.
CMS’s performance reviews highlight state workload issues.  Although state survey agencies generally prioritized nursing home  complaints in accordance with CMS’s performance standard, we found  that many agencies had difficulty managing a heavy workload of actual  harm-high complaints. In 2009, state survey agencies prioritized 45 percent  of the more than 53,000 nursing home complaints they received as actual- harm high, which requires initiation of an investigation within 10 working  days of prioritization. In fiscal year 2009, 19 state survey agencies failed to  meet the CMS timeliness standard for these complaints. Staffing shortages  and heavy workloads were cited as key reasons by survey agency officials  we interviewed whose states had failed this standard. CMS’s policy for  scoring the prioritization standard may contribute to these workload  issues by creating an incentive for the agency staff who prioritize  complaints to assign higher priority levels than are warranted. While CMS  is correct in asserting that prioritizing complaints at too high a level is  preferable to the reverse, this practice can have a significant impact on  state survey agencies’ workload and thus on their ability to meet  requirements for timely investigations. Additionally, CMS data for 2009  showed that, among investigated complaints prioritized as either  immediate jeopardy or actual harm-high, the percentage substantiated  with at least one federal deficiency cited was higher if the investigation  was initiated within required time frames than if it was not. Though many  factors can affect whether complaints are substantiated, including  whether there is evidence to support them, considerable variation in  substantiation rates, among the states or over time, could indicate  potential concerns with state survey agencies’ complaint investigations.
Some performance standards scores are unreliable due to small  samples and varying interpretations of requirements. CMS has also  made efforts to refine its performance standards for nursing home  complaints. However, as with the complaints data, scores on some  standards are unreliable, because of inadequate sample sizes and varying  interpretations of the standards by the CMS regional office officials who  conduct the performance reviews. While we recognize that CMS may have  opted for small samples for some standards in order to limit the amount of  documentation reviewers must examine each year, sample sizes could be  increased without increasing reviewers’ workloads if performance on  certain standards—those that require document review—were assessed  less frequently than once a year. Less frequent reviews could also help  address the issue of state survey agencies receiving their final scores and  submitting their corrective action plans so far into the next performance  cycle that little time remains for them to improve their performance. The  credibility of the scores could be further enhanced by ensuring that the  standards are consistently interpreted by the CMS regional offices.  Clarifying CMS guidance could help in this regard as well as in ensuring  that state survey agencies understand their responsibilities with respect to  each aspect of the complaint investigation process, including the manner  in which investigation results are communicated to complainants.
CMS is considering making state survey agencies’ scores on the  performance standards publicly available. While we support such a step,  we believe that it is important to consider the reliability of data, as well as  its comparability over time, when deciding which scores to publish. For  such performance reports to be useful to the public, they should also  include meaningful trend data that reflect agencies’ actual progress over  time, as well as a clear explanation of the limitations of the data.
section_title: Recommendations
paragraphs: To ensure that information entered into CMS’s complaints database is  reliable and consistent, we recommend that the Administrator of CMS:  Identify issues with data quality and clarify guidance to states about how  particular fields in the database should be interpreted, such as what it  means to substantiate a complaint.
To strengthen CMS’s assessment of state survey agencies’ performance in  the management of nursing home complaints, we recommend that the  Administrator of CMS take the following three actions:    Conduct additional monitoring of state performance using information  from CMS’s complaints database, such as additional timeliness measures.
Assess state survey agencies’ performance in certain areas—specifically,  documentation of deficiencies, prioritization of complaints, and quality of  investigations—less frequently than once a year.
Assure greater consistency in assessments by identifying differences in  interpretation of the performance standards and clarifying guidance to  state survey agencies and CMS regional offices.
To strengthen and increase accountability of state survey agencies’  management of the nursing home complaints process, we recommend that  the Administrator of CMS take the following three actions:    Clarify guidance to the state survey agencies about the minimum  information that should be conveyed to complainants at the close of an  investigation.
Provide guidance encouraging state survey agencies to prioritize  complaints at the level that is warranted, not above that level.
Implement CMS’s proposed plans to publish state survey agencies’ scores  but limit publication to those performance standards that CMS considers  the most reliable and clear.
section_title: Agency and Other External Comments
paragraphs: We received written comments on a draft of this report from HHS and  from the Association of Health Facility Survey Agencies (AHFSA), the  organization that represents state survey agencies.
subsections: section_title: HHS Comments
paragraphs: HHS provided written comments, which are reproduced in app. III. HHS  generally concurred with all of our recommendations. With respect to our  first recommendation, HHS agreed that CMS should take steps to ensure  that information entered into the agency’s complaints database is reliable  and consistent. HHS said that CMS will convene a workgroup—including  staff from CMS headquarters, CMS regional offices, and state survey  agencies—to address data quality issues. HHS also agreed that CMS needs  to strengthen its assessment of state survey agencies’ performance in the  management of nursing home complaints. HHS said that CMS’s planned  workgroup will review the three specific actions we recommended and  identify ways to strengthen the agency’s oversight process. Finally, HHS  agreed that CMS needs to strengthen and increase accountability of state  survey agencies’ management of the nursing home complaints process.  Regarding the specific actions we recommended, HHS said that CMS will  provide increased guidance to states regarding the minimum information  that must be conveyed to complainants at the close of an investigation and  provide clarification and guidance to ensure that complaints are  prioritized at the appropriate level. With respect to our recommendation  that CMS publish state survey agencies’ scores on certain nursing home  complaint performance standards, HHS said that CMS will work with state  officials and others to identify key information about state survey  agencies’ performance that would be of public value. HHS also provided  technical comments, which we incorporated as appropriate.
section_title: AHFSA Comments
paragraphs: AHFSA emphasized the critical importance of enforcing federal and state  survey and certification standards and noted that in many states,  complaint systems have significant connections to state and local licensing  and enforcement activities, which are outside CMS’s jurisdiction. AHFSA  noted that several of the policy and operational issues raised in our report  create challenges for states. These include lack of clarity about what it  means to substantiate a complaint and lack of timely notification to the  states of any changes in CMS’s performance standards for nursing home  complaints. AHFSA also commented that CMS’s guidance on prioritizing  complaints could be improved but questioned whether many states were  prioritizing complaints at a higher level than is warranted in order to meet  CMS’s prioritization standard. In addition, AHFSA said that the complaint  system is the primary safety net for vulnerable nursing home residents and  therefore suggested that states should err on the side of caution when  prioritizing complaints in order to better protect residents. AHFSA also  provided some state-specific comments, which we incorporated as  appropriate.
As agreed with your office, unless you publicly announce the contents of  this report earlier, we plan no further distribution until 30 days from the  report date. At that time, we will send copies to the Secretary of Health  and Human Services, the Administrator of the Centers for Medicare &  Medicaid Services, and other interested parties. In addition, the report will  be available at no charge on the GAO Web site at http://www.gao.gov.
If you or your staff have any questions about this report, please contact me  at (202) 512-7114 or at dickenj@gao.gov. Contact points for our Offices of  Congressional Relations and Public Affairs may be found on the last page  of this report. GAO staff who made key contributions to this report are  listed in appendix IV.
section_title: Appendix I: CMS’s State-Level Data on Complaints Received, Investigated, and Substantiated by State Survey Agencies, 2009
paragraphs: This appendix provides additional information on the number of  complaints received, investigated, and substantiated by all 50 state survey  agencies and the survey agency for the District of Columbia for 2009 based  on complaints in Centers for Medicare & Medicaid Services’ (CMS)  national complaints database. We included only complaints and excluded  facility-reported incidents, which nursing homes are required to self-report  to state survey agencies. Additionally, we included only complaints that  alleged a violation of federal requirements. In the course of our work, we  found some limitations to the data we obtained, including that state survey  agencies interpret certain variables, such as substantiation, differently  from one another and that data are missing for certain variables, such as  the date on which the state survey agency acknowledged the complaint.  Additionally, we learned that CMS’s national database may not include all  complaints because the state survey agencies may not have entered all of  the complaints they received. Because of the data limitations we found, we  included in our analysis only those variables that we found to be reliable,  and we consider the number of complaints from CMS’s national  complaints database to be a conservative estimate of the total number of  complaints received by state survey agencies.
section_title: Appendix II: Performance Scores for Selected Nursing Home Complaint Performance Standards, Fiscal Year 2009
paragraphs: Score  (Percent)
Pass (√)    or fail (X)
Score (Percent)
Pass (√)    or fail (X)
Score (Percent)
Score  (Percent)
Pass (√)    or fail (X)
Score (Percent)
Pass (√)    or fail (X)
Score (Percent) √= passed performance standard X= failed performance standard. A blank in the score column  indicates that the state received a passing score (at least 90 percent for the prioritization of  complaints standard and at least 95 percent for the timeliness of investigation standards for  immediate jeopardy and actual harm-high complaints).
Pennsylvania officials reported that the state did not pass the prioritization of complaints standard  because it required all complaint investigations to be initiated within 48 hours and survey agency staff  therefore assigned a priority level of immediate jeopardy to nearly all complaints. Because CMS  guidance on this standard was not clear in fiscal year 2009, the CMS regional office that assessed  Pennsylvania’s performance considered complaints assigned a priority level higher than warranted to  be inappropriately prioritized and therefore gave the state a failing score on this standard.
section_title: Appendix III: Comments from the Department of Health and Human Services
section_title: Appendix IV: GAO Contact and Staff Acknowledgments
subsections: section_title: Acknowledgments
paragraphs: In addition to the contact name above, Walter Ochinko, Assistant Director;  Jennie Apter; Shaunessye Curry; Christie Enders; Nancy Fasciano; Dan  Lee; Lisa Motley; Matthew Rae; and Jessica Smith made key contributions  to this report.
section_title: Related GAO Products
paragraphs: Nursing Homes: Complexity of Private Investment Purchases  Demonstrates Need for CMS to Improve the Usability and Completeness  of Ownership Data. GAO-10-710. Washington, D.C.: September 30, 2010.
Poorly Performing Nursing Homes: Special Focus Facilities Are Often  Improving, but CMS’s Program Could Be Strengthened. GAO-10-197.  Washington, D.C.: March 19, 2010.
Nursing Homes: Addressing the Factors Underlying Understatement of  Serious Care Problems Requires Sustained CMS and State Commitment.  GAO-10-70. Washington, D.C.: November 24, 2009.
Nursing Homes: Opportunities Exist to Facilitate the Use of the  Temporary Management Sanction. GAO-10-37R. Washington, D.C.:  November 20, 2009.
Nursing Homes: CMS’s Special Focus Facility Methodology Should Better  Target the Most Poorly Performing Homes, Which Tended to Be Chain  Affiliated and For-Profit. GAO-09-689. Washington, D.C.: August 28, 2009.
Medicare and Medicaid Participating Facilities: CMS Needs to  Reexamine Its Approach for Funding State Oversight of Health Care  Facilities. GAO-09-64. Washington, D.C.: February 13, 2009.
Nursing Homes: Federal Monitoring Surveys Demonstrate Continued  Understatement of Serious Care Problems and CMS Oversight  Weaknesses. GAO-08-517. Washington, D.C.: May 9, 2008.
Nursing Home Reform: Continued Attention Is Needed to Improve  Quality of Care in Small but Significant Share of Homes. GAO-07-794T.  Washington, D.C.: May 2, 2007.
Nursing Homes: Efforts to Strengthen Federal Enforcement Have Not  Deterred Some Homes from Repeatedly Harming Residents. GAO-07-241.  Washington, D.C.: March 26, 2007.
Nursing Homes: Despite Increased Oversight, Challenges Remain in  Ensuring High-Quality Care and Resident Safety. GAO-06-117.  Washington, D.C.: December 28, 2005.
Nursing Home Quality: Prevalence of Serious Problems, While  Declining, Reinforces Importance of Enhanced Oversight. GAO-03-561.  Washington, D.C.: July 15, 2003.
Nursing Homes: Public Reporting of Quality Indicators Has Merit, but  National Implementation Is Premature. GAO-03-187. Washington, D.C.:  October 31, 2002.
Nursing Homes: Federal Efforts to Monitor Resident Assessment Data  Should Complement State Activities. GAO-02-279. Washington, D.C.:  February 15, 2002.
Nursing Homes: Sustained Efforts Are Essential to Realize Potential   of the Quality Initiatives. GAO/HEHS-00-197. Washington, D.C.:  September 28, 2000.
Nursing Home Care: Enhanced HCFA Oversight of State Programs  Would Better Ensure Quality. GAO/HEHS-00-6. Washington, D.C.:  November 4, 1999.
Nursing Home Oversight: Industry Examples Do Not Demonstrate That  Regulatory Actions Were Unreasonable. GAO/HEHS-99-154R. Washington,  D.C.: August 13, 1999.
Nursing Homes: Proposal to Enhance Oversight of Poorly Performing  Homes Has Merit. GAO/HEHS-99-157. Washington, D.C.: June 30, 1999.
Nursing Homes: Complaint Investigation Processes Often Inadequate to  Protect Residents. GAO/HEHS-99-80. Washington, D.C.: March 22, 1999.
Nursing Homes: Additional Steps Needed to Strengthen Enforcement   of Federal Quality Standards. GAO/HEHS-99-46. Washington, D.C.:   March 18, 1999.
California Nursing Homes: Care Problems Persist Despite Federal and  State Oversight. GAO/HEHS-98-202. Washington, D.C.: July 27, 1998.