Nursing Home Quality: CMS Should Continue to Improve Data and Oversight
Published: 2015-10-30T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: To help ensure nursing home residents receive quality care, CMS, an agency within the Department of Health and Human Services (HHS), defines quality standards homes must meet to participate in the Medicare and Medicaid programs. To monitor compliance with these standards, CMS enters into agreements with state survey agencies to conduct on-site surveys of the state's homes and also collects other data on nursing home quality. CMS and others have reported some potential improvements in nursing home quality.
GAO was asked to study these trends. This report examines (1) the extent to which reported nursing home quality has changed in recent years and the factors that may have affected any observed changes, and (2) how CMS oversight activities have changed in recent years. GAO analyzed four sets of CMS quality data—deficiencies cited on standard surveys (2005-2014), consumer complaints (2005-2014), staffing levels (2009-2014), and a sub-set of clinical quality measures (2011-2014)—at both national and state levels. We also reviewed relevant documents, including CMS guidance and Standards for Internal Control in the Federal Government, and interviewed CMS and state agency officials at 5 states selected on factors such as size.
section_title: What GAO Found
paragraphs: In recent years, trends in four key sets of data that give insight into nursing home quality show mixed results, and data issues complicate the ability to assess quality trends. Nationally, one of the four data sets—consumer complaints—suggests that consumers' concerns over quality have increased, while the other three data sets—deficiencies, staffing levels, and clinical quality measures—indicate potential improvement in nursing home quality. For example, the average number of consumer complaints reported per home increased by 21 percent from 2005-2014, indicating a potential decrease in quality. Conversely, the number of serious deficiencies identified per home with an on-site survey, referred to as a standard survey, decreased by 41 percent over the same period, indicating potential improvement. The Centers for Medicare & Medicaid Services' (CMS) ability to use available data to assess nursing home quality is complicated by various issues with these data, which make it difficult to determine whether observed trends reflect actual changes in quality, data issues, or both. For example, clinical quality measures use data that are self-reported by nursing homes, and while CMS has begun auditing the self-reported data, it does not have clear plans to continue. Federal internal control standards require agencies to monitor performance data to assess the quality of performance over time.
In recent years, CMS has made numerous modifications to its nursing home oversight activities, but has not monitored the potential effect of these modifications on nursing home quality oversight. Some of the modifications have expanded or added new oversight activities, while others have reduced existing oversight activities. According to CMS, some of the reductions to oversight activities are in response to an increase in oversight responsibilities and limited number of staff and financial resources. However, CMS has not monitored how the modifications might affect CMS's ability to assess nursing home quality. For example, CMS reduced the number of nursing homes participating in the Special Focus Facility program—which provides additional oversight of homes with a history of poor performance—from 152 in 2013 to 62 in 2014. State survey agency officials who conduct surveys for CMS also made modifications which could have either a positive or negative effect on oversight, but CMS does not have an effective mechanism for monitoring. Federal internal control standards require ongoing monitoring as a part of normal program operations; without this monitoring, CMS cannot ensure that any modifications in oversight do not adversely affect its ability to assess nursing home quality.
section_title: What GAO Recommends
paragraphs: GAO recommends, among other things, that CMS implement a clear plan for ongoing auditing of self-reported data and establish a process for monitoring oversight modifications to better assess their effects. HHS agreed with GAO's recommendations.

section_title: Letter
paragraphs: Nationwide, approximately 15,800 nursing homes provide care to about  1.4 million nursing home residents—a population of elderly and disabled  individuals. To help ensure that this population receives quality care, the  Centers for Medicare & Medicaid Services (CMS), an agency within the  Department of Health and Human Services (HHS), defines the quality  standards that nursing homes must meet in order to participate in the  Medicare and Medicaid programs. To monitor compliance with these  standards, CMS enters into agreements with agencies in each state  government—known as state survey agencies—to conduct required  surveys, or evaluations, of the state’s nursing homes.
For many years we and the HHS Office of the Inspector General have  reported on problems in nursing home quality and on weaknesses in  CMS’s oversight. For example, in multiple reports dating back to the  1990s, we have identified weaknesses in federal and state activities  designed to correct quality problems in nursing homes. CMS and state  survey agencies have made some changes in how they conduct oversight  of nursing home quality, and some potential improvements in nursing  home quality have been reported in recent years; for example, CMS has  reported a decrease in the percentage of homes that, as part of the  survey process, were cited for serious health deficiencies from 2006 to  2012. In addition, CMS and others have reported on improvements in  specific nursing home clinical measures such as reductions in the use of  physical restraints, which can be a sign of improved quality of care. The  characteristics of nursing home residents have also shifted in recent  years; for example, some studies have described a growing number of  nursing home residents with acute medical needs and examined the  potential impact of these patient characteristics on nursing home quality  data.
Policymakers and others have questions about whether changes in  reported nursing home quality represent actual improvements in quality  or, for example, may be the result of changes in how oversight is  performed. In light of these questions, you asked us to provide  information on the quality of care in nursing homes and to study whether  changes in quality are due to improvements in quality or to changes in  oversight. This report examines:  1.  the extent to which reported nursing home quality has changed in  recent years and the factors that may have affected any observed  changes; and  2.  how CMS oversight activities have changed in recent years.
To examine the extent to which reported nursing home quality has  changed in recent years and the factors that may have affected any  observed changes, we analyzed four key sets of quality data from CMS.  Each of these four key sets of data provide an important perspective on  quality and together can give a multi-dimensional view of potential  changes in nursing home quality over time. The four sets of data are: (1)  data on deficiencies cited during standard surveys conducted by state  survey agencies on all nursing homes from 2005-2014; (2) data on  complaints submitted by nursing home residents, families, state nursing  home ombudsmen, and others from 2005-2014; (3) data on nurse staffing  levels from 2009-2014; and (4) data from 2011-2014 on nursing homes’  performance on a sub-set of CMS’s clinical nursing home quality  measures derived from standardized clinical assessments of all nursing  home residents. We analyzed the four sets of data at both the national  and state level for the time periods identified above, which represent the  most recent data available for a ten-year period or its closest equivalent.  At the national level we collected and analyzed data for all 50 states and  Washington, D.C. At the state level we selected five states to focus our  review—California, Florida, Massachusetts, Michigan, and West  Virginia—based on factors such as variation in geographic region, size  (number of nursing homes), and state performance standard scores. We  reviewed documents (such as CMS’s Nursing Home Data Compendium  and a CMS regional office’s annual report) and interviewed officials from  CMS central office, CMS regional offices, and state survey agencies for  the five selected states on the results of our data analysis. We assessed  the reliability of each of the four sets of data and determined that they  were sufficiently reliable, for purposes of describing trends, through  interviews with knowledgeable CMS officials, reviews of supporting  documentation, and comparisons with other published data. (For more  detail on our data analysis, see Appendix I.) In our report we describe  various issues associated with these data. We also reviewed relevant  published literature, interviewed officials from nursing home consumer  and provider groups, and interviewed nursing home researchers who  have published studies on nursing home quality. As part of our review, we  examined whether the data used by CMS to assess nursing home quality  are consistent with federal standards for internal control as well as  leading practices identified by GAO for the effective implementation of the  Government Performance and Results Act of 1993 (GPRA).
To examine how CMS’s oversight activities have changed in recent years  we reviewed relevant documents (such as CMS’s State Operations  Manual, Nursing Home Action Plan, Survey & Certification memos, and  Mission & Priority Documents) and interviewed CMS central office  officials (such as officials from CMS’s Survey & Certification Group). We  also reviewed relevant documents and interviewed officials from the five  selected states and from the states’ corresponding CMS regional offices  regarding the officials’ efforts to oversee nursing home quality. As part of  our review, we examined whether CMS’s oversight of nursing home  quality is consistent with federal standards for internal control.
We conducted this performance audit from December 2014 to October  2015 in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives. We believe  that the evidence obtained provides a reasonable basis for our findings  and conclusions based on our audit objectives.
subsections: section_title: Background
subsections: section_title: Federal Legislation
paragraphs: Titles XVIII and XIX of the Social Security Act, as amended, establish  minimum requirements nursing homes must meet to participate in the  Medicare and Medicaid programs, respectively, with key legislative  provisions enacted below.
The Omnibus Budget Reconciliation Act of 1987 (OBRA ‘87) included  wide-ranging reforms. For example, the law revised the care  requirements providers must meet in order to participate in the  Medicare or Medicaid programs, modified the survey process,  introduced additional enforcement actions, and required nursing  homes to periodically assess the health of nursing home residents.  OBRA ‘87 is considered largely responsible for the quality  environment under which nursing homes operate.  In 2010, Title VI of the Patient Protection and Affordable Care Act  (PPACA) added additional federal and state oversight and  enforcement requirements. Specifically, PPACA requires CMS to  establish a national system to collect and report payroll data on nurse  staffing hours and develop a standardized complaint form. It also  requires states to establish a complaint resolution process.
The Protecting Access to Medicare Act of 2014 (PAMA) requires CMS  to establish a value-based purchasing program, which will increase or  reduce Medicare payments to nursing homes based on an  assessment of their performance against quality measures related to  rates of hospital readmissions. Under this program, lower- performing nursing homes will receive lower incentive payments  compared to better-performing peers, or they may receive a reduction  to their Medicare payment rate. CMS is required to implement the  program starting in fiscal year 2019.
Finally, the Improving Medicare Post-Acute Care Transformation Act  of 2014 (the IMPACT Act) requires the standardization of certain  types of Medicare data across multiple health care settings, including  long term care hospitals, home health agencies, inpatient  rehabilitation facilities, and nursing homes. For example, the  IMPACT Act requires the reporting to CMS of standardized patient  assessment data so that information can be used to help facilitate  coordinated care and improve Medicare beneficiary outcomes.
section_title: Oversight of Nursing Homes
paragraphs: Oversight of nursing homes is a shared federal-state responsibility, with  specific activities occurring at the national, regional, and state levels  performed by the entities listed below.
CMS central office. At the national level, CMS central office oversees  the federal quality standards nursing homes must meet to participate  in the Medicare and Medicaid programs. The office also establishes  the responsibilities of CMS’s regional offices and state survey  agencies in ensuring that federal quality standards for nursing homes  are met. For example, the office issues guidance on how regional and  state entities should assess compliance with federal nursing home  standards.
CMS regional offices. CMS’s 10 regional offices oversee state  activities and report back to CMS central office the results of their  efforts. Specifically, each year regional offices are required to conduct  federal monitoring surveys in at least five percent of each state’s  nursing homes surveyed by the state to assess the adequacy of  surveys conducted by state survey agencies. Regional offices also  use the State Performance Standards System to evaluate state  surveyors’ performance on factors such as the frequency and quality  of state surveys.
State survey agencies. Under agreement with CMS, a state survey  agency in each state assesses whether nursing homes meet CMS’s  standards, allowing them to participate in the Medicare and Medicaid  programs. State survey agencies assess nursing homes using  standard surveys and the statewide average between standard  surveys may not exceed one year. State survey agencies also  conduct complaint investigations as needed. These investigations  generally focus on a specific allegation regarding resident care or  safety made by residents, families, ombudsmen, or others.
section_title: Nursing Home Quality Data
paragraphs: CMS collects data on nursing home quality through a number of sources,  including annual standard surveys and complaint investigations, as well  as other sources such as staffing data and clinical quality measures. The  four key sources that we use in this report are described below.
Standard surveys. By law, every nursing home receiving Medicare or  Medicaid payment must undergo a standard survey not less  frequently than once every 15 months, with a statewide average  frequency of once every 12 months. During a standard survey,  teams of state surveyors conduct a comprehensive on-site evaluation  of compliance with federal quality standards.
In 2005, CMS launched a new survey process called the Quality  Indicator Survey (QIS), designed to improve the accuracy and  consistency of standard surveys and the documentation of  deficiencies. Though the QIS is similar to the traditional survey  processes used for standard surveys, the QIS is electronic rather  than paper-based and draws on a random sample of residents for  closer analysis, as opposed to a sample hand-picked by the  surveyor. As of late 2014, 23 states had transitioned completely to  QIS, while 3 states were using a mixture of QIS and traditional  surveys.
Deficiencies in nursing home care identified during standard  surveys are classified into 1 of 12 categories, each designated  with a different letter, according to scope—the number of  residents potentially affected—and severity—the potential for or  occurrence of harm to residents. (See table 1.)
For most deficiencies, a home is required to prepare a plan of  correction, and, depending on the severity of the deficiency,  surveyors may conduct a revisit to ensure that the nursing home  has implemented its plan and corrected the deficiency. The scope  and severity of a deficiency determine the enforcement actions— such as requiring training for staff, imposing monetary fines,  temporary management changes, or termination from the  Medicare and Medicaid programs—that CMS may impose on a  nursing home.
Complaint investigations. Nursing homes are also surveyed on an  as-needed basis with complaint investigations. Complaints can be  filed with state survey agencies by residents, families, ombudsmen,  or others acting on a resident’s behalf. During a complaint  investigation, state surveyors conduct a focused evaluation of the  nursing home’s compliance with a specific federal quality standard.  CMS sets guidelines state survey agencies should follow when  recording, investigating, and resolving complaints.
Staffing data. Nurse staffing levels are considered a key component  of nursing home quality. Higher nurse staffing levels—particularly  registered nurse staffing levels—are typically linked with higher  quality nursing home care. CMS currently tracks nurse staffing data in  nursing homes.
Clinical quality measures. Nursing homes are required to provide  data on certain clinical quality measures—such as pressure ulcers— for all residents to CMS. CMS currently tracks data for 18 clinical  quality measures.
Nursing homes with consistently poor performance can be selected for  the Special Focus Facility (SFF) program, which requires more frequent  surveys. To select nursing homes for the SFF program, CMS scores the  relative performance of nursing homes and identifies the poorest  performing homes in each state as candidates. State survey agencies  then work with CMS to choose some of the candidates to participate;  homes that are selected receive more intensive oversight, including more  frequent surveys. According to CMS guidance, SFF nursing homes that  fail to significantly improve after three standard surveys, or about 18  months, may be involuntarily terminated from Medicare and Medicaid.  Originally created by CMS in 1998, the SFF program is now statutorily  required under PPACA; CMS is now mandated to conduct its SFF  program for homes that have “substantially failed” to meet applicable  requirements of the Social Security Act, and must conduct surveys of  each facility in the program no less than once every six months.
CMS publicly reports a summary of each nursing home’s quality data on  its Nursing Home Compare website using a five-star quality rating. The  Five-Star Quality Rating System assigns each nursing home an overall  rating and three component ratings—surveys (standard and complaint),  staffing, and quality measures—based on the extent to which the nursing  home meets CMS’s quality standards and other measures. CMS also  works to influence nursing home quality through specific quality  improvement efforts—such as the agency’s effort to improve dementia  care—and through Quality Improvement Organizations (QIOs). CMS  contracts with QIOs to help nursing homes address quality problems such  as pressure ulcers. Nursing homes’ participation in QIO efforts is  voluntary.
section_title: Nursing Home Quality Data Show Mixed Results, Although Data Issues Complicate Ability to Assess Quality Trends
paragraphs: In recent years, trends in four key sets of data that give insight into  nursing home quality show mixed results. Specifically, one of the four  data sets suggests that consumers’ concerns over nursing home quality  have increased, which may indicate a potential decrease in quality, while  the other three sets of data may indicate potential improvement in nursing  home quality. However, data issues complicate the ability to assess  trends in nursing home quality over time.
subsections: section_title: Data on Nursing Home Quality Show Mixed Results
paragraphs: Nationally, in recent years, one of four data sets—number of consumer  complaints—demonstrated a potential decrease in nursing home quality,  while the other three data sets—serious deficiencies cited on standard  surveys, staffing data, and selected clinical quality measures— demonstrated potential quality improvement.
Consumer complaints: From 2005 through 2014, the average number  of consumer complaints reported per nursing home increased nationally  from 3.2 to 3.9, a 21 percent increase over the 10-year period. After an  initial increase, the number of complaints decreased from 2008 through  2011 and then again increased through 2014. (See fig. 1.) Specifically,  52,411 complaints were reported in 2005 and 61,466 complaints were  reported in 2014. At the state level, 30 states had increases in the  number of complaints per home, with increases of more than 50 percent  in 11 of those states, and 21 states had decreases in the number of  complaints per home, with decreases of more than 50 percent in 4 of  those states. (See Appendix II for data for all states.)
Deficiencies cited on standard surveys: From 2005 through 2014, the  number of serious deficiencies—deficiencies that at a minimum caused  harm to the resident—cited per nursing home surveyed decreased  nationally from 0.35 to 0.21, a 41 percent decline over the 10-year period.  (See fig. 2.) Specifically, 4,840 serious deficiencies were cited during  surveys for 13,800 nursing homes in 2005, and 2,660 serious deficiencies  were cited during surveys for 12,759 nursing homes in 2014. At the state  level, we also found a decreasing trend in 36 of the states, and an  increasing trend in the remaining 15 states.
Nurse staffing: From 2009 through 2014, the average total nurse hours  per resident per day—a measure of registered nurse, licensed practical  nurse, and nurse assistant hours—increased nationally from 4.2 to 4.6, a  9.0 percent increase over the 6-year period. (See fig. 3.) In addition, the  average registered nurse hours per resident per day also increased over  the same time period from 0.5 to 0.8, a 51.2 percent increase.  Furthermore, the average total nurse hours per resident per day  increased in all but one state, and the average registered nurse hours per  resident per day increased in all states. Studies suggest that higher levels  of nurse staffing—particularly registered nurse staffing—can result in  higher quality of nursing home care.
Selected quality measures: From 2011 through 2014, nationwide  nursing homes’ scores on all eight of our selected quality measures  improved, at least somewhat, by showing decreases in the number of  reported quality problems, such as falls resulting in major injury. The rate  of decline varied greatly by quality measure. For example, the percentage  of long-stay residents with too much weight loss decreased 1.3 percent  over the 4-year period, while the percentage of short-stay residents with  new or worsening pressure ulcers decreased 52.2 percent. (See fig. 4.)  Similar trends were seen at the state level for most of the quality  measures, although two of the quality measures—long-stay residents with  too much weight loss and long-stay residents experiencing one or more  falls with major injury—had more state-level differences in trends.
In our analysis we also attempted to identify trends across the four data  sets at the nursing home level. Specifically, we examined the data to  determine whether there were nursing homes that consistently performed  poorly across the four data sets over the time periods we reviewed. We  identified 416 homes nationwide with consistently poor performance.  These homes were located in 36 states; the remaining 15 states did not  have any of the consistently poorly performing homes. Of the 416 homes,  71 (17 percent) were included in the SFF program at some point between  2005 and 2014. The number of consistently poorly performing homes is  greater than the number of SFFs allotted in 2015—416 homes and 85  homes, respectively. As will be discussed, the number of nursing homes  included in the SFF program is affected by budget resources, according  to CMS. We also attempted to identify commonalities among homes that  consistently performed poorly compared to homes that performed well  across the four data sets and found that the poorest performing homes  were more likely to be for-profit or large homes (greater than 100 beds)  compared to homes that performed well; our analysis did not reveal a link  between performance and urban or rural location.
section_title: Data Issues Complicate Ability to Assess Quality Trends
paragraphs: CMS’s ability to use available data to assess nursing home quality trends  is complicated by various issues with these data. Specifically, each of the  four key sets of nursing home data we analyzed have issues that make it  difficult to determine whether observed trends reflect actual changes in  quality, data issues, or a combination of both. (See table 2 for examples  of these issues). Under federal internal control standards, agencies  should monitor performance data to assess the quality of performance  over time, and CMS’s ability to do so is hindered by these data issues.  Furthermore, according to GPRA leading practices identified by GAO,  agencies should ensure that data are complete, accurate, and consistent  enough to document performance and support decision making. In the  discussion that follows table 2, we describe in more detail the data issues  that exist in each of the four key data sets CMS uses to assess the quality  of nursing home care.
Consumer complaints: Although the average number of consumer  complaints reported per nursing home increased between 2005 and  2014, it is unclear to what extent this can be attributed to a change in  quality or to state variation in the recording of complaints. State survey  agency officials from the states we interviewed with dramatic increases in  the average number of consumer complaints per nursing home over the  10-year period—California and Michigan—both explained that changes in  how they recorded complaints into CMS’s complaint tracking system  could in part account for the jump in reported complaints. In addition,  officials at one state survey agency explained that the increase in  complaints could also reflect state-level efforts to provide consumers with  more user-friendly options for filing complaints, such as via email. In April  2011, we found differences in how states record and track complaints and  made recommendations to CMS to clarify guidance to states. CMS  concurred with the recommendations. As of July 2015, CMS had not fully  addressed these recommendations; however, the agency had taken  some steps. For example, CMS officials reported that the agency was in  the early stages of a planned multi-year review of its business practices,  including those related to nursing home complaint investigations, and  would provide clarification to states, as needed. Also in 2011, CMS  created a standardized complaint form, as required by PPACA, and made  it available to states and consumers on its website. Use of the form is  voluntary, but it provides consistent information to consumers wishing to  file complaints and facilitates their ability to compose and file complaints  with appropriate supporting information.
Deficiencies cited on standard surveys: Although the decline in the  number of serious deficiencies cited on standard surveys between 2005  and 2014 may indicate an improvement in quality, it may also be  attributed to inconsistencies in measurement. One reason these  measurement inconsistencies occur is the use of both traditional paper- based surveys and QIS electronic surveys, which, for example, have  different methodologies for selecting residents for closer analysis during  the survey. This use of multiple survey types complicates the ability to  compare the results of standard surveys nationally. As of late 2014, 23  states used QIS surveys, 25 states used traditional, and 3 states used  both. An internal CMS review that analyzed survey data from 2012 to  2014 found that states using traditional surveys cited a slightly higher rate  of severe deficiencies than states using the QIS methodology. Some  regional offices and state survey agencies we spoke with noted that QIS  results in fewer deficiencies cited, especially for more serious deficiencies  and deficiencies related to quality of care. As a result, the decreasing  trend of serious deficiencies cited on standard surveys could be the result  of an expanding use of QIS surveys over the same time period, rather  than an improvement in the quality of nursing homes. Officials at one  state survey agency suggested that this change in the number of  deficiencies cited on QIS surveys could be attributed to the way that the  QIS process guides surveyors through a structured investigation.
Another reason for measurement inconsistencies is that state survey  agencies face challenges in completing standard surveys, particularly in  states where there are less experienced surveyors or surveyors with very  heavy workloads, according to CMS and state survey agency officials.  CMS officials said these challenges led to reduced state survey agency  capacity to conduct surveys, which could contribute to the decrease in the  number of deficiencies cited on standard surveys. According to CMS  officials, the recession had the significant and lasting effect of reducing  some state survey agencies’ ability to complete high quality standard  surveys, in part because it caused them to rely on smaller and less  experienced workforces to conduct surveys. Officials from one of the  state survey agencies we interviewed said an increasingly heavy survey  workload distributed among a limited number of surveyors could have  contributed to the decrease in deficiencies cited on standard surveys in  that state. In addition, CMS officials found that the number of hours  surveyors spent completing standard surveys has increased as the  number of deficiencies cited has decreased, which they said suggests  that state survey agencies are relying on newer, less experienced staff to  conduct surveys. Finally, in 2012 and 2013, CMS central office notified  two state survey agencies that their performance was persistently  substandard, and that if the state survey agencies did not improve, then  CMS may terminate its agreement with them to oversee nursing home  quality in their states.
CMS has taken some steps to address the inconsistencies in  measurement for deficiencies cited on standard surveys, and, according  to CMS officials, continues to work on addressing inconsistencies.  Regarding the different survey methodologies, CMS suspended further  implementation of QIS in 2012 to address issues such as deficiency  patterns, software compatibility, the time required to complete QIS, and  surveyor training. States already using QIS continued to do so, but other  states continued to do traditional paper-based surveys. In May 2015 CMS  acknowledged the challenges created by operating two survey types.  CMS officials told us they plan to develop a hybrid model of the QIS and  traditional surveys, with the long-term goal of moving all states to this  hybrid model. However, CMS officials said dates for developing and  implementing the new hybrid model have not been set. CMS officials also  commented on the challenges faced by state survey agencies in  completing standard surveys, and have documented that some level of  variation across states may always exist, but that its systems, such as  national training and state performance standards, are intended to  improve consistency and limit the variation.
Information gathered from the five states we interviewed suggests how  some of the data issues for complaints and deficiencies may be affecting  the trends in quality data within these states. Specifically, figure 5 below  illustrates this potential effect on the trends in the number of consumer  complaints reported and the number of serious deficiencies cited on  standard surveys.
Nurse staffing: Although CMS data show that the average total nurse  hours per resident day increased from 2009 through 2014, CMS does not  have assurances that these data are accurate. CMS uses data on nurse  staffing hours that are self-reported by the nursing homes, but the agency  does not regularly audit these data to ensure their accuracy. CMS has  conducted little auditing of staffing data outside of when state survey  agency surveyors are on-site for inspections, and as a result may be less  likely to identify intentional or unintentional inaccuracies in the self- reported data. Many of the regional office and state survey agency  officials we spoke with expressed concern over the self-reported nature of  these data, noting that it may be easy to misrepresent nurse staff hours.  For instance, one state survey agency stated that nursing home residents  would sometimes tell surveyors that the high numbers of staff on site  during the survey were not normally present and other regional office and  state survey agency officials noted that some homes will “staff up” when  expecting a standard survey in order to make their staffing levels look  better.
Although provisions in PPACA required nursing homes to submit staffing  information based on payroll and other verifiable and auditable data in a  uniform format by March 2012, CMS did not develop a system to begin  collecting data by that date. According to CMS officials, CMS did not  receive funding to develop the electronic payroll-based data system until  the IMPACT Act, enacted in October 2014, provided the necessary multi- year funding. In April 2015 CMS issued a memo outlining a plan to begin  collecting staffing data through its payroll-based system on a voluntary  basis beginning October 2015 and on a mandatory basis beginning July  2016. In August 2015, CMS issued a final rule confirming this timetable  for implementation. According to CMS, the new payroll-based staffing  data system will allow homes to directly upload payroll data or to  manually enter the required information. CMS indicated that the system  will allow staffing and census information to be collected on a regular and  more frequent basis than under the previous method. In addition, CMS  expects the system to be auditable to check accuracy. However, as of  August 2015, CMS had not developed an audit plan and said that it was  too soon in the implementation of the new system to do so. While  updating the method for collecting staffing data could improve data  quality, it is still necessary to audit the data to ensure accuracy.
Selected quality measures: Although nursing homes generally improved  their performance on the eight selected quality measures we reviewed, it  is unclear to what extent this can be attributed to a change in quality or  possible inaccuracies in self-reported data. As previously noted, these  improvements indicate a reduction in reported quality problems at nursing  homes from 2011 through 2014. However, like the nurse staffing data  used by CMS, data on nursing homes’ performance on these measures  are self-reported by nursing homes, and until 2014 CMS conducted little  to no auditing of these data to ensure their accuracy. As a result, CMS  has no assurance that nursing homes’ reported performance on these  measures are accurate improvements. Some regional office and state  survey agency officials told us that public reporting may provide an  incentive for nursing homes to make quality improvements on these  measures. However, some officials noted that nursing homes may  change how they collect and report data on the measures, leading to  improvements in measures without corresponding improvements in actual  quality.
CMS has begun taking steps to help mitigate the problem with self- reported data by starting to audit the data through focused surveys. For  the surveys, CMS selected a sample of nursing homes in each state for  state survey agency surveyors to evaluate whether the self-reported  quality data matches the residents’ medical records. CMS guidance  states that data inaccuracies found during the focused surveys can result  in deficiency citations to the nursing homes. These new surveys were  piloted in 2014 for a sample of five homes in each of the five states and  the pilot found some inconsistencies between self-reported data and  residents’ medical records. In 2015, CMS expanded the focused surveys  to include some homes in each state. According to agency officials, the  2015 focused surveys will be completed by the end of the fiscal year.  CMS officials stated that they intend to continue the focused surveys  nationwide in 2016. The agency did not state firm plans after 2016, so it is  uncertain whether the necessary auditing will continue.
Collectively, these data issues have broader implications related to  nursing home quality trends, including potential effects on the quality  benchmarks CMS sets, consumers’ decisions about which nursing home  to select, and Medicare payments to the homes. Specifically, CMS  established benchmarks for some of its quality data through its Five-Star  Rating System, which indicates the specific staffing levels and quality  measure scores a home needs to receive each star rating. In addition,  consumers can use the Five-Star ratings to help determine which nursing  home to use. Therefore, underlying problems with the data may affect the  benchmarks a nursing home uses to assess its quality performance, the  ratings a home receives, and the home a consumer selects. Furthermore,  data used by CMS to assess quality measures are also used when  determining Medicare payments to nursing homes, so data issues—and  CMS’s internal controls related to the data—could affect the accuracy of  payments. Moreover, the use of quality data for payment purposes will  expand in fiscal year 2019 when a nursing home value-based purchasing  program will be implemented, which will increase or reduce Medicare  payments to nursing homes based on certain quality measures.
section_title: CMS Has Modified Oversight Activities, But Has Not Monitored Potential Effect on Nursing Home Quality Oversight
paragraphs: In recent years CMS has made numerous modifications to its nursing  home oversight activities. Some of these modifications expanded or  added new oversight activities. For example, as previously described,  CMS has introduced, evaluated, and, ultimately, suspended additional  implementation of the QIS survey methodology to additional states;  begun implementing the PPACA requirement to collect and report data on  nurse staffing hours; and begun implementing a process for auditing  quality measure data. In addition, CMS has also expanded the number of  tools available to state surveyors when investigating medication-related  adverse events, increased the amount of nursing home quality data  available to the public, and created new trainings for surveyors on  unnecessary medication usage. (A summary of key oversight  modifications CMS has made can be found in Appendix III.)
Other modifications have reduced existing oversight activities. For  example, CMS has made modifications to the federal monitoring survey  program and the Special Focus Facility program.
Federal monitoring surveys: CMS has reduced the scope of the  federal monitoring surveys regional offices use to evaluate state  surveyors’ skills in assessing nursing home quality. CMS requires  regional offices to complete federal monitoring surveys in at least 5  percent of nursing homes surveyed by the state each year. Before  2013, CMS required that 80 percent of these federal monitoring  surveys be standard surveys—the most comprehensive type—which  cover a broad range of quality issues within a nursing home. The  remaining 20 percent of surveys were permitted to be either revisit or  complaint surveys, which are more narrow in scope. These surveys  focus on a particular deficiency cited on a previous survey or a  specific care issue for which a complaint was reported, respectively,  and are also less-resource intensive as they take less surveyor time  to complete than standard surveys. Starting in 2013, CMS required  fewer federal monitoring surveys to be standard surveys and allowed  more monitoring surveys to be revisits and complaint investigations.
Special Focus Facilities: CMS has reduced the number of nursing  homes participating in the SFF program. Nursing homes placed in the  SFF program receive additional oversight because of the homes’  history of poor performance. For example, instead of being surveyed  at least once every 15 months, SFF homes are surveyed at least once  every 6 months. If homes do not improve the quality of their care,  CMS can terminate their participation in Medicare and Medicaid. In  2013, CMS began to reduce the number of homes in the program by  instructing states to terminate homes that had been in the program for  18 months without improvement and not to select replacements for  these homes or homes that left the program by improving their  performance. As we have previously reported, between 2013 and  2014, the number of nursing homes in the SFF program dropped by  more than half—from 152 to 62. In 2014, CMS began the process of  re-building the number of facilities in the SFF program; however,  according to CMS officials, the process will be slow (as of July 2015  there were 85 SFF homes).
According to CMS officials, these reductions in the scope of CMS’s  nursing home oversight activities were made in order to help the agency  meet its increasing responsibilities with its limited resources. Specifically,  CMS officials said that increasing oversight responsibilities, such as those  required by PPACA, and a limited number of staff and financial resources  at the central, regional, and state levels required the agency to evaluate  its activities and reduce the scope of some activities. For example, CMS  officials noted that reductions to the SFF program were made,  specifically, as a result of the decrease in CMS’s budget under the  Budget Control Act of 2011.
The effect of CMS’s modifications in nursing home oversight activities is  uncertain but could potentially be significant, especially because the  modifications included reductions to activities that CMS considers  essential to oversight. For example, by reducing the scope of federal  monitoring surveys, CMS may be decreasing its ability to monitor state  survey agencies—which is essential because they are one of CMS’s  primary tools for assessing nursing home quality, and a lack of effective  state oversight could, for example, lead to understatement of care  problems. Similarly, by reducing the number of nursing homes in the SFF  program, CMS may be limiting its ability to monitor nursing homes with  poor performance. As previously noted, we found—both in our analysis  for this report and in a prior report—that the number of homes with poor  performance exceeds the number of homes included in the SFF program;  a difference that is made even greater with the reduction to the SFF  program.
CMS officials said a variety of factors, including a review of statutory  requirements, were considered prior to making modifications; however,  the agency is not monitoring how the modifications might affect CMS’s  ability to assess nursing home quality. Therefore, the agency is not able  to determine whether the modifications are the most effective use of its  limited resources for assessing nursing home quality. Under federal  internal control standards, ongoing monitoring should occur in the course  of normal program operations. When discussing the potential effects of  the modifications, CMS officials acknowledged the potential for adverse  impacts on their ability to oversee nursing home quality.
Just as CMS’s central office has made modifications to its nursing home  oversight activities, regional offices and state survey agencies have made  modifications to some of their own nursing home oversight activities— both expansions and reductions. For example, state survey agency  officials we interviewed from one of the states indicated that partly  because of resource constraints, the state had reduced the number of  standard surveys until the frequency between surveys for many nursing  homes reached 36 months—instead of the required frequency of once  every 15 months. Also, state survey agency officials from another state  said that in part due to political changes at the state level their state  survey agency modified its regulatory philosophy towards nursing homes;  in speaking about this shift officials from the state survey agency noted  that the modification resulted in state survey agency surveyors  emphasizing more of a partner role with nursing homes rather than acting  as a strict regulator.
Other officials described modifications that could be helpful to share with  other regional offices and state survey agencies. For example, officials  from one regional office described how they share staff with other  regional offices in order to complete oversight activities—such as federal  monitoring surveys—within required timelines. In addition, these regional  office officials develop an annual report that includes oversight data for  their region, which could be a useful template for other regions,  particularly as officials from another regional office expressed the need  for greater data analysis in their office. Given the tight resource  environment, regional offices and state survey agencies could benefit  from adopting strategies that other agencies have used to successfully  meet their nursing home oversight requirements in an efficient and  effective manner.
However, while CMS’s central office has some ways of collecting  information from regional offices and state survey agencies, the agency  does not have a national approach for routinely collecting such  information on modifications to nursing home oversight activities— whether positive or negative. CMS’s state performance standard system,  which is intended to identify whether a state survey agency is generally  compliant with CMS’s oversight requirements, may elicit isolated  information on negative modifications when asking state survey agencies  to explain poor performance. However, as currently designed, it does not  routinely collect information on state survey agency modifications that  could negatively impact nursing home oversight or provide examples of  best practices. As a result, CMS does not have enough information to  respond to state survey agency modifications—and make adjustments  where needed—in an ongoing or timely manner. As we previously noted,  under federal internal control standards, ongoing monitoring should occur  in the course of normal program operations.
section_title: Conclusions
paragraphs: CMS collects several types of data that give some insight into the quality  of nursing homes, and these data show mixed results. However, these  data could provide a clearer picture of nursing home quality if some  underlying problems with the data are corrected. CMS is in the process of  taking steps to address some of these problems—such as the rollout of  focused surveys to evaluate the data used in quality measures and plans  to use and audit payroll data rather than self-reported data to determine  nursing home staffing levels. If properly implemented, completion of these  steps—as well as pursuing other, longer-term plans such as the eventual  standardization of the survey methodology across all states—has the  potential to make nursing home quality data more comparable and  accurate, allowing more effective tracking of nursing home quality trends.  However, without specific timeframes with milestones to track  implementation of a standardized survey methodology and clear ongoing  audit plans, it is unclear whether these important steps will occur. Federal  internal control standards require agencies to monitor performance data  to assess the quality of performance over time, and CMS’s ability to do so  is hindered by data issues. Timely completion of these actions is  particularly important because Medicare payments to nursing homes will  be dependent on quality data, through the implementation of the value  based purchasing program, starting in fiscal year 2019.
In addition to problems with the data used to measure nursing home  quality, according to CMS officials, the agency faces the challenge of  conducting effective oversight of nursing home quality with its limited  resources, while meeting all of its oversight requirements. CMS has made  modifications to some activities it considered essential to its oversight,  without knowing whether the modifications have affected the agency’s  ability to assess nursing home quality. Further, some modifications made  by CMS regional offices and state survey agencies to their own nursing  home oversight activities could adversely affect the CMS central office’s  ability to oversee nursing home quality, while other modifications could be  effective strategies that could be adopted more widely among regional  offices and state survey agencies. Consistent with federal internal control  standards, establishing an effective process for monitoring modifications  of essential oversight activities made at the CMS central office, CMS  regional office, and state survey agency levels—whether positive or  negative—could allow CMS to better understand the effects these  modifications may have on nursing home quality and make improvements  to its own oversight.
section_title: Recommendations
paragraphs: To improve the measurement of nursing home quality, the Administrator  of CMS should take the following two actions:
Establish specific timeframes, including milestones to track progress,  for the development and implementation of a standardized survey  methodology across all states.
Establish and implement a clear plan for ongoing auditing to ensure  reliability of data self-reported by nursing homes, including payroll- based staffing data and data used to calculate clinical quality  measures.
To help ensure modifications of CMS’s oversight activities do not  adversely affect the agency’s ability to assess nursing home quality and  that effective modifications are adopted more widely, the Administrator of  CMS should establish a process for monitoring modifications of essential  oversight activities made at the CMS central office, CMS regional office,  and state survey agency levels to better understand the effects on  nursing home quality oversight.
section_title: Agency Comments
paragraphs: We provided a draft of this report to HHS for its review and comment.  HHS provided written comments, which are reprinted in appendix IV. In its  written comments, HHS described its efforts to improve nursing home  quality. HHS also concurred with the report’s three recommendations. To  address our first recommendation, HHS stated that it would set  timeframes and milestones for the development and implementation of a  standardized survey methodology. To address our second  recommendation, HHS stated that it would continue to work to address  the reliability of self-reported data by, for example, continuing through  fiscal year 2017 the auditing of clinical quality measures data, which  began in fiscal year 2015. As we describe in this report, ongoing auditing  of self-reported data is important for ensuring data accuracy; as a result,  whenever self-reported data are used for understanding nursing home  quality—including the new electronic payroll system for collecting staffing  data and data used to calculate clinical quality measures—our  recommendation indicates that HHS should plan for and conduct audits in  a continuing manner. To address our third recommendation, HHS stated  that it would review its monitoring of key oversight activities and make  adjustments as indicated. HHS also provided technical comments, which  we incorporated into the final version of this report as appropriate.
As agreed with your office, unless you publicly announce the contents of  this report earlier, we plan no further distribution of it until 30 days from its  date. At that time, we will send copies to the Secretary of Health and  Human Services. In addition, the report will be available at no charge on  the GAO website at http://www.gao.gov.
If you or your staff have any questions about this report, please contact  me at (202) 512-7114 or kohnl@gao.gov. Contact points for our Offices of  Congressional Relations and Public Affairs may be found on the last page  of this report. GAO staff who made key contributions to this report are  listed in appendix IV.
section_title: Appendix I: Additional Detail on Analysis
paragraphs: This appendix describes our scope and methodology for examining the  extent to which reported nursing home quality has changed in recent  years and the factors that may have affected any observed changes. For  this examination, we analyzed four sets of quality data from the Centers  for Medicare & Medicaid Services (CMS). Each set of data provide an  important perspective on quality and together can give a multi- dimensional view of potential changes in nursing home quality over time.
We analyzed the four sets of data at both the national and state level for  the time periods identified below, which represent the most recent data  available for a ten-year period or its closest equivalent. At the national  level we collected and analyzed data for all 50 states and Washington,  D.C. At the state level we selected five states to focus our review— California, Florida, Massachusetts, Michigan, and West Virginia—based  on factors such as variation in geographic region, size (number of nursing  homes), and state performance standard scores.
Deficiencies cited on standard surveys. To identify trends in the  number of serious deficiencies—deficiencies at the actual harm or  immediate jeopardy levels—cited during nursing home standard surveys,  we analyzed data from CMS’s Certification and Survey Provider  Enhanced Reports system for years 2005 through 2014. Specifically, we  calculated the number of serious deficiencies cited during standard  surveys in each year.
Consumer complaints. To identify trends in the number of consumer  complaints regarding resident care or safety reported by residents,  families, ombudsmen, or others, we analyzed data from CMS’s  Automated Survey Processing Environment Complaint/Incident Tracking  System. Specifically, we calculated the total number of complaints  reported—not substantiated—for all nursing homes for years 2005  through 2014.
Nurse staffing. To identify trends in nurse staffing data, specifically the  number of nursing hours per resident day, we analyzed data from CMS’s  Certification and Survey Provider Enhanced Reports. Specifically, we  collected quarterly staffing data on the nursing hours per resident day for  years 2009 through 2014, calculated an average nurse staffing level, and  used CMS’s formula to create adjusted nurse staffing levels.
Clinical quality measures. To identify trends in clinical quality measures,  we analyzed data from CMS’s Minimum Data Set—the data set  containing the standardized clinical assessments nursing homes  complete for all residents and report to CMS—for years 2011 through  2014. We selected eight CMS quality measures to include in our analysis  based on factors such as endorsement by the National Quality Forum and  data reliability. Six of the eight measures are used by CMS for long-stay  residents—the percentage of residents who report moderate to severe  pain; the percentage of high-risk residents with pressure ulcers; the  percentage of residents who lose too much weight; the percentage of  residents who were physically restrained; the percentage of residents  experiencing one or more falls with major injury; and the percentage of  residents who received antipsychotic medication—and the remaining two  measures are used for short-stay residents—the percentage of residents  who report moderate to severe pain and the percentage of residents with  pressure ulcers that are new or worsening. To create an annual score for  each quality measure we averaged quarterly data.
Analysis across four data sets. For each of the four data sets, we  ranked nursing homes by quartile and identified those at the upper  quartile (worst performing) and lower quartile (best performing) for each  year. We then counted the number of years each home fell into the upper  or lower quartile for each quality measure to identify homes with  consistently poor or good performance. We then identified homes with  poor or good performance across all data sets. We also received a list  from CMS of all Special Focus Facilities (SFF) for 2005 through 2014 to  identify how many of the poor performers were or had been in the SFF  program. Finally, we attempted to identify any commonalities among  homes that consistently performed poorly compared to homes that  performed well across the four data sets; for example, using Certification  and Survey Provider Enhanced Reports files for each home, we  examined bed size, non-profit or for-profit status, and urban or rural  location (using zip codes and the Health Resources and Services  Administration’s Area Resource File).
We assessed the reliability of each of the four sets of data and  determined that they were sufficiently reliable for purposes of describing  trends through interviews with knowledgeable CMS officials, reviews of  supporting documentation, and comparisons with other published data.
section_title: Appendix II: State Quality Data Trends in Recent Years
paragraphs: Tables 3-5 provide state-level data for each of the four data sets.  Specifically, Table 3 provides deficiencies cited on standard surveys and  consumer complaint data, Table 4 provides nurse staffing data, and Table  5 provides selected quality measure data.
section_title: Appendix III: Summary of Key Nursing Home Oversight Changes CMS Made from 2005 through 2014
paragraphs: CMS divides its nursing home activities into six dimensions—with the  agency considering four of these dimensions “essential” and two “highly  advisable”. In recent years, CMS has made adjustments to oversight  activities within all dimensions.
section_title: Appendix IV: Comments from the Department of Health and Human Services
section_title: Appendix V: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Staff Acknowledgments
paragraphs: In addition to the contact name above, Will Simerl, Assistant Director;  Wesley Dunn, Julianne Flowers, Krister Friday, Q. Akbar Husain, Kathryn  Richter, Helen Sauer, and Karin Wallestad made key contributions to the  report.