Nursing Home Quality: CMS Should Continue to Improve Data and Oversight
Released: 2015-11-30T00:00:00

Letter

Background

Federal Legislation

Titles XVIII and XIX of the Social Security Act, as amended, establish minimum requirements nursing homes must meet to participate in the Medicare and Medicaid programs, respectively, with key legislative provisions enacted below.

The Omnibus Budget Reconciliation Act of 1987 (OBRA ‘87) included wide-ranging reforms. For example, the law revised the care requirements providers must meet in order to participate in the Medicare or Medicaid programs, modified the survey process, introduced additional enforcement actions, and required nursing homes to periodically assess the health of nursing home residents. OBRA ‘87 is considered largely responsible for the quality environment under which nursing homes operate. In 2010, Title VI of the Patient Protection and Affordable Care Act (PPACA) added additional federal and state oversight and enforcement requirements. Specifically, PPACA requires CMS to establish a national system to collect and report payroll data on nurse staffing hours and develop a standardized complaint form. It also requires states to establish a complaint resolution process.

The Protecting Access to Medicare Act of 2014 (PAMA) requires CMS to establish a value-based purchasing program, which will increase or reduce Medicare payments to nursing homes based on an assessment of their performance against quality measures related to rates of hospital readmissions. Under this program, lower- performing nursing homes will receive lower incentive payments compared to better-performing peers, or they may receive a reduction to their Medicare payment rate. CMS is required to implement the program starting in fiscal year 2019.

Finally, the Improving Medicare Post-Acute Care Transformation Act of 2014 (the IMPACT Act) requires the standardization of certain types of Medicare data across multiple health care settings, including long term care hospitals, home health agencies, inpatient rehabilitation facilities, and nursing homes. For example, the IMPACT Act requires the reporting to CMS of standardized patient assessment data so that information can be used to help facilitate coordinated care and improve Medicare beneficiary outcomes.

Oversight of Nursing Homes

Oversight of nursing homes is a shared federal-state responsibility, with specific activities occurring at the national, regional, and state levels performed by the entities listed below.

CMS central office. At the national level, CMS central office oversees the federal quality standards nursing homes must meet to participate in the Medicare and Medicaid programs. The office also establishes the responsibilities of CMS’s regional offices and state survey agencies in ensuring that federal quality standards for nursing homes are met. For example, the office issues guidance on how regional and state entities should assess compliance with federal nursing home standards.

CMS regional offices. CMS’s 10 regional offices oversee state activities and report back to CMS central office the results of their efforts. Specifically, each year regional offices are required to conduct federal monitoring surveys in at least five percent of each state’s nursing homes surveyed by the state to assess the adequacy of surveys conducted by state survey agencies. Regional offices also use the State Performance Standards System to evaluate state surveyors’ performance on factors such as the frequency and quality of state surveys.

State survey agencies. Under agreement with CMS, a state survey agency in each state assesses whether nursing homes meet CMS’s standards, allowing them to participate in the Medicare and Medicaid programs. State survey agencies assess nursing homes using standard surveys and the statewide average between standard surveys may not exceed one year. State survey agencies also conduct complaint investigations as needed. These investigations generally focus on a specific allegation regarding resident care or safety made by residents, families, ombudsmen, or others.

Nursing Home Quality Data

CMS collects data on nursing home quality through a number of sources, including annual standard surveys and complaint investigations, as well as other sources such as staffing data and clinical quality measures. The four key sources that we use in this report are described below.

Standard surveys. By law, every nursing home receiving Medicare or Medicaid payment must undergo a standard survey not less frequently than once every 15 months, with a statewide average frequency of once every 12 months. During a standard survey, teams of state surveyors conduct a comprehensive on-site evaluation of compliance with federal quality standards.

In 2005, CMS launched a new survey process called the Quality Indicator Survey (QIS), designed to improve the accuracy and consistency of standard surveys and the documentation of deficiencies. Though the QIS is similar to the traditional survey processes used for standard surveys, the QIS is electronic rather than paper-based and draws on a random sample of residents for closer analysis, as opposed to a sample hand-picked by the surveyor. As of late 2014, 23 states had transitioned completely to QIS, while 3 states were using a mixture of QIS and traditional surveys.

Deficiencies in nursing home care identified during standard surveys are classified into 1 of 12 categories, each designated with a different letter, according to scope—the number of residents potentially affected—and severity—the potential for or occurrence of harm to residents. (See table 1.)

For most deficiencies, a home is required to prepare a plan of correction, and, depending on the severity of the deficiency, surveyors may conduct a revisit to ensure that the nursing home has implemented its plan and corrected the deficiency. The scope and severity of a deficiency determine the enforcement actions— such as requiring training for staff, imposing monetary fines, temporary management changes, or termination from the Medicare and Medicaid programs—that CMS may impose on a nursing home.

Complaint investigations. Nursing homes are also surveyed on an as-needed basis with complaint investigations. Complaints can be filed with state survey agencies by residents, families, ombudsmen, or others acting on a resident’s behalf. During a complaint investigation, state surveyors conduct a focused evaluation of the nursing home’s compliance with a specific federal quality standard. CMS sets guidelines state survey agencies should follow when recording, investigating, and resolving complaints.

Staffing data. Nurse staffing levels are considered a key component of nursing home quality. Higher nurse staffing levels—particularly registered nurse staffing levels—are typically linked with higher quality nursing home care. CMS currently tracks nurse staffing data in nursing homes.

Clinical quality measures. Nursing homes are required to provide data on certain clinical quality measures—such as pressure ulcers— for all residents to CMS. CMS currently tracks data for 18 clinical quality measures.

Nursing homes with consistently poor performance can be selected for the Special Focus Facility (SFF) program, which requires more frequent surveys. To select nursing homes for the SFF program, CMS scores the relative performance of nursing homes and identifies the poorest performing homes in each state as candidates. State survey agencies then work with CMS to choose some of the candidates to participate; homes that are selected receive more intensive oversight, including more frequent surveys. According to CMS guidance, SFF nursing homes that fail to significantly improve after three standard surveys, or about 18 months, may be involuntarily terminated from Medicare and Medicaid. Originally created by CMS in 1998, the SFF program is now statutorily required under PPACA; CMS is now mandated to conduct its SFF program for homes that have “substantially failed” to meet applicable requirements of the Social Security Act, and must conduct surveys of each facility in the program no less than once every six months.

CMS publicly reports a summary of each nursing home’s quality data on its Nursing Home Compare website using a five-star quality rating. The Five-Star Quality Rating System assigns each nursing home an overall rating and three component ratings—surveys (standard and complaint), staffing, and quality measures—based on the extent to which the nursing home meets CMS’s quality standards and other measures. CMS also works to influence nursing home quality through specific quality improvement efforts—such as the agency’s effort to improve dementia care—and through Quality Improvement Organizations (QIOs). CMS contracts with QIOs to help nursing homes address quality problems such as pressure ulcers. Nursing homes’ participation in QIO efforts is voluntary.

Nursing Home Quality Data Show Mixed Results, Although Data Issues Complicate Ability to Assess Quality Trends

In recent years, trends in four key sets of data that give insight into nursing home quality show mixed results. Specifically, one of the four data sets suggests that consumers’ concerns over nursing home quality have increased, which may indicate a potential decrease in quality, while the other three sets of data may indicate potential improvement in nursing home quality. However, data issues complicate the ability to assess trends in nursing home quality over time.

Data on Nursing Home Quality Show Mixed Results

Nationally, in recent years, one of four data sets—number of consumer complaints—demonstrated a potential decrease in nursing home quality, while the other three data sets—serious deficiencies cited on standard surveys, staffing data, and selected clinical quality measures— demonstrated potential quality improvement.

Consumer complaints: From 2005 through 2014, the average number of consumer complaints reported per nursing home increased nationally from 3.2 to 3.9, a 21 percent increase over the 10-year period. After an initial increase, the number of complaints decreased from 2008 through 2011 and then again increased through 2014. (See fig. 1.) Specifically, 52,411 complaints were reported in 2005 and 61,466 complaints were reported in 2014. At the state level, 30 states had increases in the number of complaints per home, with increases of more than 50 percent in 11 of those states, and 21 states had decreases in the number of complaints per home, with decreases of more than 50 percent in 4 of those states. (See Appendix II for data for all states.)

Deficiencies cited on standard surveys: From 2005 through 2014, the number of serious deficiencies—deficiencies that at a minimum caused harm to the resident—cited per nursing home surveyed decreased nationally from 0.35 to 0.21, a 41 percent decline over the 10-year period. (See fig. 2.) Specifically, 4,840 serious deficiencies were cited during surveys for 13,800 nursing homes in 2005, and 2,660 serious deficiencies were cited during surveys for 12,759 nursing homes in 2014. At the state level, we also found a decreasing trend in 36 of the states, and an increasing trend in the remaining 15 states.

Nurse staffing: From 2009 through 2014, the average total nurse hours per resident per day—a measure of registered nurse, licensed practical nurse, and nurse assistant hours—increased nationally from 4.2 to 4.6, a 9.0 percent increase over the 6-year period. (See fig. 3.) In addition, the average registered nurse hours per resident per day also increased over the same time period from 0.5 to 0.8, a 51.2 percent increase. Furthermore, the average total nurse hours per resident per day increased in all but one state, and the average registered nurse hours per resident per day increased in all states. Studies suggest that higher levels of nurse staffing—particularly registered nurse staffing—can result in higher quality of nursing home care.

Selected quality measures: From 2011 through 2014, nationwide nursing homes’ scores on all eight of our selected quality measures improved, at least somewhat, by showing decreases in the number of reported quality problems, such as falls resulting in major injury. The rate of decline varied greatly by quality measure. For example, the percentage of long-stay residents with too much weight loss decreased 1.3 percent over the 4-year period, while the percentage of short-stay residents with new or worsening pressure ulcers decreased 52.2 percent. (See fig. 4.) Similar trends were seen at the state level for most of the quality measures, although two of the quality measures—long-stay residents with too much weight loss and long-stay residents experiencing one or more falls with major injury—had more state-level differences in trends.

In our analysis we also attempted to identify trends across the four data sets at the nursing home level. Specifically, we examined the data to determine whether there were nursing homes that consistently performed poorly across the four data sets over the time periods we reviewed. We identified 416 homes nationwide with consistently poor performance. These homes were located in 36 states; the remaining 15 states did not have any of the consistently poorly performing homes. Of the 416 homes, 71 (17 percent) were included in the SFF program at some point between 2005 and 2014. The number of consistently poorly performing homes is greater than the number of SFFs allotted in 2015—416 homes and 85 homes, respectively. As will be discussed, the number of nursing homes included in the SFF program is affected by budget resources, according to CMS. We also attempted to identify commonalities among homes that consistently performed poorly compared to homes that performed well across the four data sets and found that the poorest performing homes were more likely to be for-profit or large homes (greater than 100 beds) compared to homes that performed well; our analysis did not reveal a link between performance and urban or rural location.

Data Issues Complicate Ability to Assess Quality Trends

CMS’s ability to use available data to assess nursing home quality trends is complicated by various issues with these data. Specifically, each of the four key sets of nursing home data we analyzed have issues that make it difficult to determine whether observed trends reflect actual changes in quality, data issues, or a combination of both. (See table 2 for examples of these issues). Under federal internal control standards, agencies should monitor performance data to assess the quality of performance over time, and CMS’s ability to do so is hindered by these data issues. Furthermore, according to GPRA leading practices identified by GAO, agencies should ensure that data are complete, accurate, and consistent enough to document performance and support decision making. In the discussion that follows table 2, we describe in more detail the data issues that exist in each of the four key data sets CMS uses to assess the quality of nursing home care.

Consumer complaints: Although the average number of consumer complaints reported per nursing home increased between 2005 and 2014, it is unclear to what extent this can be attributed to a change in quality or to state variation in the recording of complaints. State survey agency officials from the states we interviewed with dramatic increases in the average number of consumer complaints per nursing home over the 10-year period—California and Michigan—both explained that changes in how they recorded complaints into CMS’s complaint tracking system could in part account for the jump in reported complaints. In addition, officials at one state survey agency explained that the increase in complaints could also reflect state-level efforts to provide consumers with more user-friendly options for filing complaints, such as via email. In April 2011, we found differences in how states record and track complaints and made recommendations to CMS to clarify guidance to states. CMS concurred with the recommendations. As of July 2015, CMS had not fully addressed these recommendations; however, the agency had taken some steps. For example, CMS officials reported that the agency was in the early stages of a planned multi-year review of its business practices, including those related to nursing home complaint investigations, and would provide clarification to states, as needed. Also in 2011, CMS created a standardized complaint form, as required by PPACA, and made it available to states and consumers on its website. Use of the form is voluntary, but it provides consistent information to consumers wishing to file complaints and facilitates their ability to compose and file complaints with appropriate supporting information.

Deficiencies cited on standard surveys: Although the decline in the number of serious deficiencies cited on standard surveys between 2005 and 2014 may indicate an improvement in quality, it may also be attributed to inconsistencies in measurement. One reason these measurement inconsistencies occur is the use of both traditional paper- based surveys and QIS electronic surveys, which, for example, have different methodologies for selecting residents for closer analysis during the survey. This use of multiple survey types complicates the ability to compare the results of standard surveys nationally. As of late 2014, 23 states used QIS surveys, 25 states used traditional, and 3 states used both. An internal CMS review that analyzed survey data from 2012 to 2014 found that states using traditional surveys cited a slightly higher rate of severe deficiencies than states using the QIS methodology. Some regional offices and state survey agencies we spoke with noted that QIS results in fewer deficiencies cited, especially for more serious deficiencies and deficiencies related to quality of care. As a result, the decreasing trend of serious deficiencies cited on standard surveys could be the result of an expanding use of QIS surveys over the same time period, rather than an improvement in the quality of nursing homes. Officials at one state survey agency suggested that this change in the number of deficiencies cited on QIS surveys could be attributed to the way that the QIS process guides surveyors through a structured investigation.

Another reason for measurement inconsistencies is that state survey agencies face challenges in completing standard surveys, particularly in states where there are less experienced surveyors or surveyors with very heavy workloads, according to CMS and state survey agency officials. CMS officials said these challenges led to reduced state survey agency capacity to conduct surveys, which could contribute to the decrease in the number of deficiencies cited on standard surveys. According to CMS officials, the recession had the significant and lasting effect of reducing some state survey agencies’ ability to complete high quality standard surveys, in part because it caused them to rely on smaller and less experienced workforces to conduct surveys. Officials from one of the state survey agencies we interviewed said an increasingly heavy survey workload distributed among a limited number of surveyors could have contributed to the decrease in deficiencies cited on standard surveys in that state. In addition, CMS officials found that the number of hours surveyors spent completing standard surveys has increased as the number of deficiencies cited has decreased, which they said suggests that state survey agencies are relying on newer, less experienced staff to conduct surveys. Finally, in 2012 and 2013, CMS central office notified two state survey agencies that their performance was persistently substandard, and that if the state survey agencies did not improve, then CMS may terminate its agreement with them to oversee nursing home quality in their states.

CMS has taken some steps to address the inconsistencies in measurement for deficiencies cited on standard surveys, and, according to CMS officials, continues to work on addressing inconsistencies. Regarding the different survey methodologies, CMS suspended further implementation of QIS in 2012 to address issues such as deficiency patterns, software compatibility, the time required to complete QIS, and surveyor training. States already using QIS continued to do so, but other states continued to do traditional paper-based surveys. In May 2015 CMS acknowledged the challenges created by operating two survey types. CMS officials told us they plan to develop a hybrid model of the QIS and traditional surveys, with the long-term goal of moving all states to this hybrid model. However, CMS officials said dates for developing and implementing the new hybrid model have not been set. CMS officials also commented on the challenges faced by state survey agencies in completing standard surveys, and have documented that some level of variation across states may always exist, but that its systems, such as national training and state performance standards, are intended to improve consistency and limit the variation.

Information gathered from the five states we interviewed suggests how some of the data issues for complaints and deficiencies may be affecting the trends in quality data within these states. Specifically, figure 5 below illustrates this potential effect on the trends in the number of consumer complaints reported and the number of serious deficiencies cited on standard surveys.

Nurse staffing: Although CMS data show that the average total nurse hours per resident day increased from 2009 through 2014, CMS does not have assurances that these data are accurate. CMS uses data on nurse staffing hours that are self-reported by the nursing homes, but the agency does not regularly audit these data to ensure their accuracy. CMS has conducted little auditing of staffing data outside of when state survey agency surveyors are on-site for inspections, and as a result may be less likely to identify intentional or unintentional inaccuracies in the self- reported data. Many of the regional office and state survey agency officials we spoke with expressed concern over the self-reported nature of these data, noting that it may be easy to misrepresent nurse staff hours. For instance, one state survey agency stated that nursing home residents would sometimes tell surveyors that the high numbers of staff on site during the survey were not normally present and other regional office and state survey agency officials noted that some homes will “staff up” when expecting a standard survey in order to make their staffing levels look better.

Although provisions in PPACA required nursing homes to submit staffing information based on payroll and other verifiable and auditable data in a uniform format by March 2012, CMS did not develop a system to begin collecting data by that date. According to CMS officials, CMS did not receive funding to develop the electronic payroll-based data system until the IMPACT Act, enacted in October 2014, provided the necessary multi- year funding. In April 2015 CMS issued a memo outlining a plan to begin collecting staffing data through its payroll-based system on a voluntary basis beginning October 2015 and on a mandatory basis beginning July 2016. In August 2015, CMS issued a final rule confirming this timetable for implementation. According to CMS, the new payroll-based staffing data system will allow homes to directly upload payroll data or to manually enter the required information. CMS indicated that the system will allow staffing and census information to be collected on a regular and more frequent basis than under the previous method. In addition, CMS expects the system to be auditable to check accuracy. However, as of August 2015, CMS had not developed an audit plan and said that it was too soon in the implementation of the new system to do so. While updating the method for collecting staffing data could improve data quality, it is still necessary to audit the data to ensure accuracy.

Selected quality measures: Although nursing homes generally improved their performance on the eight selected quality measures we reviewed, it is unclear to what extent this can be attributed to a change in quality or possible inaccuracies in self-reported data. As previously noted, these improvements indicate a reduction in reported quality problems at nursing homes from 2011 through 2014. However, like the nurse staffing data used by CMS, data on nursing homes’ performance on these measures are self-reported by nursing homes, and until 2014 CMS conducted little to no auditing of these data to ensure their accuracy. As a result, CMS has no assurance that nursing homes’ reported performance on these measures are accurate improvements. Some regional office and state survey agency officials told us that public reporting may provide an incentive for nursing homes to make quality improvements on these measures. However, some officials noted that nursing homes may change how they collect and report data on the measures, leading to improvements in measures without corresponding improvements in actual quality.

CMS has begun taking steps to help mitigate the problem with self- reported data by starting to audit the data through focused surveys. For the surveys, CMS selected a sample of nursing homes in each state for state survey agency surveyors to evaluate whether the self-reported quality data matches the residents’ medical records. CMS guidance states that data inaccuracies found during the focused surveys can result in deficiency citations to the nursing homes. These new surveys were piloted in 2014 for a sample of five homes in each of the five states and the pilot found some inconsistencies between self-reported data and residents’ medical records. In 2015, CMS expanded the focused surveys to include some homes in each state. According to agency officials, the 2015 focused surveys will be completed by the end of the fiscal year. CMS officials stated that they intend to continue the focused surveys nationwide in 2016. The agency did not state firm plans after 2016, so it is uncertain whether the necessary auditing will continue.

Collectively, these data issues have broader implications related to nursing home quality trends, including potential effects on the quality benchmarks CMS sets, consumers’ decisions about which nursing home to select, and Medicare payments to the homes. Specifically, CMS established benchmarks for some of its quality data through its Five-Star Rating System, which indicates the specific staffing levels and quality measure scores a home needs to receive each star rating. In addition, consumers can use the Five-Star ratings to help determine which nursing home to use. Therefore, underlying problems with the data may affect the benchmarks a nursing home uses to assess its quality performance, the ratings a home receives, and the home a consumer selects. Furthermore, data used by CMS to assess quality measures are also used when determining Medicare payments to nursing homes, so data issues—and CMS’s internal controls related to the data—could affect the accuracy of payments. Moreover, the use of quality data for payment purposes will expand in fiscal year 2019 when a nursing home value-based purchasing program will be implemented, which will increase or reduce Medicare payments to nursing homes based on certain quality measures.

CMS Has Modified Oversight Activities, But Has Not Monitored Potential Effect on Nursing Home Quality Oversight

In recent years CMS has made numerous modifications to its nursing home oversight activities. Some of these modifications expanded or added new oversight activities. For example, as previously described, CMS has introduced, evaluated, and, ultimately, suspended additional implementation of the QIS survey methodology to additional states; begun implementing the PPACA requirement to collect and report data on nurse staffing hours; and begun implementing a process for auditing quality measure data. In addition, CMS has also expanded the number of tools available to state surveyors when investigating medication-related adverse events, increased the amount of nursing home quality data available to the public, and created new trainings for surveyors on unnecessary medication usage. (A summary of key oversight modifications CMS has made can be found in Appendix III.)

Other modifications have reduced existing oversight activities. For example, CMS has made modifications to the federal monitoring survey program and the Special Focus Facility program.

Federal monitoring surveys: CMS has reduced the scope of the federal monitoring surveys regional offices use to evaluate state surveyors’ skills in assessing nursing home quality. CMS requires regional offices to complete federal monitoring surveys in at least 5 percent of nursing homes surveyed by the state each year. Before 2013, CMS required that 80 percent of these federal monitoring surveys be standard surveys—the most comprehensive type—which cover a broad range of quality issues within a nursing home. The remaining 20 percent of surveys were permitted to be either revisit or complaint surveys, which are more narrow in scope. These surveys focus on a particular deficiency cited on a previous survey or a specific care issue for which a complaint was reported, respectively, and are also less-resource intensive as they take less surveyor time to complete than standard surveys. Starting in 2013, CMS required fewer federal monitoring surveys to be standard surveys and allowed more monitoring surveys to be revisits and complaint investigations.

Special Focus Facilities: CMS has reduced the number of nursing homes participating in the SFF program. Nursing homes placed in the SFF program receive additional oversight because of the homes’ history of poor performance. For example, instead of being surveyed at least once every 15 months, SFF homes are surveyed at least once every 6 months. If homes do not improve the quality of their care, CMS can terminate their participation in Medicare and Medicaid. In 2013, CMS began to reduce the number of homes in the program by instructing states to terminate homes that had been in the program for 18 months without improvement and not to select replacements for these homes or homes that left the program by improving their performance. As we have previously reported, between 2013 and 2014, the number of nursing homes in the SFF program dropped by more than half—from 152 to 62. In 2014, CMS began the process of re-building the number of facilities in the SFF program; however, according to CMS officials, the process will be slow (as of July 2015 there were 85 SFF homes).

According to CMS officials, these reductions in the scope of CMS’s nursing home oversight activities were made in order to help the agency meet its increasing responsibilities with its limited resources. Specifically, CMS officials said that increasing oversight responsibilities, such as those required by PPACA, and a limited number of staff and financial resources at the central, regional, and state levels required the agency to evaluate its activities and reduce the scope of some activities. For example, CMS officials noted that reductions to the SFF program were made, specifically, as a result of the decrease in CMS’s budget under the Budget Control Act of 2011.

The effect of CMS’s modifications in nursing home oversight activities is uncertain but could potentially be significant, especially because the modifications included reductions to activities that CMS considers essential to oversight. For example, by reducing the scope of federal monitoring surveys, CMS may be decreasing its ability to monitor state survey agencies—which is essential because they are one of CMS’s primary tools for assessing nursing home quality, and a lack of effective state oversight could, for example, lead to understatement of care problems. Similarly, by reducing the number of nursing homes in the SFF program, CMS may be limiting its ability to monitor nursing homes with poor performance. As previously noted, we found—both in our analysis for this report and in a prior report—that the number of homes with poor performance exceeds the number of homes included in the SFF program; a difference that is made even greater with the reduction to the SFF program.

CMS officials said a variety of factors, including a review of statutory requirements, were considered prior to making modifications; however, the agency is not monitoring how the modifications might affect CMS’s ability to assess nursing home quality. Therefore, the agency is not able to determine whether the modifications are the most effective use of its limited resources for assessing nursing home quality. Under federal internal control standards, ongoing monitoring should occur in the course of normal program operations. When discussing the potential effects of the modifications, CMS officials acknowledged the potential for adverse impacts on their ability to oversee nursing home quality.

Just as CMS’s central office has made modifications to its nursing home oversight activities, regional offices and state survey agencies have made modifications to some of their own nursing home oversight activities— both expansions and reductions. For example, state survey agency officials we interviewed from one of the states indicated that partly because of resource constraints, the state had reduced the number of standard surveys until the frequency between surveys for many nursing homes reached 36 months—instead of the required frequency of once every 15 months. Also, state survey agency officials from another state said that in part due to political changes at the state level their state survey agency modified its regulatory philosophy towards nursing homes; in speaking about this shift officials from the state survey agency noted that the modification resulted in state survey agency surveyors emphasizing more of a partner role with nursing homes rather than acting as a strict regulator.

Other officials described modifications that could be helpful to share with other regional offices and state survey agencies. For example, officials from one regional office described how they share staff with other regional offices in order to complete oversight activities—such as federal monitoring surveys—within required timelines. In addition, these regional office officials develop an annual report that includes oversight data for their region, which could be a useful template for other regions, particularly as officials from another regional office expressed the need for greater data analysis in their office. Given the tight resource environment, regional offices and state survey agencies could benefit from adopting strategies that other agencies have used to successfully meet their nursing home oversight requirements in an efficient and effective manner.

However, while CMS’s central office has some ways of collecting information from regional offices and state survey agencies, the agency does not have a national approach for routinely collecting such information on modifications to nursing home oversight activities— whether positive or negative. CMS’s state performance standard system, which is intended to identify whether a state survey agency is generally compliant with CMS’s oversight requirements, may elicit isolated information on negative modifications when asking state survey agencies to explain poor performance. However, as currently designed, it does not routinely collect information on state survey agency modifications that could negatively impact nursing home oversight or provide examples of best practices. As a result, CMS does not have enough information to respond to state survey agency modifications—and make adjustments where needed—in an ongoing or timely manner. As we previously noted, under federal internal control standards, ongoing monitoring should occur in the course of normal program operations.

Conclusions

CMS collects several types of data that give some insight into the quality of nursing homes, and these data show mixed results. However, these data could provide a clearer picture of nursing home quality if some underlying problems with the data are corrected. CMS is in the process of taking steps to address some of these problems—such as the rollout of focused surveys to evaluate the data used in quality measures and plans to use and audit payroll data rather than self-reported data to determine nursing home staffing levels. If properly implemented, completion of these steps—as well as pursuing other, longer-term plans such as the eventual standardization of the survey methodology across all states—has the potential to make nursing home quality data more comparable and accurate, allowing more effective tracking of nursing home quality trends. However, without specific timeframes with milestones to track implementation of a standardized survey methodology and clear ongoing audit plans, it is unclear whether these important steps will occur. Federal internal control standards require agencies to monitor performance data to assess the quality of performance over time, and CMS’s ability to do so is hindered by data issues. Timely completion of these actions is particularly important because Medicare payments to nursing homes will be dependent on quality data, through the implementation of the value based purchasing program, starting in fiscal year 2019.

In addition to problems with the data used to measure nursing home quality, according to CMS officials, the agency faces the challenge of conducting effective oversight of nursing home quality with its limited resources, while meeting all of its oversight requirements. CMS has made modifications to some activities it considered essential to its oversight, without knowing whether the modifications have affected the agency’s ability to assess nursing home quality. Further, some modifications made by CMS regional offices and state survey agencies to their own nursing home oversight activities could adversely affect the CMS central office’s ability to oversee nursing home quality, while other modifications could be effective strategies that could be adopted more widely among regional offices and state survey agencies. Consistent with federal internal control standards, establishing an effective process for monitoring modifications of essential oversight activities made at the CMS central office, CMS regional office, and state survey agency levels—whether positive or negative—could allow CMS to better understand the effects these modifications may have on nursing home quality and make improvements to its own oversight.

Recommendations

To improve the measurement of nursing home quality, the Administrator of CMS should take the following two actions:

Establish specific timeframes, including milestones to track progress, for the development and implementation of a standardized survey methodology across all states.

Establish and implement a clear plan for ongoing auditing to ensure reliability of data self-reported by nursing homes, including payroll- based staffing data and data used to calculate clinical quality measures.

To help ensure modifications of CMS’s oversight activities do not adversely affect the agency’s ability to assess nursing home quality and that effective modifications are adopted more widely, the Administrator of CMS should establish a process for monitoring modifications of essential oversight activities made at the CMS central office, CMS regional office, and state survey agency levels to better understand the effects on nursing home quality oversight.

Agency Comments

We provided a draft of this report to HHS for its review and comment. HHS provided written comments, which are reprinted in appendix IV. In its written comments, HHS described its efforts to improve nursing home quality. HHS also concurred with the report’s three recommendations. To address our first recommendation, HHS stated that it would set timeframes and milestones for the development and implementation of a standardized survey methodology. To address our second recommendation, HHS stated that it would continue to work to address the reliability of self-reported data by, for example, continuing through fiscal year 2017 the auditing of clinical quality measures data, which began in fiscal year 2015. As we describe in this report, ongoing auditing of self-reported data is important for ensuring data accuracy; as a result, whenever self-reported data are used for understanding nursing home quality—including the new electronic payroll system for collecting staffing data and data used to calculate clinical quality measures—our recommendation indicates that HHS should plan for and conduct audits in a continuing manner. To address our third recommendation, HHS stated that it would review its monitoring of key oversight activities and make adjustments as indicated. HHS also provided technical comments, which we incorporated into the final version of this report as appropriate.

As agreed with your office, unless you publicly announce the contents of this report earlier, we plan no further distribution of it until 30 days from its date. At that time, we will send copies to the Secretary of Health and Human Services. In addition, the report will be available at no charge on the GAO website at http://www.gao.gov.

If you or your staff have any questions about this report, please contact me at (202) 512-7114 or kohnl@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this report are listed in appendix IV.

Appendix I: Additional Detail on Analysis

This appendix describes our scope and methodology for examining the extent to which reported nursing home quality has changed in recent years and the factors that may have affected any observed changes. For this examination, we analyzed four sets of quality data from the Centers for Medicare & Medicaid Services (CMS). Each set of data provide an important perspective on quality and together can give a multi- dimensional view of potential changes in nursing home quality over time.

We analyzed the four sets of data at both the national and state level for the time periods identified below, which represent the most recent data available for a ten-year period or its closest equivalent. At the national level we collected and analyzed data for all 50 states and Washington, D.C. At the state level we selected five states to focus our review— California, Florida, Massachusetts, Michigan, and West Virginia—based on factors such as variation in geographic region, size (number of nursing homes), and state performance standard scores.

Deficiencies cited on standard surveys. To identify trends in the number of serious deficiencies—deficiencies at the actual harm or immediate jeopardy levels—cited during nursing home standard surveys, we analyzed data from CMS’s Certification and Survey Provider Enhanced Reports system for years 2005 through 2014. Specifically, we calculated the number of serious deficiencies cited during standard surveys in each year.

Consumer complaints. To identify trends in the number of consumer complaints regarding resident care or safety reported by residents, families, ombudsmen, or others, we analyzed data from CMS’s Automated Survey Processing Environment Complaint/Incident Tracking System. Specifically, we calculated the total number of complaints reported—not substantiated—for all nursing homes for years 2005 through 2014.

Nurse staffing. To identify trends in nurse staffing data, specifically the number of nursing hours per resident day, we analyzed data from CMS’s Certification and Survey Provider Enhanced Reports. Specifically, we collected quarterly staffing data on the nursing hours per resident day for years 2009 through 2014, calculated an average nurse staffing level, and used CMS’s formula to create adjusted nurse staffing levels.

Clinical quality measures. To identify trends in clinical quality measures, we analyzed data from CMS’s Minimum Data Set—the data set containing the standardized clinical assessments nursing homes complete for all residents and report to CMS—for years 2011 through 2014. We selected eight CMS quality measures to include in our analysis based on factors such as endorsement by the National Quality Forum and data reliability. Six of the eight measures are used by CMS for long-stay residents—the percentage of residents who report moderate to severe pain; the percentage of high-risk residents with pressure ulcers; the percentage of residents who lose too much weight; the percentage of residents who were physically restrained; the percentage of residents experiencing one or more falls with major injury; and the percentage of residents who received antipsychotic medication—and the remaining two measures are used for short-stay residents—the percentage of residents who report moderate to severe pain and the percentage of residents with pressure ulcers that are new or worsening. To create an annual score for each quality measure we averaged quarterly data.

Analysis across four data sets. For each of the four data sets, we ranked nursing homes by quartile and identified those at the upper quartile (worst performing) and lower quartile (best performing) for each year. We then counted the number of years each home fell into the upper or lower quartile for each quality measure to identify homes with consistently poor or good performance. We then identified homes with poor or good performance across all data sets. We also received a list from CMS of all Special Focus Facilities (SFF) for 2005 through 2014 to identify how many of the poor performers were or had been in the SFF program. Finally, we attempted to identify any commonalities among homes that consistently performed poorly compared to homes that performed well across the four data sets; for example, using Certification and Survey Provider Enhanced Reports files for each home, we examined bed size, non-profit or for-profit status, and urban or rural location (using zip codes and the Health Resources and Services Administration’s Area Resource File).

We assessed the reliability of each of the four sets of data and determined that they were sufficiently reliable for purposes of describing trends through interviews with knowledgeable CMS officials, reviews of supporting documentation, and comparisons with other published data.

Appendix II: State Quality Data Trends in Recent Years

Tables 3-5 provide state-level data for each of the four data sets. Specifically, Table 3 provides deficiencies cited on standard surveys and consumer complaint data, Table 4 provides nurse staffing data, and Table 5 provides selected quality measure data.

Appendix III: Summary of Key Nursing Home Oversight Changes CMS Made from 2005 through 2014

CMS divides its nursing home activities into six dimensions—with the agency considering four of these dimensions “essential” and two “highly advisable”. In recent years, CMS has made adjustments to oversight activities within all dimensions.

Appendix IV: Comments from the Department of Health and Human Services

Appendix V: GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the contact name above, Will Simerl, Assistant Director; Wesley Dunn, Julianne Flowers, Krister Friday, Q. Akbar Husain, Kathryn Richter, Helen Sauer, and Karin Wallestad made key contributions to the report.