Nursing Homes: CMS's Special Focus Facility Methodology Should Better Target the Most Poorly Performing Homes, Which Tended to Be Chain Affiliated and For-Profit
Published: 2009-08-28T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: In 1998, CMS established the Special Focus Facility (SFF) Program as one way to address poor performance by nursing homes. The SFF methodology assigns points to deficiencies cited on standard surveys and complaint investigations, and to revisits conducted to ensure that deficiencies have been corrected. CMS uses its methodology periodically to identify candidates for the program--nursing homes with the 15 worst scores in each state--but the program is limited to 136 homes at any point in time because of resource constraints. In 2008, CMS introduced a Five-Star Quality Rating System that draws on the SFF methodology to rank homes from one to five stars. GAO assessed CMS's SFF methodology, applied it on a nationwide basis using statistical scoring thresholds, and adopted several refinements to the methodology. Using this approach, GAO determined (1) the number of most poorly performing homes nationwide, (2) how their performance compared to that of homes identified using the SFF methodology, and (3) the characteristics of such homes.
section_title: What GAO Found
paragraphs: According to GAO's estimate, almost 4 percent (580) of the roughly 16,000 nursing homes in the United States could be considered the most poorly performing. These 580 homes overlap somewhat with the 755 SFF Program candidates--the 15 worst homes in each state--and the 136 homes actually selected by states as SFFs. For example, GAO's estimate includes 40 percent of SFF Program candidates and about half of the active SFFs as of December 2008 and February 2009, respectively. Under GAO's estimate, however, the most poorly performing homes are distributed unevenly across states, with 8 states having no such homes and 10 others having from 21 to 52 such homes. CMS has structured the SFF Program so that every state (except Alaska) has at least one SFF even though the worst performing homes in each state are not necessarily the worst performing homes in the nation. To identify the worst homes in the nation, GAO applied CMS's SFF methodology on a nationwide basis using statistical scoring thresholds and made three refinements to that methodology, which strengthened GAO's estimate. The scoring thresholds were (1) necessary because there were no natural break points that delineated the most poorly performing homes from all other nursing homes and (2) conservative, focusing on chronic poor performance generally over a 2- or 3-year period or very poor performance over about 1 year. The most poorly performing homes identified by GAO averaged over 46 percent more serious deficiencies that caused harm to residents and over 19 percent more deficiencies that placed residents at risk of death or serious injury (immediate jeopardy), compared to the 755 SFF Program candidates identified by CMS's approach. GAO's three refinements to CMS's SFF methodology had a moderate effect on the composition of the list of homes that GAO identified as the most poorly performing. First, deficiency points from CMS's Five-Star Quality Rating System were used because they decreased the disparity between immediate jeopardy and lower-level deficiencies, such as those with the potential for more than minimal harm, which compensates somewhat for the understatement of serious deficiencies in some states. Second, homes received extra points when certain actual harm deficiencies occurred in standards areas that CMS categorizes as substandard quality of care, an important change because we found that many homes had at least one such deficiency. Third, the full deficiency history of homes was included. CMS recognizes that its methodology overlooks deficiencies for some homes, which almost always results in scores that are lower than if all deficiencies were included in the scores. GAO found that the most poorly performing nursing homes had notably more deficiencies with the potential for more than minimal harm or higher and more revisits than all other nursing homes. For example, the most poorly performing nursing homes averaged about 56 such deficiencies and 2 revisits, compared to about 20 such deficiencies and less than 1 revisit for all other homes. In addition, the most poorly performing homes tended to be chain affiliated and for-profit and have more beds and residents.

section_title: Letter
paragraphs: The nation’s 1.4 million nursing home residents are a highly vulnerable  population of elderly and disabled individuals who rely on nursing homes  to provide high-quality care. The Centers for Medicare & Medicaid  Services (CMS) contracts with state survey agencies to conduct  inspections, known as standard surveys, and complaint investigations to  determine whether the nation’s roughly 16,000 nursing homes are  complying with federal quality standards. Nursing homes must meet those  standards in order to participate in Medicare and Medicaid. Our prior  reports have found that some nursing homes are chronically  noncompliant; that is, they have been cited repeatedly by state survey  agencies for serious deficiencies, such as residents having preventable  pressure sores that harmed them or put them at risk of death or serious  injury. In 1998, CMS established the Special Focus Facility (SFF) Program  as one way to address poor performance by nursing homes.
CMS uses a formula—the SFF methodology—to score the relative  performance of nursing homes and identify the 15 poorest performing  homes in each state as candidates for the SFF Program. State survey  agencies then work with CMS to choose some of those candidates to  participate in the program; those selected receive more intensive  oversight, including more frequent surveys. CMS limits the SFF Program to  136 nursing homes nationwide (fewer than 1 percent of nursing homes) at  any point in time because of resource constraints. Our prior reports have  demonstrated that not all chronically noncompliant nursing homes are  included in this program and that the number of poorly performing nursing  homes nationwide is therefore greater than 136.
You expressed interest in CMS’s efforts to influence the performance of  poorly performing nursing homes. In this report, we (1) determined the  number of nursing homes in the United States that can be identified as the  most poorly performing, (2) analyzed how those homes’ performance  compared to that of nursing homes identified using CMS’s SFF  methodology, and (3) assessed the characteristics of the most poorly  performing nursing homes that distinguish them from all other nursing  homes. You also asked us to examine the operation of the SFF Program,  including its effect on the performance of homes selected as SFFs. We will  address this portion of your request in a subsequent report.
To determine the number of poorly performing nursing homes in the  nation and how the performance of such homes compared to that of  homes identified using CMS’s SFF methodology, we began by interviewing  CMS officials, reviewing documentation related to the agency’s SFF  methodology, obtaining a copy of the computer program CMS used to  score and rank nursing homes, and analyzing data on deficiencies and  revisits from CMS’s On-Line Survey, Certification, and Reporting system  (OSCAR) database. The SFF methodology creates a total score for each  nursing home over three cycles by assigning points to (1) deficiencies  cited during the three most recent standard surveys, (2) deficiencies cited  on the last 3 years of complaint investigations, and (3) the number of  revisits surveyors made to ensure that the nursing home had corrected the  deficiencies cited on the three most recent standard surveys. To  determine the adequacy of the SFF methodology, we compared it to other  compliance-based measures of poor performance and tested the sensitivity  of the methodology to variations such as not giving greater weight to  recent poor performance compared to poor performance that occurred in  earlier years. Based on our examinations of the SFF methodology,  document review, and interviews, we concluded that the SFF methodology  is reasonable and comprehensive. Using our assessment of the SFF  methodology, we developed our estimate of the most poorly performing  nursing homes by (1) applying the SFF methodology on a nationwide basis  using statistical scoring thresholds and (2) adopting three refinements that  helped to identify some homes that would otherwise have been missed but  that had a moderate impact on the composition of the list of homes we  identified as the most poorly performing. We estimated the number of  most poorly performing nursing homes in the nation and the number of  homes identified using CMS’s SFF methodology—which we refer to as  SFF Program candidates—by analyzing OSCAR data from a particular  point in time (December 2008). We also analyzed a list obtained from  CMS of SFFs that participated in the program from January 2005 through  February 2009 to determine the number of poorly performing nursing  homes that were also SFFs.
To determine the characteristics of the most poorly performing nursing  homes that distinguish them from all other nursing homes, we analyzed  CMS data as of December 2008 on deficiencies, revisits, and other  information that describe nursing home characteristics. We also analyzed  case-mix-adjusted nurse staffing hours available from CMS’s Five-Star  Quality Rating System (Five-Star System), which were dated November  2008, and data related to certain enforcement actions obtained from CMS  in October 2008.
Our work was also informed by analyzing information available at CMS’s  Providing Data Quickly Web site; reviewing our prior reports; interviewing  experts in long-term care research; interviewing officials from CMS’s  central office, all 10 CMS regions, and 14 state survey agencies; and  reviewing some states’ approaches to rating nursing home quality. For a  more detailed discussion of our scope and methodology, see appendix I.  To ensure the reliability of the OSCAR deficiency and revisit data for our  purposes as well as the reliability of the data we analyzed to determine the  characteristics of the most poorly performing nursing homes compared to  those of all other nursing homes, we interviewed CMS officials, reviewed  CMS documentation, conducted electronic testing to identify obvious  errors, and traced a selection of records to another CMS reporting system.  Based on these activities, we determined that the data we analyzed were  sufficiently reliable for our purposes.
We conducted this performance audit from February 2008 through August  2009 in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives. We believe  that the evidence obtained provides a reasonable basis for our findings  and conclusions based on our audit objectives.
subsections: section_title: Background
paragraphs: Titles XVIII and XIX of the Social Security Act establish minimum  requirements that all nursing homes must meet to participate in the  Medicare and Medicaid programs, respectively. The Omnibus Budget  Reconciliation Act of 1987 focused the requirements on the quality of care  actually provided by a home. To help beneficiaries make informed  decisions when selecting or evaluating nursing homes, CMS increased the  amount of information publicly available on its Nursing Home Compare  Web site in 2008 by rating the quality of each nursing home on a five-level  scale.
subsections: section_title: Standard Surveys and Complaint Investigations
paragraphs: To assess whether nursing homes meet federal quality standards, state  survey agencies conduct standard surveys, which occur roughly once per  year, and complaint investigations. A standard survey involves a  comprehensive assessment of quality standards. In contrast, complaint  investigations generally focus on a specific allegation regarding resident  care or safety made by a resident, family member, or nursing home staff.
Federal quality standards focus on the delivery of care, resident outcomes,  and facility conditions. These quality standards, totaling approximately  200, are grouped into 15 categories, such as Resident Rights, Quality of  Care, Quality of Life, and Resident Behavior and Facility Practices.  Nursing homes that meet these quality standards can be certified to  participate in Medicare, Medicaid, or both programs. Homes may  occasionally change their participation type, or, according to CMS, states  may require nursing homes to change their participation type. We refer to  this type of change that results in a new provider identification number as  a “technical status change.” Such a change may affect the source of  payment—Medicare or Medicaid—that the nursing home is eligible to  receive. When a technical status change occurs, CMS’s SFF methodology  as applied does not incorporate the nursing home’s complete survey  history.
section_title: Survey Deficiencies
paragraphs: States classify deficiencies identified during either standard surveys or  complaint investigations in 1 of 12 categories according to their scope   (i.e., the number of residents potentially or actually affected) and severity  (i.e., the potential for or occurrence of harm to residents). (See table 1.)
An A-level deficiency is the least serious and is isolated in scope, while an  L-level deficiency is the most serious and is widespread throughout the  nursing home. Nursing homes with deficiencies at the A, B, or C levels are  considered to be in substantial compliance with quality standards,  whereas nursing homes with D-level through L-level deficiencies are  considered noncompliant. For most deficiencies, a home is required to  prepare a plan of correction, and depending on the severity of the  deficiency, surveyors may conduct a revisit to ensure that the nursing  home has implemented its plan and corrected the deficiency. Revisits are  not required for most deficiencies below the actual harm level—A through  F.
As we reported in May 2008, there can be considerable variation among  states in the proportion of nursing homes cited for deficiencies at the G  through L levels. We concluded that this interstate variation suggests that  surveyors in some states are missing some serious deficiencies or  understating their scope and severity. We provided examples of such  understatement in our May 2008 report. Specifically, we reported that  during fiscal years 2002 through 2007, about 15 percent of federal  comparative surveys nationwide found that the state surveys had failed to  cite at least one deficiency at the most serious levels of noncompliance (G  through L levels) and about 70 percent of them found that the state  surveys had failed to cite at least one deficiency at the potential for more  than minimal harm level (D through F levels).
section_title: Enforcement Actions
paragraphs: When deficiencies are cited, federal enforcement actions known as  sanctions can be imposed to encourage homes to make corrections.  Sanctions are generally reserved for serious deficiencies—those at the G  through L levels—that constitute actual harm and immediate jeopardy to  residents. Sanctions include fines known as civil money penalties, denial  of payment for new Medicare or Medicaid admissions, and termination  from the Medicare and Medicaid programs. Such sanctions can affect a  home’s revenues and therefore provide financial incentives to return to  and maintain compliance. CMS requires states to refer for immediate  sanction homes that receive at least one G- through L-level deficiency on  successive standard surveys or intervening complaint investigations. In  addition, a nursing home with one or more deficiencies at the F through L  level—but not G level—in Quality of Care, Quality of Life, or Resident  Behavior and Facility Practices must be cited for substandard quality of  care (SQC), which generally results in the home’s losing its approval to  hold in-house or facility-sponsored nurse aide training.
section_title: CMS Efforts That Identify Poorly Performing Nursing Homes
paragraphs: Two of CMS’s efforts that identify poorly performing nursing homes are  the SFF Program and the Five-Star System. CMS’s Nursing Home Compare  Web site identifies nursing homes that are in the SFF Program, provides a  rating of from one to five stars, and also includes data on deficiencies cited  during standard surveys and complaint investigations, selected quality of  care measures, and nurse staffing hours. Both the SFF Program and the  Five-Star System score nursing homes by assigning points to deficiencies  and the number of revisits, but the points assigned to certain deficiencies  differ.
subsections: section_title: The Special Focus Facility Program
paragraphs: CMS compiles a list of SFF candidates for each state generally on a  quarterly basis by using the numeric score generated by its SFF  methodology. The SFF candidates are those nursing homes with the 15  highest total scores in each state. From the candidate list, state officials  select, with CMS concurrence, nursing homes they think should  participate in the program based on their knowledge of the candidates’  circumstances. With the exception of Alaska, each state has between one  and six SFFs, depending on the number of nursing homes in the state.  CMS requires states to survey SFFs twice as frequently as other nursing  homes to help motivate SFFs to improve. If an SFF meets CMS’s criteria  for improved performance, CMS removes the SFF designation and the  nursing home “graduates” from the program. According to CMS guidance  to states, SFFs that fail to significantly improve after three standard  surveys, or about 18 months, may be involuntarily terminated from  Medicare and Medicaid. Nursing homes may also choose to terminate from  Medicare and Medicaid voluntarily. (See fig. 1.)
The SFF methodology assigns points to deficiencies on standard surveys  and complaint investigations, and to revisits associated with deficiencies  cited on standard surveys, as follows:    Deficiencies. More points are assigned to deficiencies that are higher in  scope and severity. Additional points are assigned to deficiencies  classified as SQC. For example, a nursing home with one J-level deficiency  in the Quality of Care category would be assigned 75 points (50 points plus  an additional 25 points because the deficiency was SQC). See table 2 for a  comparison of the deficiency points assigned by the SFF methodology and  the Five-Star System.
Revisits. Multiple revisits are an indicator of more serious problems in  achieving or sustaining compliance. The points for revisits are as follows:  0 for the first revisit, 50 for the second revisit, an additional 75 (total 125)  for the third revisit, and an additional 100 (total 225) for the fourth  revisit.
MS has changed its scope and    From 1999 to 2004, each state had two SFFs at any one time, which they  selected from a list of four candidates, and the SFF methodology assign a different numb eficiency data.  der of points to deficiencies using only about 1 year of  In 2005, CMS expanded the program’s scope by changing the number of  SFFs from 1 to 6 per state (excluding Alaska), for a total of 136, and  altered the SFF methodology by changing the points assigned to  deficiencies and using about 3 years of deficiency data, weighted equally.
In 2007, CMS began requiring states to notify a nursing home and its other  accountable parties (i.e., the nursing home’s administrator, owners,  operators, and governing body) when the nursing home was designated as  an SFF.
In 2008, CMS began designating SFFs on the Nursing Home Compare Web  site and also changed the scoring methodology to assign weights to each  year, such that the most recent year’s standard and complaint surveys are  given the greatest weight.
section_title: The Five-Star Quality Rating System
paragraphs: During the course of our work, CMS implemented its Five-Star System for  nursing homes. Every nursing home in the United States is rated from one  (much below average) to five (much above average) stars.
The Five-Star System provides an overall quality rating based on individual  ratings for three separate components: (1) assessment of federal quality  standards from standard surveys and complaint investigations, which CMS  refers to in the Five-Star System as health inspections; (2) ratings on  nursing home staffing levels; and (3) ratings on quality of care measures.  In December 2008, CMS’s Nursing Home Compare Web site began  reporting the star ratings that nursing homes receive for each component  of the Five-Star System as well as an overall quality rating. According to  CMS officials, as of March 2009 the rating for the health inspections  component was based on CMS’s SFF methodology, with one variation: the  Five-Star System assigns more points to D- through I-level deficiencies  than does the SFF methodology. (See table 2.) CMS explained that it  changed some of the points assigned to deficiencies in the Five-Star  System because the purpose is different from that of the SFF Program.  The SFF Program focuses on facilities in each state whose performance is  consistently extremely poor, and so it assigns many points to immediate  jeopardy deficiencies relative to other, lower-level deficiencies. In  contrast, the purpose of the Five-Star System is to distinguish performance  across all nursing homes, rather than focus on the poorest performers, and  so CMS modified the points to provide more emphasis on deficiencies at  the potential for more than minimal harm and actual harm levels. The  rating for the second component, staffing data, is based on two elements— total nursing hours per resident day and registered nurse hours per  resident day. The rating for the third component of the Five-Star System  is based on nursing home performance on 10 quality of care measures,  such as percentage of high-risk residents who have pressure sores.
section_title: Five Hundred Eighty Nursing Homes Could Be Considered the Most Poorly Performing—Fewer Than CMS’s SFF Program Candidates but More Than the Number of SFFs
paragraphs: We estimated that almost 4 percent—or 580—of the nation’s roughly  16,000 nursing homes could be considered the most poorly performing.  These 580 homes overlap somewhat with the 755 SFF Program candidates  and the 136 nursing homes actually selected as SFFs. For example, our  estimate of 580 most poorly performing nursing homes includes (1) 302, or  40 percent, of the 755 SFF Program candidates as of December 2008 (see  fig. 2) and (2) 65 nursing homes that 31 states selected as SFFs from  among the SFF Program candidates, or about half of the active SFFs as of  February 2009. In addition, our estimate resulted in some states having  fewer or more poorly performing homes than CMS currently allocates to  states under the SFF Program. For example, 10 states each had over 20 of  the most poorly performing nursing homes. Indiana had the greatest  number, with 52 such nursing homes, or almost 9 percent of the total of  580 homes. Eight states had no such nursing homes. (See fig. 3.)
section_title: CMS’s Application of the SFF Methodology Misses Many of the Nation’s Most Poorly Performing Nursing Homes
paragraphs: CMS has structured the SFF Program so that every state (except Alaska)  has at least one SFF, and therefore the agency applies the SFF  methodology to identify the 15 worst performing nursing homes in each  state, which are not necessarily the worst performing homes in the  nation. We developed an estimate that identified homes with worse  compliance histories—more deficiencies at the potential for more than  minimal harm level or higher and more revisits—than SFF Program  candidates by applying CMS’s SFF methodology on a nationwide basis and  using statistical scoring thresholds. These two changes had the greatest  impact on the composition of the list of homes we identified as the most  poorly performing compared to CMS’s approach. Our estimate also  incorporated several refinements to the SFF methodology that moderately  improve its ability to identify the most poorly performing nursing homes.
Compliance history. Our estimate of 580 nursing homes identified homes  with more deficiencies at the potential for more than minimal harm level  or higher and more revisits, on average, compared to the 755 SFF   Program candidates. For example, the most poorly performing nursing  homes averaged 46.5 percent more actual harm–level deficiencies and   19.5 percent more immediate jeopardy–level deficiencies, compared to the  755 SFF Program candidates. (See table 3.)
Nationwide estimate. We developed a nationwide estimate because the  worst performing nursing homes in some states had high total scores from  a combination of numerous deficiencies, serious deficiencies, and revisits,  while the worst performing nursing homes in other states did not (see   fig. 4). For example, in the preceding three cycles, we found that the  worst performing nursing home in South Dakota had a score of about 68.  The score was composed of 32 deficiencies at the D level or higher, 2 of  which were at the actual harm level (where the highest scope and severity  level was H) and none of which were at the immediate jeopardy level. In  contrast, during the same three cycles, the worst performing nursing home  in Tennessee had a score of about 1,512, with 63 deficiencies at the D level  or higher. Of these deficiencies, 3 were at the actual harm level and 22 at  the immediate jeopardy level (where the highest scope and severity level  was L). Even the 15th highest-scoring home in Tennessee, with a score of  about 253, had notably more deficiencies at the D level or higher and more  severe deficiencies than the highest-scoring home in South Dakota:  specifically, the Tennessee home had 63 deficiencies at the D level or  higher, 2 of which were at the actual harm level and 8 of which were at   the immediate jeopardy level. If CMS applied its SFF methodology to  identify the worst 755 homes in the nation rather than the worst 15 in each  state, the home ranked 755 would have a score of about 127; however,   48 percent of the SFF Program candidates had scores below this  threshold. As a result, the SFF Program is missing some of the worst  performing nursing homes in the nation.
Statistical scoring thresholds. Absent a fixed number of homes per  state, we developed statistical scoring thresholds because there was no  natural break point delineating the most poorly performing nursing homes  from all other homes. The two statistical scoring thresholds we used  were conservative, because they focused on chronic poor performance  and nonchronic, very poor performance. About 87 percent of the 580  nursing homes that we identified as the most poorly performing exhibited  chronic poor performance; that is, they had high scores in at least two of  the three cycles measured, as well as a high total score. The remaining  roughly 13 percent of nursing homes had nonchronic but very poor  performance; that is, they had very serious poor performance in one cycle  only, which resulted in a very high total score.
Homes that met our chronic poor performance threshold had total scores  above the 93rd percentile for all nursing homes, or total scores ranging  from approximately 168 to approximately 1,017. All of the nonchronic   but very poor performing homes had total scores at or above the   99th percentile for all nursing homes, or total scores ranging from  approximately 330 to approximately 1,577. Table 4 summarizes the  compliance history of two of the most poorly performing homes identified  by our estimate, and appendix II provides a detailed compliance history  for these two homes.
Additional homes might have been identified as the most poorly  performing had we used different thresholds. For example, one nursing  home with a total score of about 324 did not meet our definition for  chronic poor performance and was below the threshold of 330 for  nonchronic, very poor performance. During the three-cycle period, this  nursing home had 41 D- through F-level deficiencies, 5 immediate jeopardy  deficiencies, and a second revisit that contributed to the score, but most of  the deficiencies and the revisit occurred in one cycle.
Refinements made to CMS’s SFF methodology. Our three refinements  to CMS’s SFF methodology had a moderate effect on the composition of  the list of homes we identified as the most poorly performing.
Deficiency points. We believe that the deficiency points used in the Five- Star System are more appropriate for identifying the most poorly  performing nursing homes nationwide than those used in the SFF  methodology because they compensate somewhat for understatement and  the interstate variation in the citation of serious deficiencies. First, given  the significant disparity between immediate jeopardy (50 to 150 points)  compared to lower-level deficiencies (2 to 6 points for D- through F-level  deficiencies), our use of SFF deficiency points to identify the most poorly  performing nursing homes nationwide might have missed poorly  performing nursing homes in states with significant understatement.  Second, there is considerable interstate variation in the citation of serious  deficiencies, including immediate jeopardy–level deficiencies. For  example, in 2008, about 11.3 percent of deficiencies were at the immediate  jeopardy level in one state, but less than 1.0 percent of deficiencies were  cited at that level in 26 states. The Five-Star System, on average, doubles  the points assigned to deficiencies below the immediate jeopardy level,  giving a D-level deficiency 4 points and a G-level deficiency 20 points,  compared to 2 and 10 points, respectively, using the SFF deficiency  methodology. As a result, using the Five-Star System deficiency points,  homes with numerous D- through I-level deficiencies are more likely to be  identified as the most poorly performing. CMS officials told us that they  planned to evaluate the effect of using the Five-Star System deficiency  points on identifying SFF candidates; our analysis showed that it changed  the composition of SFF Program candidates by an average of about 2.5  candidates per state.
Substandard quality of care. In comparison with the SFF methodology  and the Five-Star System, we assigned 5 more points to G-level  deficiencies that occurred in any of the three categories of standards that  CMS considers to be SQC. As noted earlier, CMS does not classify any   G-level deficiencies as SQC. Without this modification, an F-level  deficiency in an SQC area is assigned the same number of points as a G- level deficiency even if the G-level deficiency is in an SQC standard—10  and 20 points, respectively, under the SFF or Five-Star System  methodologies. (See table 2.) This adjustment was important because  approximately 45 percent of all nursing homes had one or more G-level  deficiencies in an SQC category during the three cycles used for  calculating SFF scores. Therefore, assigning SQC points to G-level  deficiencies had an effect on total scores for the nursing homes, which we  used to determine the most poorly performing homes nationwide, and  would have an effect on the composition of CMS’s SFF candidate list. For  example, about 4 percent of the SFF Program candidates—or less than  one candidate per state on average—would change if CMS assigned SQC  points to G-level deficiencies.
Technical status changes. While the SFF methodology does not consider  all deficiencies and revisits identified within the three-cycle period that  occurred before the nursing home’s technical status change, we  incorporated the full histories of nursing homes that underwent a  technical status change. At the time of a technical status change, a new  provider identification number is assigned, and the nursing home’s  complete history under the old number is not combined with that of the  new provider number. For example, a nursing home with a status change  on January 1, 2008, might have a compliance history for only 1 year at the  time we did our work instead of the three cycles called for in the SFF  methodology. The SFF scores for nursing homes that have undergone  technical status changes within the last three cycles are almost always  lower than would be the case if three cycles of deficiency history were  included and, therefore, more favorable than would be justified by the  complete history. We found that almost 1 percent of all nursing homes  (148), including 11 of the 580 we identified as the most poorly performing,  had a technical status change during the last three cycles that affected  their SFF scores. For most states, this change would not have affected  their SFF candidate lists.
section_title: Key Characteristics, such as Chain Affiliation and For- Profit Status Differentiated the Most Poorly Performing Nursing Homes
paragraphs: Compared to all other nursing homes, the most poorly performing nursing  homes in the nation averaged notably more deficiencies at the D level or  higher, more serious deficiencies, and more revisits. They were also more  likely to be for-profit and part of a chain and have more beds and  residents. In addition, they had an average of almost 24 percent fewer  registered nurse hours per resident per day.
subsections: section_title: Actual Harm and Immediate Jeopardy Deficiencies Occurred Significantly More Often for the Most Poorly Performing Nursing Homes
paragraphs: Compared to all other nursing homes, deficiencies over the last three  cycles at the actual harm (G through I) level occurred over 5 times as  often, and deficiencies at the immediate jeopardy (J through L) level  occurred 15 times as often for the most poorly performing homes. (See  table 5.) Furthermore, we found that revisits were made to the most poorly  performing nursing homes 6 times as often as to all other nursing homes.
The most poorly performing nursing homes were more frequently cited for  deficiencies in important care areas and specific standards related to the  delivery of care compared to all other nursing homes. Seven of the 10 most  frequently cited deficiencies at the immediate jeopardy level involved  standards in the categories of care that CMS considers to be SQC and four  of the 10 are related to abuse or neglect. For example, about 42 percent of  the most poorly performing nursing homes had at least one immediate  jeopardy deficiency related to being free of accident hazards in the last  three cycles, compared with about 5 percent for all other nursing homes.  (See table 6.) A larger proportion of the most poorly performing nursing  homes were cited for actual harm in each of the three SQC areas—about  90 percent in Quality of Care, about 31 percent in Resident Behavior and  Facility Practices, and about 17 percent in Quality of Life. In comparison, a  smaller proportion of all other nursing homes were cited for actual harm  in those same categories of care—about 42 percent in Quality of Care,  about 6 percent in Resident Behavior and Facility Practices, and about   2 percent in Quality of Life. (See app. III.)
We also found that from fiscal years 2006 through 2008 the most poorly  performing nursing homes were much more likely to have had deficiencies  that could have resulted in the imposition of at least one immediate  sanction compared to all other nursing homes. For example, in fiscal  year 2008, about 33 percent of the most poorly performing homes may  have been at risk of having at least one immediate sanction imposed,  compared to about 4 percent for all other nursing homes. Nursing homes  that receive at least one G- through L-level deficiency on successive  standard surveys or complaint investigations must be referred for  immediate sanctions, and about 15 percent of the deficiencies for the most  poorly performing nursing homes, on average over the last three cycles,  were at the actual harm or immediate jeopardy level.
section_title: The Most Poorly Performing Nursing Homes Differed from All Others in Several Ways, Including Chain Affiliation and For-Profit Status
paragraphs: We found that the most poorly performing nursing homes differed from all  other nursing homes in terms of the proportion of each group that was  chain affiliated, for-profit, or both. They also differed in size and nurse  staffing.
Type of organization. We found that the most poorly performing nursing  homes were less likely to be hospital based compared to all other nursing  homes. Additionally, compared to all other nursing homes, we found that  the most poorly performing nursing homes were more likely to be part of  for-profit organizations, more likely to be affiliated with a chain  organization, and more likely to be both for-profit and affiliated with a  chain organization. About 55 percent of the most poorly performing  nursing homes were for-profit and chain affiliated, compared to about   41 percent of all other homes. (See table 7.)
Participation in Medicare and Medicaid. We found that a higher  percentage of the most poorly performing homes participated in both  Medicare and Medicaid, and a smaller percentage of such homes  participated only in Medicare or only in Medicaid, compared to all other  nursing homes. (See table 8.)
Beds and residents. We found that a larger percentage of the most  poorly performing homes had more than 100 beds, compared to all other  nursing homes. On average, the most poorly performing nursing homes  had about 23 percent more beds than all other nursing homes.  Additionally, our analysis found that on average, the most poorly  performing homes had almost 14 percent more residents, a lower  occupancy rate, and a greater share of Medicaid patients. (See table 9.)
Nurse staffing levels. Compared to all other nursing homes, the most  poorly performing homes had almost 24 percent fewer registered nurse  hours per resident per day on average. One effect of this difference is  that the most poorly performing nursing homes averaged fewer registered  nurse hours per resident per day as a share of total nursing hours.  Specifically, registered nurse hours made up about 8 percent of total nurse  staffing hours in the most poorly performing nursing homes, compared to  about 10 percent in all other nursing homes. (See table 10.)
section_title: Conclusions
paragraphs: Our estimate of the most poorly performing nursing homes nationwide is  more than four times greater than the 136 homes that receive enhanced  scrutiny under CMS’s SFF Program. We believe that our estimate is  conservative, because we focused only on those nursing homes with  chronic poor performance over time or with very poor performance in one  survey cycle. Because of resource constraints, CMS limits the size of the  SFF Program, requiring every state except Alaska to select from 1 to 6  homes—an allocation based on the number of nursing homes in each  state—from a list of 15 candidates. The homes selected are not necessarily  the most poorly performing homes in the nation but rather are among the  poorest performers in each state. In contrast, the 580 homes we identified  have more deficiencies at the potential for more than minimal harm level  or higher and more revisits on average than the 755 homes identified as  potential SFF candidates using CMS’s SFF methodology. Our estimate also  revealed that the state-by-state distribution of the most poorly performing  homes nationwide is uneven, calling into question the approach CMS uses  to allocate SFFs across states.
Furthermore, we believe that CMS’s SFF Program and the Five-Star  System could be strengthened by incorporating the three enhancements  we made to identify the most poorly performing homes nationwide:    First, we adopted the deficiency points that CMS developed for its Five- Star System because they compensate somewhat for understatement and  the interstate variation in the citation of serious deficiencies, an important  consideration for our nationwide estimate of the most poorly performing  nursing homes. Currently, CMS uses a different set of deficiency points for  the SFF methodology, but agency officials told us that they planned to  study the effect of using a common set of numeric points—the Five-Star  System deficiency points—for both methodologies.
Second, we added points for G-level deficiencies in the three standard  areas that CMS considers to be an indication of SQC. We found that about  4 percent of the SFF candidates—or less than 1 candidate per state, on  average—would change if CMS assigned SQC points to G-level  deficiencies when they were cited in an SQC area. Without such an  adjustment, an F-level deficiency in an SQC area would receive the same  number of deficiency points as a G-level deficiency in the same standard  area. Approximately 45 percent of all nursing homes had one or more   G-level deficiencies in an SQC category during the three cycles used for  calculating SFF scores.
Third, we incorporated the full compliance history of homes that  underwent technical status changes. For example, a nursing home with a  technical status change on January 1, 2008, might have a compliance  history for only 1 year at the time we did our work instead of the three  cycles called for by the SFF methodology. The SFF scores of homes that  have undergone technical status changes within the last three cycles are  almost always lower than if all three cycles were considered. We also  found that the Five-Star System does not accurately take into  consideration technical status changes because it imputes a total score to  account for one missing standard survey rather than using actual survey  results.
section_title: Recommendations for Executive Action
paragraphs: To improve the targeting of scarce survey resources, the Administrator of  CMS should consider an alternative approach for allocating the 136 SFFs  across states, by placing more emphasis on the relative performance of  homes nationally rather than on a state-by-state basis, which could result  in some states having only one or not any SFFs and other states having  more than they are currently allocated.
To improve the SFF methodology’s ability to identify the most poorly  performing nursing homes, the Administrator of CMS should make the  following three modifications:  1.  Consider using a common set of numeric points for identifying poorly  performing nursing homes by determining the effect of adopting those  associated with the Five-Star System for the SFF methodology. 2.  Assign points to G-level deficiencies in SQC areas equivalent to those  additional points assigned to H- and I-level deficiencies in SQC areas.  3.  Account for a nursing home’s full compliance history regardless of  technical status changes.
To ensure consistency with the SFF methodology, CMS should also  consider making two of these modifications—the SQC and full compliance  history changes—to its Five-Star System.
section_title: Agency Comments and Our Evaluation
paragraphs: We obtained written comments on our draft report from CMS, which are  reprinted in appendix IV. CMS noted that our report adds value regarding  the methods that CMS and the nursing home industry should use to  address the issue of homes that consistently demonstrate quality of care  problems and indicated that the agency would seriously consider all of our  recommendations. CMS generally agreed in principle with our  recommendations.
In response to our first recommendation, CMS noted that it would  evaluate a “hybrid” approach that would assign some SFFs using homes’  performance in each state and other SFFs on their relative national  ranking. If implemented, CMS’s proposed hybrid approach would address  our recommendation that it consider placing more emphasis on the  relative performance of homes nationally, which might result in some  states having fewer SFFs and others having more than their current  allocation. We did not recommend that CMS allocate SFFs solely on the  basis of the relative performance of homes nationally, an approach CMS  would disagree with according to its comments.
CMS agreed in principle with our remaining recommendations—intended  to improve the SFF methodology’s ability to identify the most poorly  performing nursing homes and ensure its consistency with the agency’s  Five-Star System—and noted that it would evaluate the effects of adopting  them. The agency explained that there might be technical barriers to fully  implementing our recommendation that it account for a nursing home’s  full compliance history regardless of technical status changes, but noted  that it would implement the recommended adjustment to the maximum  extent practicable. CMS agreed that although this change would affect a  small number of providers, it would improve the accuracy of ratings for  those providers.
CMS also provided technical comments, which we incorporated as  appropriate.
As agreed with your offices, unless you publicly announce the contents of  this report earlier, we plan no further distribution until 30 days from the  report date. At that time, we will send copies to the Administrator of the  Centers for Medicare & Medicaid Services and appropriate congressional  committees. The report will also be available at no charge on the GAO  Web site at http://www.gao.gov.
If you or your staff have any questions about this report, please contact me  at (202) 512-7114 or at dickenj@gao.gov. Contact points for our Offices of  Congressional Relations and Public Affairs may be found on the last page  of this report. GAO staff who made major contributions to this report are  listed in appendix V.
section_title: Appendix I: Scope and Methodology
paragraphs: This appendix provides a more detailed description of our scope and  methodology.
subsections: section_title: Determining the Number and Comparing the Performance of the Most Poorly Performing Nursing Homes
paragraphs: To determine the number of most poorly performing nursing homes in the  nation and compare their performance to that of homes identified using  the Centers for Medicare & Medicaid Services’ (CMS) approach, we began  by interviewing agency officials about the Special Focus Facility (SFF)  Program and methodology and by reviewing documentation related to the  methodology. In addition, we interviewed officials in all 10 CMS regional  offices and 14 state survey agencies regarding their impressions of the SFF  methodology and also asked the state survey agencies what they consider  to be indicators of poor performance. To ensure that we calculated the  scores for each nursing home consistent with CMS’s SFF methodology, we  obtained a copy of the computer programming that CMS used to score and  rank nursing homes, verified that our use of CMS’s program generated  results that were consistent with output on scores that CMS provided to  states, and used the program as the basis for our estimate of the most  poorly performing nursing homes in the United States. The SFF  methodology creates a total score for each nursing home over three cycles  by assigning points to the following data, which we obtained from   CMS’s On-Line Survey, Certification, and Reporting system (OSCAR)  database: (1) deficiencies cited on the three most recent standard surveys,  (2) deficiencies cited on the last 3 years of complaint investigations, and  (3) revisits associated with the three most recent standard surveys.  Additional points are assigned to deficiencies classified as substandard  quality of care (SQC). Each cycle consists of one standard survey, revisits  associated with the standard survey, and 12 months of complaint  investigations. We extracted these data from OSCAR in December 2008.
To learn about methods used to rate nursing home performance, we  interviewed officials of two nursing home associations—the American  Health Care Association and the American Association of Homes and  Services for the Aging, interviewed experts in long-term care research,  attended meetings that CMS held to seek input from long-term care  researchers on the development of the agency’s Five-Star Quality Rating  System (Five-Star System), analyzed information available at CMS’s  Providing Data Quickly Web site, reviewed prior GAO reports, and  interviewed officials from some states with nursing home rating systems.  We also reviewed documentation describing additional approaches to  rating nursing home performance. Specifically, we reviewed eight nursing  home rating systems, which considered a variety of rating factors.
subsections: section_title: Examination of the SFF Methodology
paragraphs: To determine the adequacy of the SFF methodology, we compared the  methodology to other compliance-based measures of poor performance  and tested the sensitivity of the methodology to variations, such as  weighting.
We compared the SFF methodology to two other compliance-based  measures of poor performance—SQC and immediate sanctions. We found  that those nursing homes with the worst total scores in the nation were  much more likely to have met the criteria for SQC in the last 1, 2, and 3  years compared to all other nursing homes. Similarly, we found that the  same nursing homes were much more likely to have had deficiencies that  could have resulted in the imposition of at least one immediate sanction in  the last 1, 2, and 3 years compared to all other nursing homes.  In addition,  we tested the sensitivity of CMS’s SFF methodology to several variations,  some of which led us to consider making modifications to the  methodology that affected facility scores. For example, in one variation,  we modified the SFF methodology so that the cycle scores were no longer  weighted—CMS began weighting the cycle scores in June 2008. We  concluded from this test that the SFF methodology was sensitive to  weighting, which influenced our decision to impose a requirement in our  scoring thresholds such that the most poorly performing nursing homes  have high scores in at least two of three cycles or a very high score overall.
Based on our examinations of the SFF methodology, document review,  and interviews, we concluded that the SFF methodology is reasonable and  comprehensive because it uses multiple years of data, includes all  deficiencies as opposed to a subset of deficiencies, includes deficiencies  from standard surveys and complaint investigations, and accounts for the  scope and severity of deficiencies and the number of revisits.  Furthermore, CMS has refined the SFF methodology over time.
section_title: Development of the Nationwide Statistical Scoring Thresholds
paragraphs: Although we concluded that estimating the number of the most poorly  performing nursing homes on a state-by-state rather than on a national  basis would yield inconsistent results, we determined that there was no  natural break point that differentiated the most poorly performing nursing  homes from all other homes. As a result, we investigated several statistical  approaches and determined that Tukey’s method was appropriate because  the distribution of nursing homes’ total scores is highly skewed. Tukey’s  method is meant to identify the extreme ends of the distribution. It labels  an observation as a potential outlier if its value is greater than the  threshold identified by the following equation:  Potential Outlier Threshold = Q3 + 1.5 * (Q3 – Q1)  Where: Q3 = 75th percentile and  Q1 = 25th percentile  The range identified by (Q3 – Q1), called the interquartile range, covers   50 percent of the observations in the center of the distribution. We applied  this method by identifying nursing homes that had scores that were above  the potential outlier threshold.
We then explored several options to identify the most poorly performing  nursing homes using Tukey’s method as a basis. For each option, we  analyzed the group of resulting nursing homes identified as poor  performers and those missed by the thresholds. As the result of our  examination of the SFF methodology and prior work in which we  classified nursing homes as low, moderately, or high performing, we knew  that nursing homes that have serious deficiencies in one year may not  demonstrate consistent poor performance—what we term chronic poor  performance in this report. Thus, another option we considered was to  identify as a poor performer any nursing home that had a total score that  (1) was above the potential outlier threshold and (2) was also above the  potential outlier threshold for at least two of its three cycle scores. This  option identified 507 nursing homes. Because these nursing homes had  poor performance in at least two of three cycles as well as high total  scores, we concluded that this threshold identified chronic poor  performance. However, we found that when we limited the most poorly  performing nursing homes to this group of chronic poor performers we  missed some nursing homes with very poor performance that was not  chronic. Therefore, we established a second threshold to identify those  very poor performers—those nursing homes that were at or above the   99th percentile—or approximately 330—of total score. This threshold  added another 73 nursing homes.
section_title: Determining the Characteristics of the Most Poorly Performing Nursing Homes
paragraphs: To determine the characteristics of the most poorly performing nursing  homes that distinguish them from all other nursing homes, we analyzed  deficiencies and revisits from the three most recent cycles—that is, the  three most recent standard surveys as of the date of our data extract  (December 17, 2008) and any associated revisits, as well as deficiencies  cited on complaint investigations conducted 3 years before our data  extract. We also analyzed other data that describe the characteristics of  nursing homes: a December 17, 2008, extract of other OSCAR variables;  case-mix-adjusted nurse staffing hours available from CMS’s Five-Star  System, which were dated November 2008; and a list developed by CMS of  nursing homes whose deficiency histories could have subjected them to  immediate sanctions, which we obtained from CMS in October 2008.
Following are highlights of how we analyzed certain characteristics:    We calculated the number of nursing homes in each fiscal year that had  deficiencies that could have resulted in the imposition of at least one  immediate sanction.
Nursing homes self-report their ownership type. We created the ownership  type of for-profit by combining three categories of for-profit nursing  homes designated in CMS’s data (individual, partnership, and corporation)  and the category of limited liability corporation. Similarly, we created the  ownership type of nonprofit by combining three categories of nonprofit  nursing homes (corporation, church related, and other), and the  ownership type of government from the six designations made in CMS  data (state, county, city, city/county, hospital district, and federal).
CMS maintains a variable in its data called multi–nursing home (chain)  ownership, which is self-reported by nursing homes and which we refer to  as chain affiliation. According to CMS, multi–nursing home chains have  two or more homes under one ownership or operation. We determined the  percentage of nursing homes that were for-profit and chain affiliated,  nonprofit and chain affiliated, or government owned and chain affiliated  by combining the ownership type described above with CMS’s designation  of multi–nursing home (chain) ownership.
We used the number of beds certified for payment for Medicare, Medicaid,  or both to calculate the following: the average number of beds per nursing  home and the percentage of nursing homes by bed size category (0 to 49,  50 to 99, 100 to 199, and more than 199 beds).
We calculated the percentage share of residents by resident type  (Medicare, Medicaid, or other) by dividing the number of Medicare,  Medicaid, and other patients by the number of total residents.
We calculated the occupancy rate by dividing the total number of residents  by the number of certified beds. We used certified beds to calculate the  occupancy rate instead of total beds because CMS officials told us that  certified beds provided more reliable information.
We analyzed the following nurse staffing hours, which were case-mix  adjusted by CMS for use in its Five-Star System: registered nurse hours per  resident per day, licensed practical nurse and vocational nurse hours per  resident per day, nurse aide hours per resident per day, and total staffing  hours per resident per day. We calculated resident nurse hours as a share  of the total. Unadjusted nurse staffing hours data are collected by CMS,  self-reported by nursing homes, and represent staffing levels for a 2-week  period before the state inspection. CMS case-mix adjusted the staffing data  using the average minutes of nursing care used to care for residents in a  given resource utilization group category as reflected in the Medicare  skilled nursing facility prospective payment system. CMS acknowledges  that the staff hours collected from nursing homes have certain limitations.  In order to increase the accuracy and comprehensiveness of the staffing  data, CMS has been investigating whether it can use nursing home payroll  data to report staffing levels on the Nursing Home Compare Web site.
section_title: Appendix II: Detailed Compliance History for Two of the Most Poorly Performing Nursing Homes, as of December 2008
paragraphs: The following table provides the detailed compliance history over three  cycles for two of the most poorly performing homes in the nation.
section_title: Appendix III: Performance by Standard Area for the Most Poorly Performing and All Other Nursing Homes, as of December 2008
paragraphs: The following table provides the percentages of the most poorly  performing and all other nursing homes that were cited for actual harm or  immediate jeopardy by standards area over three cycles.
section_title: Appendix IV: Comments from the Centers for Medicare & Medicaid Services
section_title: Appendix V: GAO Contact and Staff Acknowledgments
paragraphs: John E. Dicken, (202) 512-7114 or dickenj@gao.gov.
subsections: section_title: Acknowledgments
paragraphs: In addition to the contact named above, Walter Ochinko, Assistant  Director; Ramsey Asaly; Daniel Lee; Shannon Slawter Legeer; Jessica  Morris; Jessica Nysenbaum; Dae Park; Roseanne Price; Jennifer Rellick;  Kathryn Richter; and Jessica Smith made key contributions to this report.
section_title: Related GAO Products
paragraphs: Medicare and Medicaid Participating Facilities: CMS Needs to  Reexamine Its Approach for Funding State Oversight of Health Care  Facilities. GAO-09-64. Washington, D.C.: February 13, 2009.
Nursing Homes: Federal Monitoring Surveys Demonstrate Continued  Understatement of Serious Care Problems and CMS Oversight  Weaknesses. GAO-08-517. Washington, D.C.: May 9, 2008.
Nursing Home Reform: Continued Attention Is Needed to Improve  Quality of Care in Small but Significant Share of Homes. GAO-07-794T.  Washington, D.C.: May 2, 2007.
Nursing Homes: Efforts to Strengthen Federal Enforcement Have Not  Deterred Some Homes from Repeatedly Harming Residents. GAO-07-241.  Washington, D.C.: March 26, 2007.
Nursing Homes: Despite Increased Oversight, Challenges Remain in  Ensuring High-Quality Care and Resident Safety. GAO-06-117.  Washington, D.C.: December 28, 2005.
Nursing Home Quality: Prevalence of Serious Problems, While  Declining, Reinforces Importance of Enhanced Oversight. GAO-03-561.  Washington, D.C.: July 15, 2003.
Nursing Homes: Public Reporting of Quality Indicators Has Merit, but  National Implementation Is Premature. GAO-03-187. Washington, D.C.:  October 31, 2002.
Nursing Homes: Federal Efforts to Monitor Resident Assessment Data  Should Complement State Activities. GAO-02-279. Washington, D.C.:  February 15, 2002.
Nursing Homes: Sustained Efforts Are Essential to Realize Potential   of the Quality Initiatives. GAO/HEHS-00-197. Washington, D.C.:  September 28, 2000.
Nursing Home Care: Enhanced HCFA Oversight of State Programs  Would Better Ensure Quality. GAO/HEHS-00-6. Washington, D.C.:  November 4, 1999.
Nursing Home Oversight: Industry Examples Do Not Demonstrate That  Regulatory Actions Were Unreasonable. GAO/HEHS-99-154R. Washington,  D.C.: August 13, 1999.
Nursing Homes: Proposal to Enhance Oversight of Poorly Performing  Homes Has Merit. GAO/HEHS-99-157. Washington, D.C.: June 30, 1999.
Nursing Homes: Complaint Investigation Processes Often Inadequate to  Protect Residents. GAO/HEHS-99-80. Washington, D.C.: March 22, 1999.
Nursing Homes: Additional Steps Needed to Strengthen Enforcement   of Federal Quality Standards. GAO/HEHS-99-46. Washington, D.C.:   March 18, 1999.
California Nursing Homes: Care Problems Persist Despite Federal and  State Oversight. GAO/HEHS-98-202. Washington, D.C.: July 27, 1998.