GAO 

United States Government Accountability Office

Report to Congressional Requesters

NURSING HOMES

CMS’s Special Focus 

Facility Methodology 

Should Better Target 

the Most Poorly 

Performing Homes, 

Which Tended to Be 

Chain Affiliated and 

For-Profit 

August 2009 

GAO-09-689 
What GAO Found

United States Government Accountability Office

Why GAO Did This Study

Highlights

Accountability Integrity Reliability

August 2009

NURSING HOMES

CMS’s Special Focus Facility Methodology Should 

Better Target the Most Poorly Performing Homes, 

Highlights of GAO-09-689, a report to Which Tended to Be Chain Affiliated and For-Profit 

congressional requesters 

In 1998, CMS established the 

Special Focus Facility (SFF) 

Program as one way to address 

poor performance by nursing 

homes. The SFF methodology 

assigns points to deficiencies cited 

on standard surveys and complaint 

investigations, and to revisits 

conducted to ensure that 

deficiencies have been corrected. 

CMS uses its methodology 

periodically to identify candidates 

for the program—nursing homes 

with the 15 worst scores in each 

state—but the program is limited to 

136 homes at any point in time 

because of resource constraints. In 

2008, CMS introduced a Five-Star 

Quality Rating System that draws 

on the SFF methodology to rank 

homes from one to five stars. GAO 

assessed CMS’s SFF methodology, 

applied it on a nationwide basis 

using statistical scoring thresholds, 

and adopted several refinements to 

the methodology. Using this 

approach, GAO determined (1) the 

number of most poorly performing 

homes nationwide, (2) how their 

performance compared to that of 

homes identified using the SFF 

methodology, and (3) the 

characteristics of such homes. 

What GAO Recommends

GAO is recommending that the 

CMS Administrator consider a 

home’s relative performance 

nationally when allocating SFFs 

across states and take actions to 

refine the SFF methodology to

improve the identification of SFFs. 

CMS generally agreed in principle 

with our recommendations and 

said that it would evaluate the 

effects of adopting them. 

According to GAO’s estimate, almost 4 percent (580) of the roughly 16,000 

nursing homes in the United States could be considered the most poorly 

performing. These 580 homes overlap somewhat with the 755 SFF Program 

candidates—the 15 worst homes in each state—and the 136 homes actually 

selected by states as SFFs. For example, GAO’s estimate includes 40 percent 

of SFF Program candidates and about half of the active SFFs as of December 

2008 and February 2009, respectively. Under GAO’s estimate, however, the 

most poorly performing homes are distributed unevenly across states, with 8 

states having no such homes and 10 others having from 21 to 52 such homes. 

CMS has structured the SFF Program so that every state (except Alaska) has 

at least one SFF even though the worst performing homes in each state are 

not necessarily the worst performing homes in the nation. To identify the 

worst homes in the nation, GAO applied CMS’s SFF methodology on a 

nationwide basis using statistical scoring thresholds and made three 

refinements to that methodology, which strengthened GAO’s estimate. The 

scoring thresholds were (1) necessary because there were no natural break 

points that delineated the most poorly performing homes from all other 

nursing homes and (2) conservative, focusing on chronic poor performance 

generally over a 2- or 3-year period or very poor performance over about 

1 year. The most poorly performing homes identified by GAO averaged over 

46 percent more serious deficiencies that caused harm to residents and over 

19 percent more deficiencies that placed residents at risk of death or serious 

injury (immediate jeopardy), compared to the 755 SFF Program candidates 

identified by CMS’s approach. GAO’s three refinements to CMS’s SFF 

methodology had a moderate effect on the composition of the list of homes 

that GAO identified as the most poorly performing. First, deficiency points 

from CMS’s Five-Star Quality Rating System were used because they 

decreased the disparity between immediate jeopardy and lower-level 

deficiencies, such as those with the potential for more than minimal harm, 

which compensates somewhat for the understatement of serious deficiencies 

in some states. Second, homes received extra points when certain actual harm 

deficiencies occurred in standards areas that CMS categorizes as substandard 

quality of care, an important change because we found that many homes had 

at least one such deficiency. Third, the full deficiency history of homes was 

included. CMS recognizes that its methodology overlooks deficiencies for 

some homes, which almost always results in scores that are lower than if all 

deficiencies were included in the scores. 

GAO found that the most poorly performing nursing homes had notably more 

deficiencies with the potential for more than minimal harm or higher and 

more revisits than all other nursing homes. For example, the most poorly 

performing nursing homes averaged about 56 such deficiencies and 2 revisits, 

compared to about 20 such deficiencies and less than 1 revisit for all other 

homes. In addition, the most poorly performing homes tended to be chain 

affiliated and for-profit and have more beds and residents. 

View GAO-09-689 or key components.

For more information, contact John Dicken at 

(202) 512-7114 or dickenj@gao.gov. 
Page i GAO-09-689 

Contents 

Letter 1

Background 4 

Five Hundred Eighty Nursing Homes Could Be Considered the 

Most Poorly Performing—Fewer Than CMS’s SFF Program 

Candidates but More Than the Number of SFFs 13 

CMS’s Application of the SFF Methodology Misses Many of the 

Nation’s Most Poorly Performing Nursing Homes 16 

Key Characteristics, such as Chain Affiliation and For-Profit Status 

Differentiated the Most Poorly Performing Nursing Homes 24 

Conclusions 31 

Recommendations for Executive Action 32 

Agency Comments and Our Evaluation 33 

Appendix I Scope and Methodology 35 

Appendix II Detailed Compliance History for Two of the 

Most Poorly Performing Nursing Homes, as of 

December 2008 41 

Appendix III Performance by Standard Area for the Most 

Poorly Performing and All Other Nursing Homes, 

as of December 2008 44 

Appendix IV Comments from the Centers for Medicare & 

Medicaid Services 45 

Appendix V GAO Contact and Staff Acknowledgments 49 

Related GAO Products 50 

The Most Poorly Performing Nursing Homes 
Tables 

Table 1: Scope and Severity of Deficiencies Identified during 

Nursing Home Surveys 6 

Table 2: Comparison of Points Assigned to Deficiencies in the SFF 

Methodology and in the Five-Star System 10 

Table 3: Compliance History over the Last Three Cycles for Our 

Estimate of the Most Poorly Performing Nursing Homes 

Compared to the SFF Program Candidates, as of 

December 2008 17 

Table 4: Compliance Histories over the Last Three Cycles for Two 

of the Most Poorly Performing Nursing Homes, as of 

December 2008 21 

Table 5: Compliance History over Last Three Cycles for the Most 

Poorly Performing Nursing Homes and All Other Nursing 

Homes, as of December 2008 25 

Table 6: The 10 Standards Most Often Cited at the Immediate 

Jeopardy Level in the Last Three Cycles among the Most 

Poorly Performing Homes Compared to All Other Nursing 

Homes, as of December 2008 26 

Table 7: Distribution of the Most Poorly Performing Nursing 

Homes and All Other Nursing Homes by Type of 

Organization, as of December 2008 28 

Table 8: Distribution of the Most Poorly Performing Nursing 

Homes and All Other Nursing Homes by Medicare and 

Medicaid Participation, as of December 2008 29 

Table 9: Distribution of the Most Poorly Performing Nursing 

Homes and All Other Nursing Homes by Beds and 

Residents, as of December 2008 30 

Table 10: Distribution of the Most Poorly Performing Nursing 

Homes and All Other Nursing Homes by Case-MixAdjusted Nurse Staffing Levels, as of November 2008 31 

Table 11: Detailed Compliance History over Three Cycles for Two 

of the Most Poorly Performing Nursing Homes in the 

Nation, as of December 2008 41 

Table 12: Percentage of the Most Poorly Performing and All Other 

Nursing Homes Cited for Actual Harm or Immediate 

Jeopardy, by Standards Area, in the Last Three Cycles as 

of December 2008 44 

Page ii GAO-09-689 The Most Poorly Performing Nursing Homes
Figures 

Figure 1: How CMS and States Operate the Special Focus Facility 

Program 9 

Figure 2: Overlap between Our Estimate of the Most Poorly 

Performing Nursing Homes in the Nation and the SFF 

Program Candidates, as of December 2008 14 

Figure 3: Our Estimate of the Number of Most Poorly Performing 

Nursing Homes as of December 2008 Compared to the 

Number of SFFs by State 15 

Figure 4: Total Score Ranges for the SFF Program Candidates, as 

of December 2008 19 

Abbreviations 

CMS Centers for Medicare & Medicaid Services 

LPN/LVN licensed practical nurse/licensed vocational nurse 

OSCAR On-Line Survey, Certification, and Reporting system 

SFF Special Focus Facility 

SQC substandard quality of care 

This is a work of the U.S. government and is not subject to copyright protection in the 

United States. The published product may be reproduced and distributed in its entirety 

without further permission from GAO. However, because this work may contain 

copyrighted images or other material, permission from the copyright holder may be 

necessary if you wish to reproduce this material separately. 

Page iii GAO-09-689 The Most Poorly Performing Nursing Homes
Page 1 GAO-09-689 

 

United States Government Accountability Office

Washington, DC 20548 

 

August 28, 2009 

The Honorable Herb Kohl 

Chairman 

Special Committee on Aging 

United States Senate 

The Honorable Charles E. Grassley 

Ranking Member 

Committee on Finance 

United States Senate 

The nation’s 1.4 million nursing home residents are a highly vulnerable 

population of elderly and disabled individuals who rely on nursing homes 

to provide high-quality care. The Centers for Medicare & Medicaid 

Services (CMS) contracts with state survey agencies to conduct 

inspections, known as standard surveys, and complaint investigations to 

determine whether the nation’s roughly 16,000 nursing homes are 

complying with federal quality standards. Nursing homes must meet those 

standards in order to participate in Medicare and Medicaid.1

 Our prior 

reports have found that some nursing homes are chronically 

noncompliant; that is, they have been cited repeatedly by state survey 

agencies for serious deficiencies, such as residents having preventable 

pressure sores that harmed them or put them at risk of death or serious 

injury.2

 In 1998, CMS established the Special Focus Facility (SFF) Program 

as one way to address poor performance by nursing homes. 

CMS uses a formula—the SFF methodology—to score the relative 

performance of nursing homes and identify the 15 poorest performing 

homes in each state as candidates for the SFF Program. State survey 

agencies then work with CMS to choose some of those candidates to 

1

Medicare is the federal health care program for elderly and disabled individuals. Medicaid 

is the joint federal-state health care financing program for certain categories of low-income 

individuals. Combined Medicare and Medicaid payments for nursing home services were 

about $78 billion in 2007, including a federal share of about $54 billion. 

2

Such nursing homes often have a pattern of cycling in and out of compliance. See 

GAO, Nursing Homes: Additional Steps Needed to Strengthen Enforcement of Federal 

Quality Standards, GAO/HEHS-99-46 (Washington, D.C.: Mar. 18, 1999), and Nursing 

Homes: Proposal to Enhance Oversight of Poorly Performing Homes Has Merit, 

GAO/HEHS-99-157 (Washington, D.C.: June 30, 1999). 

The Most Poorly Performing Nursing Homes 
 

participate in the program; those selected receive more intensive 

oversight, including more frequent surveys. CMS limits the SFF Program to 

136 nursing homes nationwide (fewer than 1 percent of nursing homes) at 

any point in time because of resource constraints.3

 Our prior reports have 

demonstrated that not all chronically noncompliant nursing homes are 

included in this program and that the number of poorly performing nursing 

homes nationwide is therefore greater than 136.4

You expressed interest in CMS’s efforts to influence the performance of 

poorly performing nursing homes. In this report, we (1) determined the 

number of nursing homes in the United States that can be identified as the 

most poorly performing, (2) analyzed how those homes’ performance 

compared to that of nursing homes identified using CMS’s SFF 

methodology, and (3) assessed the characteristics of the most poorly 

performing nursing homes that distinguish them from all other nursing 

homes. You also asked us to examine the operation of the SFF Program, 

including its effect on the performance of homes selected as SFFs. We will 

address this portion of your request in a subsequent report. 

To determine the number of poorly performing nursing homes in the 

nation and how the performance of such homes compared to that of 

homes identified using CMS’s SFF methodology, we began by interviewing 

CMS officials, reviewing documentation related to the agency’s SFF 

methodology, obtaining a copy of the computer program CMS used to 

score and rank nursing homes, and analyzing data on deficiencies and 

revisits from CMS’s On-Line Survey, Certification, and Reporting system 

(OSCAR) database. The SFF methodology creates a total score for each 

nursing home over three cycles by assigning points to (1) deficiencies 

cited during the three most recent standard surveys, (2) deficiencies cited 

on the last 3 years of complaint investigations, and (3) the number of 

revisits surveyors made to ensure that the nursing home had corrected the 

 

3

In a 2007 report, we recommended that the Administrator of CMS consider further 

expanding the SFF Program to include all homes that met a certain threshold established 

by CMS to qualify as poorly performing. CMS agreed with the concept of expanding the 

program, but indicated that it lacked the resources needed to expand the number of 

surveys. See GAO, Nursing Homes: Efforts to Strengthen Federal Enforcement Have Not 

Deterred Some Homes from Repeatedly Harming Residents, GAO-07-241 (Washington, 

D.C.: Mar. 26, 2007). 

4

See GAO-07-241, and GAO, Nursing Homes: Federal Actions Needed to Improve 

Targeting and Evaluation of Assistance by Quality Improvement Organizations, 

GAO-07-373 (Washington, D.C.: May 29, 2007). 

Page 2 GAO-09-689 The Most Poorly Performing Nursing Homes
 

deficiencies cited on the three most recent standard surveys.5 To 

determine the adequacy of the SFF methodology, we compared it to other 

compliance-based measures of poor performance and tested the sensitivity 

of the methodology to variations such as not giving greater weight to 

recent poor performance compared to poor performance that occurred in 

earlier years. Based on our examinations of the SFF methodology, 

document review, and interviews, we concluded that the SFF methodology 

is reasonable and comprehensive. Using our assessment of the SFF 

methodology, we developed our estimate of the most poorly performing 

nursing homes by (1) applying the SFF methodology on a nationwide basis 

using statistical scoring thresholds and (2) adopting three refinements that 

helped to identify some homes that would otherwise have been missed but 

that had a moderate impact on the composition of the list of homes we 

identified as the most poorly performing. We estimated the number of 

most poorly performing nursing homes in the nation and the number of 

homes identified using CMS’s SFF methodology—which we refer to as 

SFF Program candidates—by analyzing OSCAR data from a particular 

point in time (December 2008).6

 We also analyzed a list obtained from 

CMS of SFFs that participated in the program from January 2005 thro

February 2009 to determine the number of poorly performing nursing 

homes that were also SFFs. 

ugh 

 

To determine the characteristics of the most poorly performing nursing 

homes that distinguish them from all other nursing homes, we analyzed 

CMS data as of December 2008 on deficiencies, revisits, and other 

information that describe nursing home characteristics.7

 We also analyzed 

case-mix-adjusted nurse staffing hours available from CMS’s Five-Star 

Quality Rating System (Five-Star System), which were dated November 

5

Each cycle consists of a standard survey, which occurs roughly annually, revisits 

associated with the standard survey, and 12 months of complaint investigations. Multiple 

revisits are an indicator of more serious problems in achieving or sustaining compliance. 

6

OSCAR data change continually as new surveys are conducted and entered into the 

database, but there can be a lag time. As a result, the data we analyzed did not necessarily 

include all surveys conducted through December 2008. The number and composition of our 

estimates of the most poorly performing nursing homes and the SFF Program candidates 

will change over time as new surveys and revisits are conducted and considered. 

7

One characteristic we analyzed, “multi–nursing home (chain) ownership,” is self-reported 

by nursing homes. According to CMS, multi–nursing home chains have two or more homes 

under one ownership or operation. In this report, we refer to nursing homes that have 

indicated that they operate under multi–nursing home (chain) ownership as having chain 

affiliation. 

Page 3 GAO-09-689 The Most Poorly Performing Nursing Homes
 

2008, and data related to certain enforcement actions obtained from CMS 

in October 2008. 

Our work was also informed by analyzing information available at CMS’s 

Providing Data Quickly Web site; reviewing our prior reports; interviewing 

experts in long-term care research; interviewing officials from CMS’s 

central office, all 10 CMS regions, and 14 state survey agencies; and 

reviewing some states’ approaches to rating nursing home quality. For a 

more detailed discussion of our scope and methodology, see appendix I. 

To ensure the reliability of the OSCAR deficiency and revisit data for our 

purposes as well as the reliability of the data we analyzed to determine the 

characteristics of the most poorly performing nursing homes compared to 

those of all other nursing homes, we interviewed CMS officials, reviewed 

CMS documentation, conducted electronic testing to identify obvious 

errors, and traced a selection of records to another CMS reporting system. 

Based on these activities, we determined that the data we analyzed were 

sufficiently reliable for our purposes. 

We conducted this performance audit from February 2008 through August 

2009 in accordance with generally accepted government auditing 

standards. Those standards require that we plan and perform the audit to 

obtain sufficient, appropriate evidence to provide a reasonable basis for 

our findings and conclusions based on our audit objectives. We believe 

that the evidence obtained provides a reasonable basis for our findings 

and conclusions based on our audit objectives. 

Titles XVIII and XIX of the Social Security Act establish minimum 

requirements that all nursing homes must meet to participate in the 

Medicare and Medicaid programs, respectively. The Omnibus Budget 

Reconciliation Act of 1987 focused the requirements on the quality of care 

actually provided by a home.8

 To help beneficiaries make informed 

decisions when selecting or evaluating nursing homes, CMS increased the 

amount of information publicly available on its Nursing Home Compare 

Web site in 2008 by rating the quality of each nursing home on a five-level 

scale.9

Background 

 

8

Pub. L. No. 100-203, §§ 4201, 4211, 101 Stat. 1330, 1330-160, 1330-182 (codified in pertinent 

part at 42 U.S.C. § 1395i-3 and 42 U.S.C. § 1396r). 

9

Nursing Home Compare is available at http://www.medicare.gov/NHCompare. 

Page 4 GAO-09-689 The Most Poorly Performing Nursing Homes
 

To assess whether nursing homes meet federal quality standards, state 

survey agencies conduct standard surveys, which occur roughly once per 

year, and complaint investigations.10 A standard survey involves a 

comprehensive assessment of quality standards.11 In contrast, complaint 

investigations generally focus on a specific allegation regarding resident 

care or safety made by a resident, family member, or nursing home staff. 

Standard Surveys and 

Complaint Investigations 

Federal quality standards focus on the delivery of care, resident outcomes, 

and facility conditions. These quality standards, totaling approximately 

200, are grouped into 15 categories, such as Resident Rights, Quality of 

Care, Quality of Life, and Resident Behavior and Facility Practices.12

Nursing homes that meet these quality standards can be certified to 

participate in Medicare, Medicaid, or both programs. Homes may 

occasionally change their participation type, or, according to CMS, states 

may require nursing homes to change their participation type. We refer to 

this type of change that results in a new provider identification number as 

a “technical status change.” Such a change may affect the source of 

payment—Medicare or Medicaid—that the nursing home is eligible to 

receive. When a technical status change occurs, CMS’s SFF methodology 

as applied does not incorporate the nursing home’s complete survey 

history. 

Survey Deficiencies States classify deficiencies identified during either standard surveys or 

complaint investigations in 1 of 12 categories according to their scope 

(i.e., the number of residents potentially or actually affected) and severity 

(i.e., the potential for or occurrence of harm to residents).13 (See table 1.) 

 

10Every nursing home receiving Medicare or Medicaid payment must undergo a standard 

survey not less than once every 15 months, and the statewide average interval for these 

surveys must not exceed 12 months. See 42 U.S.C. § 1395i-3(g)(2)(A)(iii); 42 U.S.C. 

§ 1396r(g)(2)(A)(iii). CMS generally interprets these requirements to permit a statewide 

average interval of 12.9 months and a maximum interval of 15.9 months for each home. 

11The standard survey also includes an assessment of federal fire safety standards. The fire 

safety portion of a standard survey is not always conducted concurrently with the 

assessment of other standards. 

12Other areas include Admission, Transfer and Discharge Rights; Resident Assessment; 

Pharmacy Services; Administration; Nursing Services; Dietary Services; Physician Services; 

Specialized Rehabilitative Services; Dental Services; Infection Control; and Physical 

Environment. 

13In this report, we use the term states, including the District of Columbia, to refer to state 

survey agencies. 

Page 5 GAO-09-689 The Most Poorly Performing Nursing Homes
 

An A-level deficiency is the least serious and is isolated in scope, while an 

L-level deficiency is the most serious and is widespread throughout the 

nursing home. Nursing homes with deficiencies at the A, B, or C levels are 

considered to be in substantial compliance with quality standards, 

whereas nursing homes with D-level through L-level deficiencies are 

considered noncompliant. For most deficiencies, a home is required to 

prepare a plan of correction, and depending on the severity of the 

deficiency, surveyors may conduct a revisit to ensure that the nursing 

home has implemented its plan and corrected the deficiency. Revisits are 

not required for most deficiencies below the actual harm level—A through 

F.14

Table 1: Scope and Severity of Deficiencies Identified during Nursing Home 

Surveys 

Scope 

Severity Isolated Pattern Widespread 

Immediate jeopardya J K L 

Actual harm G H I 

Potential for more than minimal harm D E F 

Potential for minimal harmbA B C 

Source: CMS. 

a

Actual or potential for death/serious injury. 

b

Nursing home is considered to be in “substantial compliance.” 

As we reported in May 2008, there can be considerable variation among 

states in the proportion of nursing homes cited for deficiencies at the G 

through L levels.15 We concluded that this interstate variation suggests that 

surveyors in some states are missing some serious deficiencies or 

understating their scope and severity. We provided examples of such 

understatement in our May 2008 report. Specifically, we reported that 

during fiscal years 2002 through 2007, about 15 percent of federal 

comparative surveys nationwide found that the state surveys had failed to 

cite at least one deficiency at the most serious levels of noncompliance (G 

 

14A revisit is required for deficiencies at the G through L levels as well as certain F-level 

deficiencies. 

15See GAO, Nursing Homes: Federal Monitoring Surveys Demonstrate Continued 

Understatement of Serious Care Problems and CMS Oversight Weaknesses, GAO-08-517 

(Washington, D.C.: May 9, 2008). 

Page 6 GAO-09-689 The Most Poorly Performing Nursing Homes
 

through L levels) and about 70 percent of them found that the state 

surveys had failed to cite at least one deficiency at the potential for more 

than minimal harm level (D through F levels).16

Enforcement Actions When deficiencies are cited, federal enforcement actions known as 

sanctions can be imposed to encourage homes to make corrections. 

Sanctions are generally reserved for serious deficiencies—those at the G 

through L levels—that constitute actual harm and immediate jeopardy to 

residents.17 Sanctions include fines known as civil money penalties, denial 

of payment for new Medicare or Medicaid admissions, and termination 

from the Medicare and Medicaid programs. Such sanctions can affect a 

home’s revenues and therefore provide financial incentives to return to 

and maintain compliance. CMS requires states to refer for immediate 

sanction homes that receive at least one G- through L-level deficiency on 

successive standard surveys or intervening complaint investigations.18 In 

addition, a nursing home with one or more deficiencies at the F through L 

level—but not G level—in Quality of Care, Quality of Life, or Resident 

Behavior and Facility Practices must be cited for substandard quality of 

care (SQC), which generally results in the home’s losing its approval to 

hold in-house or facility-sponsored nurse aide training. 

CMS Efforts That Identify 

Poorly Performing Nursing 

Homes 

Two of CMS’s efforts that identify poorly performing nursing homes are 

the SFF Program and the Five-Star System. CMS’s Nursing Home Compare 

Web site identifies nursing homes that are in the SFF Program, provides a 

rating of from one to five stars, and also includes data on deficiencies cited 

during standard surveys and complaint investigations, selected quality of 

 

16In a federal comparative survey, federal surveyors independently evaluate state surveys 

by resurveying a home recently inspected by state surveyors and comparing the 

deficiencies identified during the two surveys. In our May 2008 report, we analyzed the 

results of 976 comparative surveys conducted by federal surveyors from fiscal years 2002 

through 2007. (See GAO-08-517.) 

17The scope and severity of a deficiency is one of the factors that CMS may take into 

account when imposing sanctions. CMS may also consider a home’s compliance history, 

desired corrective action and long-term compliance, and the number and severity of all the 

home’s deficiencies. 

18See GAO-07-241. 

Page 7 GAO-09-689 The Most Poorly Performing Nursing Homes
 

care measures, and nurse staffing hours.19 Both the SFF Program and the 

Five-Star System score nursing homes by assigning points to deficiencies 

and the number of revisits, but the points assigned to certain deficiencies 

differ. 

CMS compiles a list of SFF candidates for each state generally on a 

quarterly basis by using the numeric score generated by its SFF 

methodology. The SFF candidates are those nursing homes with the 15 

highest total scores in each state. From the candidate list, state officials 

select, with CMS concurrence, nursing homes they think should 

participate in the program based on their knowledge of the candidates’ 

circumstances. With the exception of Alaska, each state has between one 

and six SFFs, depending on the number of nursing homes in the state.20

CMS requires states to survey SFFs twice as frequently as other nursing 

homes to help motivate SFFs to improve. If an SFF meets CMS’s criteria 

for improved performance, CMS removes the SFF designation and the 

nursing home “graduates” from the program.21 According to CMS guidance 

to states, SFFs that fail to significantly improve after three standard 

surveys, or about 18 months, may be involuntarily terminated from 

Medicare and Medicaid. Nursing homes may also choose to terminate from 

Medicare and Medicaid voluntarily. (See fig. 1.) 

The Special Focus Facility 

Program 

 

19In addition to deficiencies cited during standard surveys and complaint investigations, 

quality of care measures are also indicators of nursing home quality. Nursing homes 

participating in Medicare and Medicaid are required to submit clinical assessment data—

known as the Minimum Data Set—on all of their residents. Data are collected on the 

residents’ health, physical functioning, mental status, and general well-being and are used 

to compute quality of care measures, which indicate potential problem areas that need 

further review and investigation. 

20Alaska does not have any SFFs because it has few nursing homes—only 15 in fiscal year 

2008. 

21The criteria for an SFF to graduate are two consecutive standard surveys and no 

intervening complaint investigations with deficiencies higher than an E level. In addition, 

an SFF may not have a deficiency at the G level or higher on the fire safety portion of its 

most recent standard survey. 

Page 8 GAO-09-689 The Most Poorly Performing Nursing Homes
 

Figure 1: How CMS and States Operate the Special Focus Facility Program 

Sources: GAO analysis of CMS’s SFF program (data); Art Explosion (images).

Methodology:

Scores are calculated for 

all nursing homes using the 

SFF methodology.

Candidates:

CMS identifies about 750 

SFF candidates nationwide, 

which are the nursing homes

that have the 15 highest 

total scores in each state.

SFFs:

As SFFs leave the program, 

states, with CMS concurrence, 

select SFFs – for a total of 136 

– based on their knowledge of 

the candidates’ circumstances.

Increased scrutiny:

SFFs receive increased 

scrutiny, including surveys

twice as frequently as other 

nursing homes.

Graduation or termination:

SFFs that meet CMS’s criteria

“graduate” from the program. 

SFFs that fail to significantly 

improve after three standard 

surveys, or approximately 18 

months, may be involuntarily 

terminated from Medicare 

and Medicaid.

Scores for all

nursing homes

State A

SFFs State A

NonSFFs

About 750

SFF candidates

136

SFFs

SFFs

graduate

SFF

SFF

surveys

Note: With 15 SFF candidates per state, including the District of Columbia, but excluding Alaska 

(because Alaska does not have SFFs), there are a total of 750 SFF candidates. There can be more 

than 750 SFF candidates if nursing homes in the same state have the same total score. 

The SFF methodology assigns points to deficiencies on standard surveys 

and complaint investigations, and to revisits associated with deficiencies 

cited on standard surveys, as follows: 

• Deficiencies. More points are assigned to deficiencies that are higher in 

scope and severity.22 Additional points are assigned to deficiencies 

classified as SQC. For example, a nursing home with one J-level deficiency 

in the Quality of Care category would be assigned 75 points (50 points plus 

an additional 25 points because the deficiency was SQC). See table 2 for a 

comparison of the deficiency points assigned by the SFF methodology and 

the Five-Star System. 

 

22To avoid potential double-counting, deficiencies that appear on complaint surveys that 

are conducted within 15 days of a standard survey (either before or after the standard 

survey) are only counted once. If the scope or severity differs on the two surveys, the 

highest scope-severity combination is used. 

Page 9 GAO-09-689 The Most Poorly Performing Nursing Homes
 

• Revisits. Multiple revisits are an indicator of more serious problems in 

achieving or sustaining compliance. The points for revisits are as follows: 

0 for the first revisit, 50 for the second revisit, an additional 75 (total 125) 

for the third revisit, and an additional 100 (total 225) for the fourth 

revisit.23

Table 2: Comparison of Points Assigned to Deficiencies in the SFF Methodology and in the Five-Star System 

Scope an severity d

inim harm than minimal harm ctu arm Immediate jeopardy 

Potential for 

m al 

Potential for more 

A al h

Methodology A B C D E F G H I J K L 

SFF points 0 0 0 2 4 6 10 20 30 50 100 150 

Five-Star System points 0 0 0 4 8 16 20 35 45 50 100 150 

Additional SQC points 0 0 0 0 0 4 0 5 5 25 25 25 

Source: CMS. 

is 

res and nursing homes that made improvements have lower total 

scores. 

MS has changed its scope and 

methodology several times. For example: 

ed 

er of points to deficiencies using only about 1 year of 

eficiency data. 

 

For each nursing home, CMS sums the points associated with the 

deficiencies (including SQC) and the revisits to create a cycle score for 

each of the last three cycles. CMS then creates the total score by weighting 

the more recent cycle scores more heavily.24 The effect of this weighting 

that nursing homes that had more recent poor performance have higher 

total sco

Since the inception of the SFF Program, C

• From 1999 to 2004, each state had two SFFs at any one time, which they 

selected from a list of four candidates, and the SFF methodology assign

a different numb

d

23The SFF methodology does not assign points to revisits associated with deficiencies cited 

on complaint investigations. 

24The most recent score is assigned a weighting factor of one-half, the second most recent 

score is assigned a weighting factor of one-third, and the third most recent score (from the 

earliest period) is assigned a weighting factor of one-sixth. 

Page 10 GAO-09-689 The Most Poorly Performing Nursing Homes
 

• In 2005, CMS expanded the program’s scope by changing the number of 

SFFs from 1 to 6 per state (excluding Alaska), for a total of 136, and 

altered the SFF methodology by changing the points assigned to 

deficiencies and using about 3 years of deficiency data, weighted equally. 

• In 2007, CMS began requiring states to notify a nursing home and its other 

accountable parties (i.e., the nursing home’s administrator, owners, 

operators, and governing body) when the nursing home was designated as 

an SFF. 

• In 2008, CMS began designating SFFs on the Nursing Home Compare Web 

site and also changed the scoring methodology to assign weights to each 

year, such that the most recent year’s standard and complaint surveys are 

given the greatest weight. 

During the course of our work, CMS implemented its Five-Star System for 

nursing homes. Every nursing home in the United States is rated from one 

(much below average) to five (much above average) stars.25

The Five-Star Quality Rating 

System 

The Five-Star System provides an overall quality rating based on individual 

ratings for three separate components: (1) assessment of federal quality 

standards from standard surveys and complaint investigations, which CMS 

refers to in the Five-Star System as health inspections; (2) ratings on 

nursing home staffing levels; and (3) ratings on quality of care measures. 

In December 2008, CMS’s Nursing Home Compare Web site began 

reporting the star ratings that nursing homes receive for each component 

of the Five-Star System as well as an overall quality rating.26 According to 

CMS officials, as of March 2009 the rating for the health inspections 

component was based on CMS’s SFF methodology, with one variation: the 

Five-Star System assigns more points to D- through I-level deficiencies 

 

25A two-star rating means a facility ranks “below average;” a three-star rating means “about 

average;” and a four-star rating means “above average.” 

26To determine the overall quality rating, CMS starts with a nursing home’s rating from the 

health inspections component. One star is then added to the rating for very high 

component ratings or subtracted from the rating for very low component ratings. The 

overall quality rating is capped in two circumstances. First, if any nursing home’s health 

inspections rating is one star, then the overall quality rating cannot exceed two stars. 

Second, nursing homes currently in the SFF Program have their overall quality rating 

capped at three stars even if they have high ratings in individual components. 

Page 11 GAO-09-689 The Most Poorly Performing Nursing Homes
 

than does the SFF methodology.27 (See table 2.) CMS explained that it 

changed some of the points assigned to deficiencies in the Five-Star 

System because the purpose is different from that of the SFF Program. 

The SFF Program focuses on facilities in each state whose performance is 

consistently extremely poor, and so it assigns many points to immediate 

jeopardy deficiencies relative to other, lower-level deficiencies. In 

contrast, the purpose of the Five-Star System is to distinguish performance 

across all nursing homes, rather than focus on the poorest performers, and 

so CMS modified the points to provide more emphasis on deficiencies at 

the potential for more than minimal harm and actual harm levels. The 

rating for the second component, staffing data, is based on two elements—

total nursing hours per resident day and registered nurse hours per 

resident day.28 The rating for the third component of the Five-Star System 

is based on nursing home performance on 10 quality of care measures, 

such as percentage of high-risk residents who have pressure sores.29

 

27The Five-Star System assigns the same number of additional points to SQC deficiencies 

and revisits as the SFF methodology does. According to CMS officials, CMS has not 

adopted the deficiency points used in CMS’s Five-Star System for the SFF methodology 

because it has not yet analyzed the effects of that change. 

28Total nursing hours includes hours for registered nurses, licensed practical 

nurses/licensed vocational nurses (LPN/LVN), and nurse aides. In general, registered 

nurses have more training than LPNs/LVNs. Nurse aides include certified nursing 

assistants, who work under the direction of licensed nurses. 

29The quality of care measures component uses 10 measures—7 measures for long-term 

residents of a facility and 3 measures for individuals who enter a nursing home for a short 

stay. The 7 measures for long-term residents are the percentage of residents who have an 

increasing need for help with daily activities, have a worsened ability to move about in and 

around their rooms, are high risk and have pressure sores, have catheters inserted and left 

in their bladders, were physically restrained, have urinary tract infections, and have 

moderate to severe pain. The 3 short-stay measures are the percentage of residents with 

pressure sores, moderate to severe pain, and delirium. 

Page 12 GAO-09-689 The Most Poorly Performing Nursing Homes
 

We estimated that almost 4 percent—or 580—of the nation’s roughly 

16,000 nursing homes could be considered the most poorly performing. 

These 580 homes overlap somewhat with the 755 SFF Program candidates 

and the 136 nursing homes actually selected as SFFs.30 For example, our 

estimate of 580 most poorly performing nursing homes includes (1) 302, or 

40 percent, of the 755 SFF Program candidates as of December 2008 (see 

fig. 2) and (2) 65 nursing homes that 31 states selected as SFFs from 

among the SFF Program candidates, or about half of the active SFFs as of 

February 2009.31 In addition, our estimate resulted in some states having 

fewer or more poorly performing homes than CMS currently allocates to 

states under the SFF Program. For example, 10 states each had over 20 of 

the most poorly performing nursing homes. Indiana had the greatest 

number, with 52 such nursing homes, or almost 9 percent of the total of 

580 homes. Eight states had no such nursing homes. (See fig. 3.) 

Five Hundred Eighty 

Nursing Homes Could 

Be Considered the 

Most Poorly 

Performing—Fewer 

Than CMS’s SFF 

Program Candidates 

but More Than the 

Number of SFFs 

 

30CMS generally identifies SFF Program candidates on a quarterly basis. Because the 

December 2008 time frame when we conducted our analysis did not coincide with CMS’s 

quarterly cycle, we use the term SFF Program candidates to refer to the nursing homes in 

each state with the 15 worst scores at the time that we conducted our analysis. We 

determined the SFF Program candidates using CMS’s SFF methodology, without our 

refinements. There are 755 SFF Program candidates because some nursing homes in the 

same state shared the same total score and because we excluded nursing homes in Alaska, 

which does not have SFFs, and included nursing homes in the District of Columbia, which 

has one SFF. 

31One reason that additional SFFs were not identified as the most poorly performing by our 

methodology may be that the homes’ performance improved since entering the program. 

Our estimate included another 13 nursing homes that were no longer active SFFs as of 

February 2009. 

Page 13 GAO-09-689 The Most Poorly Performing Nursing Homes
 

Figure 2: Overlap between Our Estimate of the Most Poorly Performing Nursing 

Homes in the Nation and the SFF Program Candidates, as of December 2008 

Source: GAO analysis of CMS data.

278 302

overlap

453

Our estimate of the

most poorly performing

nursing homes

in the nation (580 total)  SFF Program candidates:

the nursing homes

in each state with the

15 worst scores

(755 total)

Note: We determined the 755 SFF Program candidates using CMS’s SFF methodology. There are 

755 such homes because some nursing homes in the same state had the same total score and 

because we excluded nursing homes in Alaska, which does not have SFFs, and included nursing 

homes in the District of Columbia, which has one SFF. 

Page 14 GAO-09-689 The Most Poorly Performing Nursing Homes
 

Figure 3: Our Estimate of the Number of Most Poorly Performing Nursing Homes as of December 2008 Compared to the 

Number of SFFs by State 

Sources: Copyright © Corel Corp. All rights reserved (map); GAO analysis of CMS data.

Special focus facilities (136 total)

Our estimate of the most poorly performing nursing homes (580 total)

States with more than 20 of the most poorly performing nursing homes

PA

OR

NV

ID

MT

WY

UT

AZ

NM

CO

ND

SD

NE

TX

KS

OK

MN

IA

MO

AR

LA

IL

MS

IN

KY

TN

AL

FL

GA

SC

NC

VA

OH

MI

CA

WA

WI

NY

ME

WV

HI

AK

VT

NH

MA

RI

CT

NJ

DE

MD

DC

1 1

3 14

2 0

2 1

1 2

1 2

6 40

1 0

1 0

2 12

2 4

1 6

0 0

1 0

1 0

2 0

2 3

3 38

3 36

6 43

4 4

4 10

4 37

2 34

3 21

2 6

2 7 3 9

2 7

3 14

5 16

4 7

4 30

5 47 4 52 5 3

1 3

4 2

1 0

2 5

3 8

1 4

2 1

1 2

X

X

2 4

5 18

5 6

3 4

4 2

3 10

2

5

Note: Our estimate of 580 most poorly performing nursing homes included 65 nursing homes that 

were SFFs as of February 2009. 

Page 15 GAO-09-689 The Most Poorly Performing Nursing Homes
 

CMS has structured the SFF Program so that every state (except Alaska) 

has at least one SFF, and therefore the agency applies the SFF 

methodology to identify the 15 worst performing nursing homes in each 

state, which are not necessarily the worst performing homes in the 

nation. We developed an estimate that identified homes with worse 

compliance histories—more deficiencies at the potential for more than 

minimal harm level or higher and more revisits—than SFF Program 

candidates by applying CMS’s SFF methodology on a nationwide basis and 

using statistical scoring thresholds. These two changes had the greatest 

impact on the composition of the list of homes we identified as the most 

poorly performing compared to CMS’s approach. Our estimate also 

incorporated several refinements to the SFF methodology that moderately 

improve its ability to identify the most poorly performing nursing homes. 

CMS’s Application of 

the SFF Methodology 

Misses Many of the 

Nation’s Most Poorly 

Performing Nursing 

Homes 

Compliance history. Our estimate of 580 nursing homes identified homes 

with more deficiencies at the potential for more than minimal harm level 

or higher and more revisits, on average, compared to the 755 SFF 

Program candidates. For example, the most poorly performing nursing 

homes averaged 46.5 percent more actual harm–level deficiencies and 

19.5 percent more immediate jeopardy–level deficiencies, compared to the 

755 SFF Program candidates. (See table 3.) 

Page 16 GAO-09-689 The Most Poorly Performing Nursing Homes
 

Table 3: Compliance History over the Last Three Cycles for Our Estimate of the Most Poorly Performing Nursing Homes 

Compared to the SFF Program Candidates, as of December 2008 

Average number of deficiencies and revisits 

in the last three cycles 

Compliance history 

Most poorly 

performing 

nursing homes 

(580 homes)

SFF Program 

candidates 

(755 homes) 

Percentage 

differencea

Total deficiencies at the D level or higher 55.7 43.9 26.9

Deficiencies at the potential for more than minimal harm level (D-F) 47.3 37.7 25.5

Deficiencies at the actual harm level (G-I) 5.4 3.7 46.5

Deficiencies at the immediate jeopardy level (J-L) 3.0 2.5 19.5

Deficiencies by survey type (D-L) 

Deficiencies cited on standard surveys 40.8 33.7 21.3

Deficiencies cited on complaint investigations 14.9 10.2 45.6

Number of revisits 2.4 1.9 28.7

Source: GAO analysis of CMS data. 

Notes: (1) We determined the 755 SFF Program candidates using CMS’s SFF methodology. There 

are 755 of these nursing homes because some nursing homes in the same state had the same total 

score and because we excluded nursing homes in Alaska, which does not have SFFs, and included 

nursing homes in the District of Columbia, which has one SFF. (2) We did not determine whether the 

differences between groups were significant because the nursing homes in each group overlapped. 

a

Percentage differences were calculated using unrounded data and therefore differ slightly from 

percentage differences calculated directly from this table. 

Nationwide estimate. We developed a nationwide estimate because the 

worst performing nursing homes in some states had high total scores from 

a combination of numerous deficiencies, serious deficiencies, and revisits, 

while the worst performing nursing homes in other states did not (see 

fig. 4).32 For example, in the preceding three cycles, we found that the 

worst performing nursing home in South Dakota had a score of about 68. 

The score was composed of 32 deficiencies at the D level or higher, 2 of 

which were at the actual harm level (where the highest scope and severity 

level was H) and none of which were at the immediate jeopardy level. In 

contrast, during the same three cycles, the worst performing nursing home 

in Tennessee had a score of about 1,512, with 63 deficiencies at the D level 

 

32Some of this variation could be attributed to understatement. Our estimate did not adjust 

for state variation because we do not know which nursing homes are potentially affected. 

However, we know that nursing homes cited for many serious deficiencies or those that 

require multiple revisits have quality problems and therefore deserve federal and state 

attention. 

Page 17 GAO-09-689 The Most Poorly Performing Nursing Homes
 

or higher. Of these deficiencies, 3 were at the actual harm level and 22 at 

the immediate jeopardy level (where the highest scope and severity level 

was L). Even the 15th highest-scoring home in Tennessee, with a score of 

about 253, had notably more deficiencies at the D level or higher and more 

severe deficiencies than the highest-scoring home in South Dakota: 

specifically, the Tennessee home had 63 deficiencies at the D level or 

higher, 2 of which were at the actual harm level and 8 of which were at 

the immediate jeopardy level. If CMS applied its SFF methodology to 

identify the worst 755 homes in the nation rather than the worst 15 in each 

state, the home ranked 755 would have a score of about 127; however, 

48 percent of the SFF Program candidates had scores below this 

threshold. As a result, the SFF Program is missing some of the worst 

performing nursing homes in the nation. 

Page 18 GAO-09-689 The Most Poorly Performing Nursing Homes
 

Figure 4: Total Score Ranges for the SFF Program Candidates, as of December 

2008 

Total score

Source: GAO analysis of CMS data.

State

0 200 400 600 800 1,000 1,200 1,400 1,600

0 200 400 600 800 1000 1200 1400 1600

Max

Min

SD

ND

NV

ID

DE

MT

MD

WV

HI

IA

PA

MN

OH

WA

AZ

VA

MA

UT

RI

CO

WY

NE

CT

VT

DC

MI

IN

NC

NM

NJ

GA

MO

NH

OR

CA

KS

ME

AL

WI

IL

TX

FL

MS

OK

NY

SC

LA

AR

KY

TN

Page 19 GAO-09-689 The Most Poorly Performing Nursing Homes
 

Notes: (1) This figure illustrates the range of scores for the 755 SFF Program candidates, as of 

December 2008, which we determined based on CMS’s SFF methodology. There are 755 such 

homes because some nursing homes in the same state had the same total score and because we 

excluded nursing homes in Alaska, which does not have SFFs, and included nursing homes in the 

District of Columbia, which has one SFF. (2) The left side of the bars represents the homes with the 

lowest scores, and the right side represents the homes with the highest scores. 

Statistical scoring thresholds. Absent a fixed number of homes per 

state, we developed statistical scoring thresholds because there was no 

natural break point delineating the most poorly performing nursing homes 

from all other homes.33 The two statistical scoring thresholds we used 

were conservative, because they focused on chronic poor performance 

and nonchronic, very poor performance. About 87 percent of the 580 

nursing homes that we identified as the most poorly performing exhibited 

chronic poor performance; that is, they had high scores in at least two of 

the three cycles measured, as well as a high total score. The remaining 

roughly 13 percent of nursing homes had nonchronic but very poor 

performance; that is, they had very serious poor performance in one cycle 

only, which resulted in a very high total score. 

Homes that met our chronic poor performance threshold had total scores 

above the 93rd percentile for all nursing homes, or total scores ranging 

from approximately 168 to approximately 1,017.34 All of the nonchronic 

but very poor performing homes had total scores at or above the 

99th percentile for all nursing homes, or total scores ranging from 

approximately 330 to approximately 1,577. Table 4 summarizes the 

compliance history of two of the most poorly performing homes identified 

by our estimate, and appendix II provides a detailed compliance history 

for these two homes. 

 

33Because the distribution of nursing home scores is highly skewed, we identified a 

statistical approach that focuses on identifying outliers. 

34As noted earlier, our estimate of the most poorly performing nursing homes incorporated 

several refinements to the SFF methodology, which are discussed in the next section. 

These refinements had a moderate effect on the composition of the list of most poorly 

performing homes we identified. Not all nursing homes with total scores above the 93rd 

percentile are included in our estimate, just those that were also chronic poor performers. 

Page 20 GAO-09-689 The Most Poorly Performing Nursing Homes
 

Table 4: Compliance Histories over the Last Three Cycles for Two of the Most 

Poorly Performing Nursing Homes, as of December 2008 

Number of deficiencies 

Potential for 

more than 

minimal harm

Actual 

harm

Immediate 

jeopardy

Number of 

complaint 

investigations 

with 

deficienciesa

Example of 

standard for 

which highest 

deficiency 

was citedb

Example of a chronic poor performer (total score of about 507)

Cycle 1 32 3 0 4 Proper treatment to 

prevent or heal 

pressure sores 

Cycle 2 13 4 0 5 Facility is free of 

accident hazards 

Cycle 3 21 5 15 5 Facility prohibits 

abuse or neglect 

Example of a nonchronic, very poor performer (total score of about 344) 

Cycle 1 15 4 7 3 Resident’s care 

supervised by a 

physician 

Cycle 2 7 0 0 1 Facility establishes 

infection control 

program 

Cycle 3 5 0 0 0 Medication error 

rates of 5 percent 

or more 

Source: GAO analysis of CMS data. 

Notes: (1) Both homes had one revisit that contributed to their total scores. (2) Total score 

incorporates our refinements to the SFF methodology, which are discussed in the next section. 

a

This column only includes complaint investigations with deficiencies at the D level or higher. 

b

The descriptions of the standards in this column have been abbreviated from those that appear in 

CMS guidance. 

Additional homes might have been identified as the most poorly 

performing had we used different thresholds. For example, one nursing 

home with a total score of about 324 did not meet our definition for 

chronic poor performance and was below the threshold of 330 for 

nonchronic, very poor performance. During the three-cycle period, this 

nursing home had 41 D- through F-level deficiencies, 5 immediate jeopardy 

deficiencies, and a second revisit that contributed to the score, but most of 

the deficiencies and the revisit occurred in one cycle. 

Page 21 GAO-09-689 The Most Poorly Performing Nursing Homes
 

Refinements made to CMS’s SFF methodology. Our three refinements 

to CMS’s SFF methodology had a moderate effect on the composition of 

the list of homes we identified as the most poorly performing. 

• Deficiency points. We believe that the deficiency points used in the FiveStar System are more appropriate for identifying the most poorly 

performing nursing homes nationwide than those used in the SFF 

methodology because they compensate somewhat for understatement and 

the interstate variation in the citation of serious deficiencies. First, given 

the significant disparity between immediate jeopardy (50 to 150 points) 

compared to lower-level deficiencies (2 to 6 points for D- through F-level 

deficiencies), our use of SFF deficiency points to identify the most poorly 

performing nursing homes nationwide might have missed poorly 

performing nursing homes in states with significant understatement. 

Second, there is considerable interstate variation in the citation of serious 

deficiencies, including immediate jeopardy–level deficiencies. For 

example, in 2008, about 11.3 percent of deficiencies were at the immediate 

jeopardy level in one state, but less than 1.0 percent of deficiencies were 

cited at that level in 26 states.35 The Five-Star System, on average, doubles 

the points assigned to deficiencies below the immediate jeopardy level, 

giving a D-level deficiency 4 points and a G-level deficiency 20 points, 

compared to 2 and 10 points, respectively, using the SFF deficiency 

methodology. As a result, using the Five-Star System deficiency points, 

homes with numerous D- through I-level deficiencies are more likely to be 

identified as the most poorly performing. CMS officials told us that they 

planned to evaluate the effect of using the Five-Star System deficiency 

points on identifying SFF candidates; our analysis showed that it changed 

the composition of SFF Program candidates by an average of about 2.5 

candidates per state. 

• Substandard quality of care. In comparison with the SFF methodology 

and the Five-Star System, we assigned 5 more points to G-level 

deficiencies that occurred in any of the three categories of standards that 

CMS considers to be SQC.36 As noted earlier, CMS does not classify any 

G-level deficiencies as SQC. Without this modification, an F-level 

deficiency in an SQC area is assigned the same number of points as a G-

 

35About 1.7 percent of all deficiencies cited in 2008 were at the immediate jeopardy (J-L) 

level, about 4.9 percent were at the actual harm (G-I) level, and about 84.6 percent were at 

the D through F levels. 

36This is the same number of additional points that CMS assigns to H- and I-level 

deficiencies if they occur in areas considered to be SQC. (See table 2.) 

Page 22 GAO-09-689 The Most Poorly Performing Nursing Homes
 

level deficiency even if the G-level deficiency is in an SQC standard—10 

and 20 points, respectively, under the SFF or Five-Star System 

methodologies. (See table 2.) This adjustment was important because 

approximately 45 percent of all nursing homes had one or more G-level 

deficiencies in an SQC category during the three cycles used for 

calculating SFF scores. Therefore, assigning SQC points to G-level 

deficiencies had an effect on total scores for the nursing homes, which we 

used to determine the most poorly performing homes nationwide, and 

would have an effect on the composition of CMS’s SFF candidate list. For 

example, about 4 percent of the SFF Program candidates—or less than 

one candidate per state on average—would change if CMS assigned SQC 

points to G-level deficiencies. 

• Technical status changes. While the SFF methodology does not consider 

all deficiencies and revisits identified within the three-cycle period that 

occurred before the nursing home’s technical status change, we 

incorporated the full histories of nursing homes that underwent a 

technical status change.37 At the time of a technical status change, a new 

provider identification number is assigned, and the nursing home’s 

complete history under the old number is not combined with that of the 

new provider number. For example, a nursing home with a status change 

on January 1, 2008, might have a compliance history for only 1 year at the 

time we did our work instead of the three cycles called for in the SFF 

methodology. The SFF scores for nursing homes that have undergone 

technical status changes within the last three cycles are almost always 

lower than would be the case if three cycles of deficiency history were 

included and, therefore, more favorable than would be justified by the 

complete history. We found that almost 1 percent of all nursing homes 

(148), including 11 of the 580 we identified as the most poorly performing, 

had a technical status change during the last three cycles that affected 

 

37We developed a list of all provider numbers—the identification numbers used by CMS to 

identify nursing homes—associated with any nursing homes that were indicated by CMS’s 

data as having made such a change to their provider numbers from October 1, 2004 (i.e., 

the beginning of fiscal year 2005), through December 17, 2008. We did so by using a field 

that identifies the new provider number after a home undergoes a technical status change, 

or in two instances, using the nursing home’s address to link provider numbers. In the 

health inspections component of its Five-Star System, CMS adjusts for any nursing home 

with only two standard surveys by imputing a total score to account for the missing survey. 

This adjustment is less precise because it imputes the results instead of using a home’s 

actual performance history. CMS does not report star ratings for nursing homes with only 

one standard health inspection. 

Page 23 GAO-09-689 The Most Poorly Performing Nursing Homes
 

their SFF scores.38 For most states, this change would not have affected 

their SFF candidate lists. 

Compared to all other nursing homes, the most poorly performing nursing 

homes in the nation averaged notably more deficiencies at the D level or 

higher, more serious deficiencies, and more revisits. They were also more 

likely to be for-profit and part of a chain and have more beds and 

residents. In addition, they had an average of almost 24 percent fewer 

registered nurse hours per resident per day. 

Key Characteristics, 

such as Chain 

Affiliation and ForProfit Status 

Differentiated the 

Most Poorly 

Performing Nursing 

Homes 

Actual Harm and 

Immediate Jeopardy 

Deficiencies Occurred 

Significantly More Often 

for the Most Poorly 

Performing Nursing 

Homes 

Compared to all other nursing homes, deficiencies over the last three 

cycles at the actual harm (G through I) level occurred over 5 times as 

often, and deficiencies at the immediate jeopardy (J through L) level 

occurred 15 times as often for the most poorly performing homes.39 (See 

table 5.) Furthermore, we found that revisits were made to the most poorly 

performing nursing homes 6 times as often as to all other nursing homes. 

 

38In 6 of the 11 nursing homes, this modification increased their total score by less than 

about 8 percent; however, the total score of the other 5 nursing homes increased from 

about 10 to about 370 percent. 

39The OSCAR data we analyzed were as of December 2008. We analyzed only D- through Llevel deficiencies because nursing homes with deficiencies at the A, B, or C levels are 

considered to be in substantial compliance with federal quality standards. 

Page 24 GAO-09-689 The Most Poorly Performing Nursing Homes
 

Table 5: Compliance History over Last Three Cycles for the Most Poorly Performing 

Nursing Homes and All Other Nursing Homes, as of December 2008 

Average number of 

deficiencies and revisits in the 

last three cycles 

Compliance history 

Most poorly 

performing 

nursing homes

All other 

nursing 

homes

Total deficiencies at the D level or higher 55.7 20.3

Deficiencies by scope and severity level (D-L) 

Deficiencies at the potential for more than 

minimal harm level (D-F) 47.3 19.1

Deficiencies at the actual harm level (G-I) 5.4 1.0

Deficiencies at the immediate jeopardy 

level (J-L) 3.0 0.2

Deficiencies by survey type (D-L) 

Deficiencies cited on standard surveys 40.8 17.3

Deficiencies cited on complaint investigations 14.9 3.0

Number of revisits 2.4 0.4

Source: GAO analysis of CMS data. 

Note: All differences between groups are significant at the 0.05 level. 

The most poorly performing nursing homes were more frequently cited for 

deficiencies in important care areas and specific standards related to the 

delivery of care compared to all other nursing homes. Seven of the 10 most 

frequently cited deficiencies at the immediate jeopardy level involved 

standards in the categories of care that CMS considers to be SQC and four 

of the 10 are related to abuse or neglect. For example, about 42 percent of 

the most poorly performing nursing homes had at least one immediate 

jeopardy deficiency related to being free of accident hazards in the last 

three cycles, compared with about 5 percent for all other nursing homes. 

(See table 6.) A larger proportion of the most poorly performing nursing 

homes were cited for actual harm in each of the three SQC areas—about 

90 percent in Quality of Care, about 31 percent in Resident Behavior and 

Facility Practices, and about 17 percent in Quality of Life. In comparison, a 

smaller proportion of all other nursing homes were cited for actual harm 

in those same categories of care—about 42 percent in Quality of Care, 

about 6 percent in Resident Behavior and Facility Practices, and about 

2 percent in Quality of Life. (See app. III.) 

Page 25 GAO-09-689 The Most Poorly Performing Nursing Homes
 

Table 6: The 10 Standards Most Often Cited at the Immediate Jeopardy Level in the Last Three Cycles among the Most Poorly 

Performing Homes Compared to All Other Nursing Homes, as of December 2008 

Ten standards most often cited at the immediate 

jeopardy level among the most poorly performing 

nursing homesa

Federal quality standard 

category 

Percentage of the 

most poorly 

performing nursing 

homes with at least 

one immediate 

jeopardy citation

Percentage of all 

other nursing 

homes with at 

least one 

immediate 

jeopardy citation

Facility is free of accident hazards Quality of Careb42.1 5.4

Facility is administered effectively to obtain highest 

practicable physical, mental, and psychosocial well-being 

of each resident 

Administration 28.6 2.1

Provides necessary care for highest practicable physical, 

mental, and psychosocial well-being 

Quality of Careb25.7 2.5

Facility prohibits abuse or neglect Resident Behavior and Facility 

Practicesb

15.3 1.2

Policies and procedures prohibit abuse or neglect Resident Behavior and Facility 

Practicesb

12.6 0.7

Facility must inform resident, resident’s physician, and 

family of any accidents; changes in the resident’s 

physical, mental, or psychosocial status, or treatment; or 

of a decision to transfer or discharge resident 

Resident Rights 11.9 0.8

Proper treatment to prevent or heal pressure sores Quality of Careb10.9 0.6

Facility must not employ persons guilty of abuse, neglect, 

or mistreatment, and must investigate and report alleged 

violations involving abuse, neglect, or mistreatment 

Resident Behavior and Facility 

Practicesb

10.0 0.6

Facility maintains a quality assessment and assurance 

committee 

Administration 9.5 0.5

Residents have a right to be free from abuse Resident Behavior and Facility 

Practicesb

8.6 0.8

Source: GAO analysis of CMS data. 

Note: All differences between groups are significant at the 0.05 level. 

a

The descriptions of the standards in this column have been abbreviated from those that appear in 

CMS guidance. 

b

Deficiencies cited at the immediate jeopardy level in these standards are considered to be SQC. 

Page 26 GAO-09-689 The Most Poorly Performing Nursing Homes
 

We also found that from fiscal years 2006 through 2008 the most poorly 

performing nursing homes were much more likely to have had deficiencies 

that could have resulted in the imposition of at least one immediate 

sanction compared to all other nursing homes.40 For example, in fiscal 

year 2008, about 33 percent of the most poorly performing homes may 

have been at risk of having at least one immediate sanction imposed, 

compared to about 4 percent for all other nursing homes. Nursing homes 

that receive at least one G- through L-level deficiency on successive 

standard surveys or complaint investigations must be referred for 

immediate sanctions, and about 15 percent of the deficiencies for the

poorly performing nursing homes, on average over the last three cycles, 

were at the actual harm or immedia

 most 

te jeopardy level. 

 

The Most Poorly 

Performing Nursing 

Homes Differed from All 

Others in Several Ways, 

Including Chain Affiliation 

and For-Profit Status 

We found that the most poorly performing nursing homes differed from all 

other nursing homes in terms of the proportion of each group that was 

chain affiliated, for-profit, or both. They also differed in size and nurse 

staffing.41

Type of organization. We found that the most poorly performing nursing 

homes were less likely to be hospital based compared to all other nursing 

homes. Additionally, compared to all other nursing homes, we found that 

the most poorly performing nursing homes were more likely to be part of 

for-profit organizations, more likely to be affiliated with a chain 

organization, and more likely to be both for-profit and affiliated with a 

chain organization. About 55 percent of the most poorly performing 

nursing homes were for-profit and chain affiliated, compared to about 

41 percent of all other homes. (See table 7.) 

40We analyzed a CMS file that identifies nursing homes whose deficiency histories could 

have subjected them to immediate sanctions. A CMS official told us that the file is not used 

by states to refer homes for immediate sanction. As a result, the file does not indicate that 

immediate sanctions were imposed on homes but represents CMS’s analysis of probable 

instances of immediate sanctions. 

41The data we analyzed were as of December 2008, except for the data on nurse staffing, 

which were as of November 2008. 

Page 27 GAO-09-689 The Most Poorly Performing Nursing Homes
 

Table 7: Distribution of the Most Poorly Performing Nursing Homes and All Other 

Nursing Homes by Type of Organization, as of December 2008 

Type of organization 

Percentage of the 

most poorly 

performing nursing 

homes

Percentage

of all other 

nursing 

homes

Facility type 

Hospital based 1.9 7.8

Freestanding 98.1 92.2

Ownership type 

For-profit 

(individual, partnership, or corporation) 

84.1 67.0

Nonprofit (corporation, church, or other) 11.6 27.1

Government owned 4.3a5.9a

Chain affiliation 61.9 53.2

For-profit and chain affiliated 54.5 41.4

Nonprofit and chain affiliated 6.6 11.2

Government owned and chain affiliated 0.9a0.7a

Source: GAO analysis of CMS data. 

Notes: Unless otherwise noted, all differences between groups are significant at the 0.05 level. 

Individual entries may not sum to totals because of rounding. 

a

The difference between most poorly performing and all other nursing homes for this variable is not 

significant. 

Participation in Medicare and Medicaid. We found that a higher 

percentage of the most poorly performing homes participated in both 

Medicare and Medicaid, and a smaller percentage of such homes 

participated only in Medicare or only in Medicaid, compared to all other 

nursing homes. (See table 8.) 

Page 28 GAO-09-689 The Most Poorly Performing Nursing Homes
 

Table 8: Distribution of the Most Poorly Performing Nursing Homes and All Other 

Nursing Homes by Medicare and Medicaid Participation, as of December 2008 

Medicare and Medicaid 

participation 

Percentage of the most 

poorly performing 

nursing homes 

Percentage of 

all other 

nursing homes

Medicare and Medicaid 96.2 90.2

Medicare only 0.2 5.3

Medicaid only 3.6 4.5

Source: GAO analysis of CMS data. 

Note: All differences between groups are significant at the 0.05 level. 

Beds and residents. We found that a larger percentage of the most 

poorly performing homes had more than 100 beds, compared to all other 

nursing homes.42 On average, the most poorly performing nursing homes 

had about 23 percent more beds than all other nursing homes. 

Additionally, our analysis found that on average, the most poorly 

performing homes had almost 14 percent more residents, a lower 

occupancy rate, and a greater share of Medicaid patients. (See table 9.) 

 

42For this section, we analyzed the number of certified beds, which is the number of 

Medicare beds, Medicaid beds, or both. 

Page 29 GAO-09-689 The Most Poorly Performing Nursing Homes
 

Table 9: Distribution of the Most Poorly Performing Nursing Homes and All Other 

Nursing Homes by Beds and Residents, as of December 2008 

Beds and residents 

Most poorly 

performing 

nursing homes

All other 

nursing homes

Average number of beds per homea129.7 105.3

Bed size (percentage) 

0 to 49 beds 3.4 14.1

50 to 99 beds 28.8 36.6

100 to 199 beds 57.2 43.0

More than 199 beds 10.5 6.2

Average number of residents per home 101.7 89.4

Average occupancy rate (percentage)b 78.3 84.6

Average share of resident type (percentage) 

Medicare 12.2 15.8

Medicaid 69.5 59.5

Other 18.3 24.7

Source: GAO analysis of CMS data. 

Notes: All differences between groups are significant at the 0.05 level. Individual entries may not sum 

to totals because of rounding. 

a

We analyzed the number of certified beds, which is the number of Medicare beds, Medicaid beds, or 

both. 

b

Average occupancy rate is the average of the number of residents per home divided by the number 

of certified beds per home. 

Nurse staffing levels. Compared to all other nursing homes, the most 

poorly performing homes had almost 24 percent fewer registered nurse 

hours per resident per day on average.43 One effect of this difference is 

that the most poorly performing nursing homes averaged fewer registe

nurse hours per resident per day as a share of total nursing hours. 

Specifically, registered nurse hours made up about 8 percent of total nurse 

staffing hours in the most poorly performing nursing homes, compared to 

about 10 percent in all other nursing homes. (See table 10.) 

red 

 

43The staffing data we analyzed were case-mix adjusted by CMS for use in the Five-Star 

System. 

Page 30 GAO-09-689 The Most Poorly Performing Nursing Homes
 

Table 10: Distribution of the Most Poorly Performing Nursing Homes and All Other 

Nursing Homes by Case-Mix-Adjusted Nurse Staffing Levels, as of November 2008 

Nurse staffing levelsa

Most poorly 

performing 

nursing homes 

All other 

nursing

homes

Percentage 

differenceb

Type of staff 

(average hours per resident per day) 

Registered nurse 0.28 0.36 -23.51

Licensed practical and vocational nurses 1.03 0.98 4.16

Nurse aide 2.37c2.40c-1.13

Total 3.42 3.55 -3.76

Registered nurse hours as a share of total 

nurse staffing hours (percentage)d

8.15 10.07 n/a

Source: GAO analysis of CMS data. 

Note: Unless otherwise noted, all differences between groups are significant at the 0.05 level. 

a

The staffing data we analyzed were case-mix adjusted by CMS for use in the Five-Star System. 

b

Percentage differences were calculated using unrounded data and therefore differ slightly from 

percentage differences calculated directly from this table. 

c

The difference between most poorly performing and all other nursing homes for this variable is not 

significant. 

d

Registered nurse hours as a share of total nurse staffing hours is an average across all homes in 

each group; therefore, the percentages differ slightly from those as calculated directly from this table. 

Our estimate of the most poorly performing nursing homes nationwide is 

more than four times greater than the 136 homes that receive enhanced 

scrutiny under CMS’s SFF Program. We believe that our estimate is 

conservative, because we focused only on those nursing homes with 

chronic poor performance over time or with very poor performance in one 

survey cycle. Because of resource constraints, CMS limits the size of the 

SFF Program, requiring every state except Alaska to select from 1 to 6 

homes—an allocation based on the number of nursing homes in each 

state—from a list of 15 candidates. The homes selected are not necessarily 

the most poorly performing homes in the nation but rather are among the 

poorest performers in each state. In contrast, the 580 homes we identified 

have more deficiencies at the potential for more than minimal harm level 

or higher and more revisits on average than the 755 homes identified as 

potential SFF candidates using CMS’s SFF methodology. Our estimate also 

revealed that the state-by-state distribution of the most poorly performing 

homes nationwide is uneven, calling into question the approach CMS uses 

to allocate SFFs across states. 

Conclusions 

Page 31 GAO-09-689 The Most Poorly Performing Nursing Homes
 

Furthermore, we believe that CMS’s SFF Program and the Five-Star 

System could be strengthened by incorporating the three enhancements 

we made to identify the most poorly performing homes nationwide: 

• First, we adopted the deficiency points that CMS developed for its FiveStar System because they compensate somewhat for understatement and 

the interstate variation in the citation of serious deficiencies, an important 

consideration for our nationwide estimate of the most poorly performing 

nursing homes. Currently, CMS uses a different set of deficiency points for 

the SFF methodology, but agency officials told us that they planned to 

study the effect of using a common set of numeric points—the Five-Star 

System deficiency points—for both methodologies. 

• Second, we added points for G-level deficiencies in the three standard 

areas that CMS considers to be an indication of SQC. We found that about 

4 percent of the SFF candidates—or less than 1 candidate per state, on 

average—would change if CMS assigned SQC points to G-level 

deficiencies when they were cited in an SQC area. Without such an 

adjustment, an F-level deficiency in an SQC area would receive the same 

number of deficiency points as a G-level deficiency in the same standard 

area. Approximately 45 percent of all nursing homes had one or more 

G-level deficiencies in an SQC category during the three cycles used for 

calculating SFF scores. 

• Third, we incorporated the full compliance history of homes that 

underwent technical status changes. For example, a nursing home with a 

technical status change on January 1, 2008, might have a compliance 

history for only 1 year at the time we did our work instead of the three 

cycles called for by the SFF methodology. The SFF scores of homes that 

have undergone technical status changes within the last three cycles are 

almost always lower than if all three cycles were considered. We also 

found that the Five-Star System does not accurately take into 

consideration technical status changes because it imputes a total score to 

account for one missing standard survey rather than using actual survey 

results. 

To improve the targeting of scarce survey resources, the Administrator of 

CMS should consider an alternative approach for allocating the 136 SFFs 

across states, by placing more emphasis on the relative performance of 

homes nationally rather than on a state-by-state basis, which could result 

in some states having only one or not any SFFs and other states having 

more than they are currently allocated. 

Recommendations for 

Executive Action 

Page 32 GAO-09-689 The Most Poorly Performing Nursing Homes
 

To improve the SFF methodology’s ability to identify the most poorly 

performing nursing homes, the Administrator of CMS should make the 

following three modifications: 

1. Consider using a common set of numeric points for identifying poorly 

performing nursing homes by determining the effect of adopting those 

associated with the Five-Star System for the SFF methodology. 

2. Assign points to G-level deficiencies in SQC areas equivalent to those 

additional points assigned to H- and I-level deficiencies in SQC areas. 

3. Account for a nursing home’s full compliance history regardless of 

technical status changes. 

To ensure consistency with the SFF methodology, CMS should also 

consider making two of these modifications—the SQC and full compliance 

history changes—to its Five-Star System. 

We obtained written comments on our draft report from CMS, which are 

reprinted in appendix IV. CMS noted that our report adds value regarding 

the methods that CMS and the nursing home industry should use to 

address the issue of homes that consistently demonstrate quality of care 

problems and indicated that the agency would seriously consider all of our 

recommendations. CMS generally agreed in principle with our 

recommendations. 

Agency Comments 

and Our Evaluation 

In response to our first recommendation, CMS noted that it would 

evaluate a “hybrid” approach that would assign some SFFs using homes’ 

performance in each state and other SFFs on their relative national 

ranking. If implemented, CMS’s proposed hybrid approach would address 

our recommendation that it consider placing more emphasis on the 

relative performance of homes nationally, which might result in some 

states having fewer SFFs and others having more than their current 

allocation. We did not recommend that CMS allocate SFFs solely on the 

basis of the relative performance of homes nationally, an approach CMS 

would disagree with according to its comments. 

CMS agreed in principle with our remaining recommendations—intended 

to improve the SFF methodology’s ability to identify the most poorly 

performing nursing homes and ensure its consistency with the agency’s 

Five-Star System—and noted that it would evaluate the effects of adopting 

them. The agency explained that there might be technical barriers to fully 

Page 33 GAO-09-689 The Most Poorly Performing Nursing Homes
 

implementing our recommendation that it account for a nursing home’s 

full compliance history regardless of technical status changes, but noted 

that it would implement the recommended adjustment to the maximum 

extent practicable. CMS agreed that although this change would affect a 

small number of providers, it would improve the accuracy of ratings for 

those providers. 

CMS also provided technical comments, which we incorporated as 

appropriate. 

As agreed with your offices, unless you publicly announce the contents of 

this report earlier, we plan no further distribution until 30 days from the 

report date. At that time, we will send copies to the Administrator of the 

Centers for Medicare & Medicaid Services and appropriate congressional 

committees. The report will also be available at no charge on the GAO 

Web site at http://www.gao.gov. 

If you or your staff have any questions about this report, please contact me 

at (202) 512-7114 or at dickenj@gao.gov. Contact points for our Offices of 

Congressional Relations and Public Affairs may be found on the last page 

of this report. GAO staff who made major contributions to this report are 

John E. Dicken 

listed in appendix V. 

Director, Health Care 

Page 34 GAO-09-689 The Most Poorly Performing Nursing Homes
Appendix I: Scope and Methodology 

Page 35 GAO-09-689 

Appendix I: Scope and Methodology 

This appendix provides a more detailed description of our scope and 

methodology. 

To determine the number of most poorly performing nursing homes in the 

nation and compare their performance to that of homes identified using 

the Centers for Medicare & Medicaid Services’ (CMS) approach, we began 

by interviewing agency officials about the Special Focus Facility (SFF) 

Program and methodology and by reviewing documentation related to the 

methodology. In addition, we interviewed officials in all 10 CMS regional 

offices and 14 state survey agencies regarding their impressions of the SFF 

methodology and also asked the state survey agencies what they consider 

to be indicators of poor performance.1

 To ensure that we calculated the 

scores for each nursing home consistent with CMS’s SFF methodology, we 

obtained a copy of the computer programming that CMS used to score and 

rank nursing homes, verified that our use of CMS’s program generated 

results that were consistent with output on scores that CMS provided to 

states, and used the program as the basis for our estimate of the most 

poorly performing nursing homes in the United States. The SFF 

methodology creates a total score for each nursing home over three cycles 

by assigning points to the following data, which we obtained from 

CMS’s On-Line Survey, Certification, and Reporting system (OSCAR) 

database: (1) deficiencies cited on the three most recent standard surveys, 

(2) deficiencies cited on the last 3 years of complaint investigations, and 

(3) revisits associated with the three most recent standard surveys.2

Additional points are assigned to deficiencies classified as substandard 

quality of care (SQC). Each cycle consists of one standard survey, revisits 

associated with the standard survey, and 12 months of complaint 

investigations. We extracted these data from OSCAR in December 2008.3

Determining the 

Number and 

Comparing the 

Performance of the 

Most Poorly 

Performing Nursing 

Homes 

 

1

We interviewed officials in a nongeneralizable sample of 14 state survey agencies, which 

were selected based on a combination of factors, including the number of SFFs allocated to 

the state and SFF scores. 

2

Points are only assigned to facilities that require more than one revisit before being able to 

demonstrate substantial compliance. 

3

OSCAR data change continually as new surveys are conducted and entered into the 

database, but there can be a lag time. As a result, the data we analyzed did not necessarily 

include all surveys conducted through the date of our data extract. 

The Most Poorly Performing Nursing Homes 
Appendix I: Scope and Methodology 

To learn about methods used to rate nursing home performance, we 

interviewed officials of two nursing home associations—the American 

Health Care Association and the American Association of Homes and 

Services for the Aging, interviewed experts in long-term care research, 

attended meetings that CMS held to seek input from long-term care 

researchers on the development of the agency’s Five-Star Quality Rating 

System (Five-Star System), analyzed information available at CMS’s 

Providing Data Quickly Web site, reviewed prior GAO reports, and 

interviewed officials from some states with nursing home rating systems. 

We also reviewed documentation describing additional approaches to 

rating nursing home performance. Specifically, we reviewed eight nursing 

home rating systems, which considered a variety of rating factors.4

Examination of the SFF 

Methodology 

To determine the adequacy of the SFF methodology, we compared the 

methodology to other compliance-based measures of poor performance 

and tested the sensitivity of the methodology to variations, such as 

weighting. 

We compared the SFF methodology to two other compliance-based 

measures of poor performance—SQC and immediate sanctions.5

 We found 

that those nursing homes with the worst total scores in the nation were 

much more likely to have met the criteria for SQC in the last 1, 2, and 3 

years compared to all other nursing homes. Similarly, we found that the 

same nursing homes were much more likely to have had deficiencies that 

could have resulted in the imposition of at least one immediate sanction in 

the last 1, 2, and 3 years compared to all other nursing homes.6

 In addition, 

 

4

We reviewed materials describing systems that rate nursing homes in the following states: 

California, Florida, Indiana, Massachusetts, Minnesota, New Jersey, Ohio, and Texas. We 

did not conduct a comprehensive review to identify all states that have nursing home rating 

systems. New Jersey no longer maintains its own rating system and refers individuals 

instead to CMS’s Five-Star System. 

5

For these analyses, we compared the 766 nursing homes in the nation with the highest 

total scores to all other nursing homes. The number 765 is equal to 50 states (including 

Alaska) and the District of Columbia multiplied by 15 nursing homes per state, and there 

was a tie in total score among these homes, for a total of 766 nursing homes. 

6

We analyzed a CMS file that identifies nursing homes whose deficiency histories would 

have subjected them to immediate sanctions. A CMS official told us that the file is not used 

by states to refer homes for immediate sanction. As a result, the file does not indicate that 

immediate sanctions were imposed on homes but represents CMS’s analysis of probable 

instances of immediate sanctions. We did not determine which nursing homes were 

sanctioned in this way. 

Page 36 GAO-09-689 The Most Poorly Performing Nursing Homes
Appendix I: Scope and Methodology 

we tested the sensitivity of CMS’s SFF methodology to several variations, 

some of which led us to consider making modifications to the 

methodology that affected facility scores. For example, in one variation, 

we modified the SFF methodology so that the cycle scores were no longer 

weighted—CMS began weighting the cycle scores in June 2008. We 

concluded from this test that the SFF methodology was sensitive to 

weighting, which influenced our decision to impose a requirement in our 

scoring thresholds such that the most poorly performing nursing homes 

have high scores in at least two of three cycles or a very high score overall. 

Based on our examinations of the SFF methodology, document review, 

and interviews, we concluded that the SFF methodology is reasonable and 

comprehensive because it uses multiple years of data, includes all 

deficiencies as opposed to a subset of deficiencies, includes deficiencies 

from standard surveys and complaint investigations, and accounts for the 

scope and severity of deficiencies and the number of revisits. 

Furthermore, CMS has refined the SFF methodology over time. 

Development of the 

Nationwide Statistical 

Scoring Thresholds 

Although we concluded that estimating the number of the most poorly 

performing nursing homes on a state-by-state rather than on a national 

basis would yield inconsistent results, we determined that there was no 

natural break point that differentiated the most poorly performing nursing 

homes from all other homes. As a result, we investigated several statistical 

approaches and determined that Tukey’s method was appropriate because 

the distribution of nursing homes’ total scores is highly skewed.7

 Tukey’s 

method is meant to identify the extreme ends of the distribution. It labels 

an observation as a potential outlier if its value is greater than the 

threshold identified by the following equation:8

Potential Outlier Threshold = Q3 + 1.5 * (Q3 – Q1) 

Where: Q3 = 75th percentile and 

Q1 = 25th percentile 

 

7

See John W. Tukey, Exploratory Data Analysis (Reading, Mass.: Addison-Wesley 

Publishing Company, 1977), 39-47. See also, John M. Chambers, William S. Cleveland, Beat 

Kleiner, and Paul A. Tukey, Graphical Methods for Data Analysis (Boston, Mass.: 

Wadsworth International Group and Duxbury Press, 1983), 21-22. 

8

If there is no observation exactly at the value identified by the equation, the observation 

with the next lower value would be the threshold. 

Page 37 GAO-09-689 The Most Poorly Performing Nursing Homes
Appendix I: Scope and Methodology 

The range identified by (Q3 – Q1), called the interquartile range, covers 

50 percent of the observations in the center of the distribution. We applied 

this method by identifying nursing homes that had scores that were above 

the potential outlier threshold. 

We then explored several options to identify the most poorly performing 

nursing homes using Tukey’s method as a basis. For each option, we 

analyzed the group of resulting nursing homes identified as poor 

performers and those missed by the thresholds. As the result of our 

examination of the SFF methodology and prior work in which we 

classified nursing homes as low, moderately, or high performing, we knew 

that nursing homes that have serious deficiencies in one year may not 

demonstrate consistent poor performance—what we term chronic poor 

performance in this report.9

 Thus, another option we considered was to 

identify as a poor performer any nursing home that had a total score that 

(1) was above the potential outlier threshold and (2) was also above the 

potential outlier threshold for at least two of its three cycle scores. This 

option identified 507 nursing homes. Because these nursing homes had 

poor performance in at least two of three cycles as well as high total 

scores, we concluded that this threshold identified chronic poor 

performance. However, we found that when we limited the most poorly 

performing nursing homes to this group of chronic poor performers we 

missed some nursing homes with very poor performance that was not 

chronic. Therefore, we established a second threshold to identify those 

very poor performers—those nursing homes that were at or above the 

99th percentile—or approximately 330—of total score.10 This threshold 

added another 73 nursing homes.11

 

9

For a prior report, using deficiencies from three standard surveys from January 1, 1999, 

through November 1, 2002, we classified 15 percent of nursing homes as low performing, 

65 percent as moderately performing, and 20 percent as high performing. See GAO-07-373. 

10We explored using the 95th percentile of total score to identify nonchronic but very poor 

performance, but decided that the 99th percentile was a more conservative approach 

because it limited this group to those with extremely high total scores. 

11We considered additional thresholds that would have identified additional poor 

performers with many D-level or higher or many G-level or higher deficiencies. For 

example, 50 additional nursing homes would have been identified as poor performers if we 

had also included nursing homes with a large number of D-level or higher deficiencies 

(greater than or equal to the 99th percentile of the number of D-level or higher

deficiencies). An additional 25 nursing homes would have been identified as poor 

performers if we had also included nursing homes with a large number of G-level or higher 

deficiencies (greater than or equal to the 99th percentile of the number of G-level or higher 

deficiencies). 

Page 38 GAO-09-689 The Most Poorly Performing Nursing Homes
Appendix I: Scope and Methodology 

To determine the characteristics of the most poorly performing nursing 

homes that distinguish them from all other nursing homes, we analyzed 

deficiencies and revisits from the three most recent cycles—that is, the 

three most recent standard surveys as of the date of our data extract 

(December 17, 2008) and any associated revisits, as well as deficiencies 

cited on complaint investigations conducted 3 years before our data 

extract. We also analyzed other data that describe the characteristics of 

nursing homes: a December 17, 2008, extract of other OSCAR variables; 

case-mix-adjusted nurse staffing hours available from CMS’s Five-Star 

System, which were dated November 2008; and a list developed by CMS of 

nursing homes whose deficiency histories could have subjected them to 

immediate sanctions, which we obtained from CMS in October 2008.12

Determining the 

Characteristics of the 

Most Poorly 

Performing Nursing 

Homes 

Following are highlights of how we analyzed certain characteristics: 

• We calculated the number of nursing homes in each fiscal year that had 

deficiencies that could have resulted in the imposition of at least one 

immediate sanction. 

• Nursing homes self-report their ownership type. We created the ownership 

type of for-profit by combining three categories of for-profit nursing 

homes designated in CMS’s data (individual, partnership, and corporation) 

and the category of limited liability corporation. Similarly, we created the 

ownership type of nonprofit by combining three categories of nonprofit 

nursing homes (corporation, church related, and other), and the 

ownership type of government from the six designations made in CMS 

data (state, county, city, city/county, hospital district, and federal). 

• CMS maintains a variable in its data called multi–nursing home (chain) 

ownership, which is self-reported by nursing homes and which we refer to 

as chain affiliation. According to CMS, multi–nursing home chains have 

two or more homes under one ownership or operation. We determined the 

percentage of nursing homes that were for-profit and chain affiliated, 

nonprofit and chain affiliated, or government owned and chain affiliated 

by combining the ownership type described above with CMS’s designation 

of multi–nursing home (chain) ownership. 

 

12As previously noted, this file is not used by states to refer homes for immediate sanction 

and does not indicate that immediate sanctions were imposed on homes but represents 

CMS’s analysis of probable instances of immediate sanctions. 

Page 39 GAO-09-689 The Most Poorly Performing Nursing Homes
Appendix I: Scope and Methodology 

• We used the number of beds certified for payment for Medicare, Medicaid, 

or both to calculate the following: the average number of beds per nursing 

home and the percentage of nursing homes by bed size category (0 to 49, 

50 to 99, 100 to 199, and more than 199 beds). 

• We calculated the percentage share of residents by resident type 

(Medicare, Medicaid, or other) by dividing the number of Medicare, 

Medicaid, and other patients by the number of total residents. 

• We calculated the occupancy rate by dividing the total number of residents 

by the number of certified beds. We used certified beds to calculate the 

occupancy rate instead of total beds because CMS officials told us that 

certified beds provided more reliable information. 

• We analyzed the following nurse staffing hours, which were case-mix 

adjusted by CMS for use in its Five-Star System: registered nurse hours per 

resident per day, licensed practical nurse and vocational nurse hours per 

resident per day, nurse aide hours per resident per day, and total staffing 

hours per resident per day.13 We calculated resident nurse hours as a share 

of the total. Unadjusted nurse staffing hours data are collected by CMS, 

self-reported by nursing homes, and represent staffing levels for a 2-week 

period before the state inspection. CMS case-mix adjusted the staffing data 

using the average minutes of nursing care used to care for residents in a 

given resource utilization group category as reflected in the Medicare 

skilled nursing facility prospective payment system.14 CMS acknowledges 

that the staff hours collected from nursing homes have certain limitations. 

In order to increase the accuracy and comprehensiveness of the staffing 

data, CMS has been investigating whether it can use nursing home payroll 

data to report staffing levels on the Nursing Home Compare Web site. 

 

13Nurse staffing hours were not available for 6.9 percent of nursing homes we determined 

were the most poorly performing and for 6.1 percent of all other nursing homes. Reasons 

these data were not available include that CMS deemed the data to be unreliable (e.g., very 

high nursing hours per resident per day) or that CMS newly certified the nursing home. 

14See Centers for Medicare & Medicaid Services, Design for Nursing Home Compare FiveStar Quality Rating System: Technical Users’ Guide, revised April 1, 2009, 

http://www.cms.hhs.gov/CertificationandComplianc/Downloads/usersguide.pdf (accessed 

June 17, 2009). 

Page 40 GAO-09-689 The Most Poorly Performing Nursing Homes
Appendix II: Detailed Compliance History for 

Two of the Most Poorly Performing Nursing 

Homes, as of December 2008 

The following table provides the detailed compliance history over three 

cycles for two of the most poorly performing homes in the nation. 

Table 11: Detailed Compliance History over Three Cycles for Two of the Most Poorly Performing Nursing Homes in the 

Nation, as of December 2008 

Example A: Chronic poor performer 

Cycle 1 

10/29/2008 Standard survey 

2 G – I 24 D – F 

G: Proper treatment to prevent or heal pressure sores 

G: Proper care and services for residents with naso-gastric tube 

8/20/2008 Complaint survey 

1 D – F 

5/19/2008 Complaint survey 

1 G – I 1 D – F 

G: Facility is free of accident hazards 

3/25/2008 Complaint survey 

4 D – F 

12/20/2007 Complaint survey 

2 D – F 

Cycle 2 

12/6/2007 Standard survey 

7 D – F 

8/31/2007 Complaint survey 

1 D – F 

8/1/2007 Complaint survey 

2 G – I 2 D – F 

G: Proper treatment to prevent or heal pressure sores 

G: Facility provides sufficient fluid intake 

7/3/2007 Complaint survey 

1 D – F 

6/14/2007 Complaint survey 

1 G – I 1 D – F 

G: Facility is free of accident hazards 

1/26/2007 Complaint survey 

1 G – I 1 D – F 

G: Proper treatment to prevent or heal pressure sores 

Appendix II: Detailed Compliance History for 

Two of the Most Poorly Performing Nursing 

Homes, as of December 2008 

Page 41 GAO-09-689 The Most Poorly Performing Nursing Homes 
Appendix II: Detailed Compliance History for 

Two of the Most Poorly Performing Nursing 

Homes, as of December 2008 

Example A: Chronic poor performer 

Cycle 3a

11/14/2006 Standard survey 

1 G – I 9 D – F 

G: Facility is free of accident hazards 

8/4/2006 Complaint survey 

6 J – L 2 D – F 

L: Facility is administered effectively to obtain highest practicable 

 physical, mental, and psychosocial well-being of each resident

L: Plans to meet emergencies/disasters 

K: Resident not catheterized unless unavoidable 

K: Facility provides sufficient fluid intake 

J: Facility prohibits abuse or neglect 

J: Provides necessary care for highest practicable physical, 

 mental, and psychosocial well-being 

7/14/2006 Complaint survey 

1 G – I 2 D – F 

G: Resident not catheterized unless unavoidable 

6/16/2006 Complaint survey 

1 D – F 

6/8/2006 Complaint survey 

3 J – L 1 G – I 2 D – F 

J: Facility prohibits abuse or neglect 

J: Provides necessary care for highest practicable physical, 

 mental, and psychosocial well-being 

J: Facility is free of accident hazards 

G: Proper treatment to prevent or heal pressure sores 

5/4/2006 Complaint survey 

6 J – L 2 G – I 5 D – F 

L: Facility prohibits abuse or neglect 

L: Facility is administered effectively to obtain highest practicable

 physical, mental, and psychosocial well-being of each resident

K: Provides necessary care for highest practicable physical, 

 mental, and psychosocial well-being 

K: Proper treatment to prevent or heal pressure sores 

J: Facility is free of accident hazards 

J: Proper treatment/care for special care needs 

G: Facility must inform resident, resident’s physician, and family 

 of any accidents; changes in the resident’s physical, mental, or

 psychosocial status, or treatment; or of a decision to transfer 

 or discharge resident 

G: Residents have a right to be free from abuse 

Page 42 GAO-09-689 The Most Poorly Performing Nursing Homes
Appendix II: Detailed Compliance History for 

Two of the Most Poorly Performing Nursing 

Homes, as of December 2008 

Example B: Nonchronic very poor performer

Cycle 1b

8/5/2008 Complaint survey 

2 D – F 

5/8/2008 Complaint survey 

2 G – I 2 D – F 

G: Proper treatment to prevent or heal pressure sores 

G: Accuracy of assessments/coordination with professionals 

4/3/2008 Complaint survey 

5 J – L 2 G – I 1 D – F 

J: Residents’ care supervised by a physician 

J: Facility must inform resident, resident’s physician, and family of

 any accidents; changes in the resident’s physical, mental, or 

 psychosocial status, or treatment; or of a decision to transfer or

 discharge resident 

J: Services by qualified persons in accordance with care plan 

J: Proper treatment to prevent or heal pressure sores 

J: Facility is administered effectively to obtain highest practicable

 physical, mental, and psychosocial well-being of each resident

G: Responsibilities of medical director 

G: Physician promptly notified of lab results 

1/25/2008 Standard survey 

2 J – L 10 D – F 

J: Services by qualified persons in accordance with care plan 

J: Proper treatment to prevent or heal pressure sores 

Cycle 2 

11/28/2007 Complaint survey 

2 D – F 

4/25/2007 Standard survey 

5 D – F 

Cycle 3 

5/18/2006 Standard survey 

5 D – F 

Source: GAO analysis of CMS data. 

Note: The descriptions of standards are provided only for deficiencies cited at the G level or higher. 

The descriptions of the standards have been abbreviated from those that appear in CMS guidance. 

a

The chronic poor performer had one revisit that contributed to its total score during cycle 3. 

b

The nonchronic, very poor performer had one revisit that contributed to its total score during cycle 1. 

Page 43 GAO-09-689 The Most Poorly Performing Nursing Homes
Appendix III: Performance by Standard Area 

for the Most Poorly Performing and All Other Appendix III: Performance by Standard Area Nursing Homes, as of December 2008 

for the Most Poorly Performing and All Other 

Nursing Homes, as of December 2008 

The following table provides the percentages of the most poorly 

performing and all other nursing homes that were cited for actual harm or 

immediate jeopardy by standards area over three cycles. 

Table 12: Percentage of the Most Poorly Performing and All Other Nursing Homes Cited for Actual Harm or Immediate 

Jeopardy, by Standards Area, in the Last Three Cycles as of December 2008 

Percentage with at least one actual 

harm deficiency 

(G through I level) 

Percentage with at least one 

immediate jeopardy deficiency 

(J through L level) 

Standards area 

Most poorly 

performing 

nursing homes

All other 

nursing homes

Most poorly 

performing 

nursing homes

All other 

nursing homes

Administration 15.5 2.0 31.4 2.5

Admission, Transfer, and Discharge Rights 0.5a 0.2a 0.2a 0.0a

Dental Services 1.0 0.1 0.0a0.0a

Dietary Services 0.9 0.1 4.3 0.4

Infection Control 2.1 0.3 2.8 0.2

Nursing Services 5.9 0.5 4.5 0.1

Pharmacy Services 2.6 0.4 3.4 0.3

Physical Environment 1.2 0.1 3.1 0.3

Physicians Services 1.4 0.3 0.7 0.0

Quality of Care 89.7 42.1 67.2 9.1

Quality of Life 17.4 2.4 4.3 0.2

Resident Assessment 20.3 5.8 16.0 1.3

Resident Behavior and Facility Practices 30.9 5.9 28.8 2.4

Resident Rights 18.8 3.7 13.3 1.0

Specialized Rehabilitative Services 1.9 0.1 0.5 0.0

Source: GAO analysis of CMS data. 

Note: Unless otherwise noted, all differences between groups are significant at the 0.05 level. 

a

The difference between the most poorly performing and all other nursing homes for this variable is 

not significant. 

Page 44 GAO-09-689 The Most Poorly Performing Nursing Homes 
Appendix IV: Comments from the Centers for Appendix IV: Comments from the Centers for Medicare & Medicaid Services 

Medicare & Medicaid Services 

Page 45 GAO-09-689 The Most Poorly Performing Nursing Homes
Appendix IV: Comments from the Centers for 

Medicare & Medicaid Services 

Page 46 GAO-09-689 The Most Poorly Performing Nursing Homes
Appendix IV: Comments from the Centers for 

Medicare & Medicaid Services 

Page 47 GAO-09-689 The Most Poorly Performing Nursing Homes
Appendix IV: Comments from the Centers for 

Medicare & Medicaid Services 

Page 48 GAO-09-689 The Most Poorly Performing Nursing Homes
Appendix V: 

A

GAO Contact and Staff 

cknowledgments 

Page 49 GAO-09-689 

Appendix V: GAO Contact and Staff 

Acknowledgments 

John E. Dicken, (202) 512-7114 or dickenj@gao.gov. GAO Contact 

In addition to the contact named above, Walter Ochinko, Assistant 

Director; Ramsey Asaly; Daniel Lee; Shannon Slawter Legeer; Jessica 

Morris; Jessica Nysenbaum; Dae Park; Roseanne Price; Jennifer Rellick; 

Kathryn Richter; and Jessica Smith made key contributions to this report. 

Acknowledgments 

The Most Poorly Performing Nursing Homes 
Related GAO Products Related GAO Products 

Medicare and Medicaid Participating Facilities: CMS Needs to 

Reexamine Its Approach for Funding State Oversight of Health Care 

Facilities. GAO-09-64. Washington, D.C.: February 13, 2009. 

Nursing Homes: Federal Monitoring Surveys Demonstrate Continued 

Understatement of Serious Care Problems and CMS Oversight 

Weaknesses. GAO-08-517. Washington, D.C.: May 9, 2008. 

Nursing Home Reform: Continued Attention Is Needed to Improve 

Quality of Care in Small but Significant Share of Homes. GAO-07-794T. 

Washington, D.C.: May 2, 2007. 

Nursing Homes: Efforts to Strengthen Federal Enforcement Have Not 

Deterred Some Homes from Repeatedly Harming Residents. GAO-07-241. 

Washington, D.C.: March 26, 2007. 

Nursing Homes: Despite Increased Oversight, Challenges Remain in 

Ensuring High-Quality Care and Resident Safety. GAO-06-117. 

Washington, D.C.: December 28, 2005. 

Nursing Home Quality: Prevalence of Serious Problems, While 

Declining, Reinforces Importance of Enhanced Oversight. GAO-03-561. 

Washington, D.C.: July 15, 2003. 

Nursing Homes: Public Reporting of Quality Indicators Has Merit, but 

National Implementation Is Premature. GAO-03-187. Washington, D.C.: 

October 31, 2002. 

Nursing Homes: Federal Efforts to Monitor Resident Assessment Data 

Should Complement State Activities. GAO-02-279. Washington, D.C.: 

February 15, 2002. 

Nursing Homes: Sustained Efforts Are Essential to Realize Potential 

of the Quality Initiatives. GAO/HEHS-00-197. Washington, D.C.: 

September 28, 2000. 

Nursing Home Care: Enhanced HCFA Oversight of State Programs 

Would Better Ensure Quality. GAO/HEHS-00-6. Washington, D.C.: 

November 4, 1999. 

Nursing Home Oversight: Industry Examples Do Not Demonstrate That 

Regulatory Actions Were Unreasonable. GAO/HEHS-99-154R. Washington, 

D.C.: August 13, 1999. 

Page 50 GAO-09-689 The Most Poorly Performing Nursing Homes
Related GAO Products 

Nursing Homes: Proposal to Enhance Oversight of Poorly Performing 

Homes Has Merit. GAO/HEHS-99-157. Washington, D.C.: June 30, 1999. 

Nursing Homes: Complaint Investigation Processes Often Inadequate to 

Protect Residents. GAO/HEHS-99-80. Washington, D.C.: March 22, 1999. 

Nursing Homes: Additional Steps Needed to Strengthen Enforcement 

of Federal Quality Standards. GAO/HEHS-99-46. Washington, D.C.: 

March 18, 1999. 

California Nursing Homes: Care Problems Persist Despite Federal and 

State Oversight. GAO/HEHS-98-202. Washington, D.C.: July 27, 1998. 

(290693) Page 51 GAO-09-689 The Most Poorly Performing Nursing Homes
GAO’s Mission The Government Accountability Office, the audit, evaluation, and 

investigative arm of Congress, exists to support Congress in meeting its 

constitutional responsibilities and to help improve the performance and 

accountability of the federal government for the American people. GAO 

examines the use of public funds; evaluates federal programs and policies; 

and provides analyses, recommendations, and other assistance to help 

Congress make informed oversight, policy, and funding decisions. GAO’s 

commitment to good government is reflected in its core values of 

accountability, integrity, and reliability. 

The fastest and easiest way to obtain copies of GAO documents at no cost 

is through GAO’s Web site (www.gao.gov). Each weekday afternoon, GAO 

posts on its Web site newly released reports, testimony, and 

correspondence. To have GAO e-mail you a list of newly posted products, 

go to www.gao.gov and select “E-mail Updates.” 

Obtaining Copies of 

GAO Reports and 

Testimony 

Order by Phone The price of each GAO publication reflects GAO’s actual cost of 

production and distribution and depends on the number of pages in the 

publication and whether the publication is printed in color or black and 

white. Pricing and ordering information is posted on GAO’s Web site, 

http://www.gao.gov/ordering.htm. 

Place orders by calling (202) 512-6000, toll free (866) 801-7077, or 

TDD (202) 512-2537. 

Orders may be paid for using American Express, Discover Card, 

MasterCard, Visa, check, or money order. Call for additional information. 

Contact: 

Web site: www.gao.gov/fraudnet/fraudnet.htm

E-mail: fraudnet@gao.gov

Automated answering system: (800) 424-5454 or (202) 512-7470 

Ralph Dawn, Managing Director, dawnr@gao.gov, (202) 512-4400 

U.S. Government Accountability Office, 441 G Street NW, Room 7125 

Washington, DC 20548 

To Report Fraud, 

Waste, and Abuse in 

Federal Programs 

Congressional 

Relations 

Chuck Young, Managing Director, youngc1@gao.gov, (202) 512-4800 

U.S. Government Accountability Office, 441 G Street NW, Room 7149 

Washington, DC 20548 

Public Affairs 

Please Print on Recycled Paper