United States Government Accountability Office

GAO Report to Congressional Requesters

NURSING HOMES

Federal Monitoring 

Surveys Demonstrate 

Continued 

Understatement of 

Serious Care 

Problems and CMS 

Oversight Weaknesses

May 2008 

 

GAO-08-517 
What GAO Found

United States Government Accountability Office

Why GAO Did This Study

Highlights

Accountability Integrity Reliability

May 2008

NURSING HOMES

Federal Monitoring Surveys Demonstrate Continued 

Understatement of Serious Care Problems and CMS

Highlights of GAO-08-517, a report to Oversight Weaknesses

congressional requesters 

GAO reports since 1998 have

demonstrated that state surveyors, 

who evaluate the quality of nursing

home care on behalf of CMS, 

sometimes understate the extent of

serious care problems in homes 

because they miss deficiencies.

CMS oversees the effectiveness of 

state surveys through the federal 

monitoring survey program. In this 

program, federal surveyors in 

CMS’s regional offices either 

independently evaluate state

surveys by resurveying a home 

(comparative surveys) or directly 

observe state surveyors during a 

routine nursing home survey 

(observational surveys). GAO was 

asked to evaluate the information 

federal monitoring surveys provide 

on understatement and the 

effectiveness of CMS management 

and oversight of the survey 

program. To do this, GAO analyzed 

the results of federal monitoring 

surveys for fiscal years 2002

through 2007, reviewed CMS 

guidance for the survey program, 

and interviewed headquarters and 

regional office officials. 

What GAO Recommends

GAO is making four

recommendations to the CMS 

Administrator to address 

weaknesses in CMS’s management

of the federal monitoring survey 

database that affect the agency’s 

ability to track understatement and 

CMS’s ability to oversee regional

office implementation of the 

federal monitoring survey program. 

In its comments on a draft of this 

report, HHS fully endorsed and 

indicated it would implement 

GAO’s recommendations. 

A substantial proportion of federal comparative surveys identify missed 

deficiencies at the potential for more than minimal harm level or above.

During fiscal years 2002 through 2007, about 15 percent of federal 

comparative surveys nationwide identified state surveys that failed to cite at 

least one deficiency at the most serious levels of noncompliance—actual harm 

and immediate jeopardy. Overall, nine states missed serious deficiencies on 

25 percent or more of comparative surveys; in seven states federal surveyors 

identified no such missed deficiencies. During the same period, missed 

deficiencies at the lowest level of noncompliance—the potential for more than 

minimal harm—were more widespread: nationwide, approximately 70 percent 

of federal comparative surveys identified state surveys missing at least one 

deficiency at the lowest level of noncompliance, and in all but five states the 

number of state surveys with such missed deficiencies was greater than 

40 percent. Undetected care problems at this level are a concern because they

could become more serious if nursing homes are not required to take 

corrective action. The most frequently missed type of deficiency on 

comparative surveys, at the potential for more than minimal harm level and 

above, was poor quality of care, such as ensuring proper nutrition and 

hydration and preventing pressure sores. Federal observational surveys 

highlighted two factors that may contribute to understatement of deficiencies: 

weaknesses in state surveyors’ (1) investigative skills and 

(2) ability to integrate and analyze information collected to make an 

appropriate deficiency determination. These factors may contribute to 

understatement because they directly affect the appropriate identification and 

citation of deficiencies. 

CMS has taken steps to improve the federal monitoring survey program, but 

weaknesses remain in program management and oversight. For example, CMS 

has improved processes to ensure that comparative surveys more accurately

reflect conditions at the time of the state survey, such as requiring that 

comparative surveys occur within 30 working days of the state survey rather 

than within the 2 months set in statute. Despite these improvements, the 

management and oversight potential of the program has not been fully 

realized. For example, CMS has only begun to explore options for identifying 

understatement that occurs in cases where state surveys cite deficiencies at 

too low a level, for possible implementation in fiscal year 2009. In addition, 

CMS is not effectively managing the federal monitoring survey database to 

ensure that the regional offices are entering data accurately and reliably—

CMS was unaware, for example, that a considerable number of comparative 

surveys had not been entered. Furthermore, CMS is not using the database to

oversee consistent implementation of the program by the regional offices—for 

example, the agency is not using the database to identify inconsistencies

between comparative and observational survey results. 

To view the full product, including the scope 

and methodology, click on GAO-08-517. 

For more information, contact John E. Dicken 

at (202) 512-7114 or dickenj@gao.gov. 
Contents 

Letter 1

Results in Brief 4 

Background 6

Substantial Proportion of Federal Comparative Surveys Identify 

Missed Deficiencies 11 

CMS Has Taken Steps to Improve the Federal Monitoring Survey 

Program, but Weaknesses in Management and Oversight Remain 19 

Conclusions 27

Recommendations for Executive Action 28 

Agency Comments 28 

Appendix I Percentage of Nursing Homes Cited for Actual 

Harm or Immediate Jeopardy during Standard 

Surveys 31 

Appendix II Percentage of Comparative Surveys Identifying 

Missed Deficiencies at Actual Harm or Immediate 

Jeopardy Level 33 

Appendix III Percentage of Comparative Surveys Identifying 

Missed Deficiencies with Potential for More Than 

Minimal Harm 35 

Appendix IV Percentage of Comparative Surveys with at Least 

One Missed Deficiency, by Federal Quality 

Standard Category 37 

Appendix V Percentage of Below Satisfactory State Survey 

Ratings for General Investigation and Deficiency 

Determination 38 

Page i GAO-08-517 Federal Monitoring Surveys for Nursing Homes
Appendix VI Comments from the Department of Health & 

Human Services 40 

Appendix VII GAO Contact and Staff Acknowledgments 49 

Related GAO Products 50 

Tables 

Table 1: Scope and Severity of Deficiencies Identified during 

Nursing Home Surveys 3 

Table 2: States with 25 Percent or More of Comparative Surveys 

Identifying Missed Deficiencies at the Actual Harm or 

Immediate Jeopardy Levels, Fiscal Years 2002 through 

2007 13 

Table 3: States with 25 Percent or More of Comparative Surveys 

with Missed Deficiencies and Percentage of Their 

Observational Surveys with Less Than Satisfactory Ratings 

on General Investigation and Deficiency Determination, 

Fiscal Years 2002 through 2007 18 

Table 4: Percentage of Nursing Homes Cited for Actual Harm or 

Immediate Jeopardy, by State, Fiscal Years 2002 through 

2007 31 

Figures 

Figure 1: Percentage of Comparative Surveys Nationwide Citing at 

Least One Missed Deficiency at the Actual Harm or 

Immediate Jeopardy Level, Fiscal Years 2002 through 2007 12 

Figure 2: National Percentage of Comparative Surveys Citing at 

Least One Missed Deficiency at the Potential for More 

Than Minimal Harm Level, Fiscal Years 2002 through 2007 15 

Figure 3: Percentage of Comparative Surveys Nationwide with 

Understatement of Actual Harm and Immediate Jeopardy 

Deficiencies, with Scope and Severity Differences

Included, Fiscal Years 2002 through 2007 23 

Page ii GAO-08-517 Federal Monitoring Surveys for Nursing Homes
Abbreviations 

CMS Centers for Medicare & Medicaid Services 

HHS Department of Health & Human Services 

OSCAR On-Line Survey, Certification, and Reporting system 

This is a work of the U.S. government and is not subject to copyright protection in the 

United States. The published product may be reproduced and distributed in its entirety 

without further permission from GAO. However, because this work may contain 

copyrighted images or other material, permission from the copyright holder may be 

necessary if you wish to reproduce this material separately. 

Page iii GAO-08-517 Federal Monitoring Surveys for Nursing Homes
United States Government Accountability Office

Washington, DC 20548 

May 9, 2008

The Honorable Herb Kohl 

Chairman 

Special Committee on Aging 

United States Senate 

The Honorable Charles E. Grassley 

Ranking Member 

Committee on Finance 

United States Senate 

Since 1998, Congress has focused considerable attention on the need to 

improve the quality of care for the nation’s 1.5 million nursing home 

residents, a highly vulnerable population of elderly and disabled 

individuals for whom remaining at home is no longer feasible. Poor quality 

of care—worsening pressure sores or untreated weight loss—in a small 

but unacceptably high number of nursing homes continues to harm 

residents or place them in immediate jeopardy, that is, at risk of death or 

serious injury. About 1 in 5 homes nationwide were cited for such serious 

deficiencies on state inspections, known as surveys, in fiscal year 2007 

(see app. I). Our previous work, however, demonstrated that state surveys 

sometimes understated the extent of serious care problems and that 

federal oversight of state survey activities had weaknesses.1

Understatement can occur when a state surveyor fails to cite a deficiency

altogether or cites a deficiency at too low a level. 

 1

See GAO, California Nursing Homes: Care Problems Persist Despite Federal and State 

Oversight, GAO/HEHS-98-202 (Washington, D.C.: July 27, 1998); Nursing Home Care:

Enhanced HCFA Oversight of State Programs Would Better Ensure Quality, 

GAO/HEHS-00-06 (Washington, D.C.: Nov. 4, 1999); and Nursing Home Quality:

Prevalence of Serious Problems, While Declining, Reinforces Importance of Enhanced

Oversight, GAO-03-561 (Washington, D.C.: July 15, 2003). A list of related GAO products is

at the end of this report. 

Page 1 GAO-08-517 Federal Monitoring Surveys for Nursing Homes
The Centers for Medicare & Medicaid Services (CMS) is the federal agency 

responsible for ensuring the effectiveness of state surveys.2

 Through 

CMS’s federal monitoring survey program, federal surveyors either 

(1) independently evaluate state surveys by resurveying a home recently 

inspected by state surveyors and comparing the deficiencies identified 

during the two surveys, known as a comparative survey, or (2) directly 

observe state surveyors during a routine nursing home survey, known as 

an observational survey. Results from both federal comparative and 

observational surveys—which are recorded in the federal monitoring 

survey database—allow CMS to gauge states’ abilities to accurately assess 

nursing home quality. 

Recently, we reported that federal comparative surveys in five large states 

identified the continuing understatement of serious care problems by state 

surveyors.3

 You asked us to look at the understatement of serious 

deficiencies by state surveyors nationwide. In this report we address two 

questions: (1) what information do federal monitoring surveys provide 

about understatement nationwide, and (2) how effective are CMS 

management and oversight of the federal monitoring survey program? 

To answer the first question, we analyzed the results of comparative and 

observational surveys nationwide for fiscal years 2002 through 2007 using

the federal monitoring survey database.4

 During this period, federal 

surveyors conducted 976 comparative surveys and 4,023 observational 

 2

All homes that participate in the Medicare and Medicaid programs are subject to periodic 

surveys to ensure that they are in compliance with federal quality standards. CMS contracts

with state survey agencies to conduct the surveys. Medicare, the federal health care 

program for elderly and disabled individuals, covers up to 100 days of skilled nursing home 

care following a hospital stay. Medicaid, the joint federal-state health care financing 

program for certain categories of low-income individuals, pays for the nursing home care

of qualifying individuals who can no longer live at home. Combined Medicare and Medicaid 

payments for nursing home services were about $75 billion in 2006, including a federal 

share of about $51 billion. 

3

See GAO, Despite Increased Oversight, Challenges Remain in Ensuring High-Quality

Care and Resident Safety, GAO-06-117 (Washington, D.C.: Dec. 28, 2005); and Nursing 

Home Reform: Continued Attention Is Needed to Improve Quality of Care in Small but 

Significant Share of Homes, GAO-07-794T (Washington, D.C.: May 2, 2007). In both 

reports, we examined understatement for five states—California, Florida, New York, Ohio,

and Texas. Results from the May 2007 report showed that understatement of serious 

deficiencies varied across these states from a low of 4 percent in Ohio to a high of 

26 percent in New York during the 5-year period March 2002 through March 2007. 

4

Fiscal year 2002 was the first year that the database contained all the information needed 

to assess the results of federal comparative surveys. 

Page 2 GAO-08-517 Federal Monitoring Surveys for Nursing Homes
surveys. To identify understatement on comparative surveys, we focused 

our analysis on cases where federal surveyors determined that state 

surveyors should have cited a deficiency but failed to do so or where state 

surveyors cited a deficiency at too low a level. We analyzed the results of 

observational surveys in order to better understand why understatement 

might occur. Deficiencies identified during nursing home surveys are 

categorized according to their scope (i.e., the number of residents 

potentially or actually affected) and severity (i.e., the degree of relative 

harm involved). Homes with deficiencies at the A though C levels are 

considered to be in substantial compliance, while those with deficiencies 

at the D through L levels are considered out of compliance. (See table 1.) 

Throughout this report, we refer to deficiencies at the actual harm and 

immediate jeopardy levels as serious deficiencies. 

Table 1: Scope and Severity of Deficiencies Identified during Nursing Home 

Surveys 

Scope

Severity Isolated Pattern Widespread

Immediate jeopardyaJ K L

Actual harm G H I 

Potential for more than minimal harm D E F

Potential for minimal harmbA B C

Source: CMS. 

a

Actual or potential for death/serious injury. 

b

Nursing home is considered to be in substantial compliance. 

To ensure reliability of the federal monitoring survey database, we 

discussed data entry procedures with all 10 CMS regional offices, whose 

staff enter information into the database. In addition, we conducted 

several data reliability tests, including (1) automated checks of data fields 

to ensure that they contained complete information and (2) manual 

reviews of a random sample of all deficiencies cited by federal but not 

state surveyors to ensure that federal surveyors had used the data fields 

appropriately. We also eliminated a small number of deficiencies that did 

not correspond to a defined severity level or contained illogical survey 

dates—such as a comparative survey that began prior to the state survey.

Based on these activities, we determined that the information was 

sufficiently reliable for our purposes. Data on comparative surveys, 

however, cannot be used to project the extent of understatement across all 

state surveys because the state surveys selected for federal monitoring 

Page 3 GAO-08-517 Federal Monitoring Surveys for Nursing Homes
surveys are not representative of all nursing home surveys or survey teams 

within each state. 

To answer the second question, we reviewed CMS guidance for the federal 

monitoring survey program and interviewed officials in CMS headquarters 

and all 10 regional offices. Our work focused on CMS’s (1) efforts to 

improve the use of comparative surveys as an oversight tool; (2) ability to 

track the understatement of deficiencies; and (3) management of the 

federal monitoring survey database, including the use of the database to 

oversee regional office implementation of the federal monitoring survey 

program. We also analyzed (1) the comments entered into the federal 

monitoring survey database by federal surveyors for certain discrepancies

between federal and state survey findings and (2) the consistency between 

comparative and observational survey results within states and CMS 

regional offices. We performed our work from July 2007 through May 2008 

in accordance with generally accepted government auditing standards. 

A substantial proportion of federal comparative surveys identify missed 

deficiencies at the potential for more than minimal harm level or above. 

From fiscal year 2002 through 2007, about 15 percent of federal 

comparative surveys nationwide identified state surveys that failed to cite 

at least one deficiency at the most serious levels of noncompliance—the 

actual harm and immediate jeopardy levels. Overall, in nine states federal 

surveyors identified missed serious deficiencies on 25 percent or more of 

comparative surveys, but in seven states they identified no missed serious 

deficiencies. During the same period, missed deficiencies at the potential

for more than minimal harm level were more widespread: nationwide, 

approximately 70 percent of federal comparative surveys identified state 

surveys missing at least one deficiency at the potential for more than 

minimal harm level, and in all but five states the number of state surveys 

with such missed deficiencies was greater than 40 percent. Such 

undetected care problems are of concern because they could become 

more serious over time if nursing homes are not required to take 

corrective actions. The most frequently missed deficiencies identified on

comparative surveys—from the potential for more than minimal harm 

through immediate jeopardy levels—involved poor quality of care, such as 

ensuring proper nutrition and hydration and preventing pressure sores. 

Federal observational surveys highlighted two factors that may contribute 

to the understatement of deficiencies—weaknesses in state surveyors’ 

investigative skills and in their ability to integrate and analyze the 

information collected to make an appropriate deficiency determination. 

Six of the nine states that missed serious deficiencies on 25 percent or 

Results in Brief 

Page 4 GAO-08-517 Federal Monitoring Surveys for Nursing Homes
more of comparative surveys had observational survey ratings for these 

two dimensions that were worse than the national average. 

CMS has taken steps to improve the federal monitoring survey program, 

but weaknesses remain in program management and oversight. For 

example, CMS has improved processes to ensure that comparative surveys 

more accurately reflect conditions at the time of the state survey, such as 

requiring that comparative surveys occur within 30 working days of the 

state survey rather than within the 2 months set in statute. However, 

despite these improvements, the management and oversight potential of 

the program has not been fully realized. First, we found that the federal

monitoring survey database did not capture the full extent of 

understatement because CMS does not require regional offices to 

determine when state surveyors cite a deficiency at too low a level. The 

agency has only begun exploring options for identifying potential scope 

and severity understatement. When we manually analyzed optional 

comment fields in the database to assess how often a deficiency was cited

at too low a level, we were able to confirm this type of understatement in 

38 percent of the cases we examined; when combined with 

understatement caused by missed deficiencies, overall understatement of 

serious deficiencies by state surveyors increased from about 15 percent to

about 16 percent for fiscal years 2002 through 2007. Second, we found that 

CMS was not effectively managing the database to ensure that the regional

offices were entering data accurately and reliably. CMS was unaware, for 

example, that a considerable number of comparative surveys had not been 

entered. Finally, we found weaknesses in CMS’s use of the database for 

regional office oversight. For example, despite the fact that 

inconsistencies in comparative and observational survey findings in 

specific states could indicate that some CMS regional offices did not 

follow CMS guidance in assessing state surveyor performance, CMS 

officials told us that they did not plan to follow up with regional offices 

about these inconsistencies. 

We are making four recommendations to the CMS Administrator to 

address weaknesses in CMS’s management and oversight of federal 

monitoring surveys. We recommend that CMS (1) require regional offices 

to determine if there was understatement when state surveyors cite a 

deficiency at a lower scope and severity level than federal surveyors and 

to track this information, (2) establish quality controls to improve the 

accuracy and reliability of information entered into the federal monitoring 

survey database, (3) routinely examine comparative survey data and hold

regional offices accountable for implementing CMS guidance that is 

intended to ensure that comparative surveys more accurately capture the 

Page 5 GAO-08-517 Federal Monitoring Surveys for Nursing Homes
conditions at the time of the state survey, and (4) regularly analyze and 

compare federal comparative and observational survey results. We 

provided a draft of this report to CMS through the Department of Health & 

Human Services (HHS). The comments we received were submitted by 

HHS. HHS fully endorsed and indicated it would implement our 

recommendations. 

Oversight of nursing homes is a shared federal-state responsibility. Based 

on statutory requirements, CMS (1) defines quality standards that nursing 

homes must meet to participate in the Medicare and Medicaid programs 

and (2) contracts with state survey agencies to assess whether homes 

meet those standards through annual surveys and complaint 

investigations.5

 Although CMS has issued extensive guidance to states on 

determining compliance with federal quality requirements, we have found

that some state surveys understate quality problems at nursing homes.6

Federal nursing home quality standards focus on the delivery of care, 

resident outcomes, and facility conditions. These standards, totaling 

approximately 200, are grouped into 15 categories, such as Resident 

Rights, Quality of Life, Resident Assessment, Quality of Care, Pharmacy 

Background 

Federal Quality Standards 

 5

In addition to nursing homes, CMS and state survey agencies are responsible for oversight

of other Medicare and Medicaid providers, such as home health agencies, intermediate care 

facilities for the mentally retarded, and hospitals.

6

Several GAO reports have documented understatement of serious deficiencies by state 

surveyors. For one report, in 1998, we arranged for a team of registered nurses to 

accompany state surveyors and conduct concurrent surveys designed specifically to 

identify quality-of-care problems. The survey methodology we used differed from the

methodology used by state surveyors in that it was more rigorous and reviewed a larger

sample of cases. Using this methodology, our surveys spotted cases in which the homes

had not intervened appropriately for residents experiencing weight loss, dehydration,

pressure sores, and incontinence—cases the state surveyors either missed or identified as 

affecting fewer residents. (See GAO/HEHS-98-202.) In addition, we have documented 

considerable interstate variation in the proportion of homes cited for serious care 

problems. For example, 8 percent of Florida’s 683 homes and 38 percent of Connecticut’s

244 homes were cited for serious care problems in fiscal year 2007. The extent of this 

variation suggests inconsistency in how states conduct surveys and understatement of

serious quality problems. App. I shows the proportion of homes in each state cited by state 

surveyors for serious deficiencies from fiscal year 2002 through 2007. 

Page 6 GAO-08-517 Federal Monitoring Surveys for Nursing Homes
Services, and Administration.7

 For example, there are 23 standards within 

the Quality of Care category ranging from “promote the prevention of 

pressure [sore] development” to “the resident environment remains as free 

of accident hazards as is possible.” CMS has also developed detailed 

investigative protocols to assist state survey agencies in determining 

whether nursing homes are in compliance with federal quality standards. 

This guidance is intended to ensure the thoroughness and consistency of 

state surveys and complaint investigations. 

Every nursing home receiving Medicare or Medicaid payment must 

undergo a standard state survey not less than once every 15 months, and 

the statewide average interval for these surveys must not exceed 

12 months. During a standard survey, teams of state surveyors—generally 

consisting of registered nurses, social workers, dieticians, or other 

specialists—evaluate compliance with federal quality standards. Based on 

the care provided to a sample of residents, the survey team (1) determines 

whether the care and services provided meet the assessed needs of the 

residents and (2) measures resident outcomes, such as the incidence of

preventable pressure sores, weight loss, and accidents. In contrast to a 

standard survey, a complaint investigation generally focuses on a specific

allegation regarding a resident’s care or safety and provides an

opportunity for state surveyors to intervene promptly if problems arise 

between standard surveys. Surveyors generally follow state procedures 

when investigating complaints, but must comply with certain federal 

guidelines and time frames. 

When deficiencies are identified, federal sanctions can be imposed to help

encourage homes to correct them. Sanctions are generally reserved for 

serious deficiencies—those at the G through L levels—which constitute 

actual harm and immediate jeopardy.8

 Sanctions for such serious quality 

problems can affect a home’s revenues and provide financial incentives to 

Standard Surveys and 

Complaint Investigations 

Enforcement 

 7

Other areas include Admission, Transfer and Discharge Rights; Resident Behavior and

Facility Practices; Nursing Services; Dietary Services; Physician Services; Specialized 

Rehabilitative Services; Dental Services; Infection Control; and Physical Environment.

Surveys also examine compliance with federal fire safety requirements. 

8

The scope and severity of a deficiency is one of four factors that CMS takes into account 

when imposing sanctions. CMS also considers a home’s prior compliance history, desired 

corrective action and long-term compliance, and the number and severity of all the home’s

deficiencies. 

Page 7 GAO-08-517 Federal Monitoring Surveys for Nursing Homes
return to and maintain compliance. Such sanctions include fines known as 

civil money penalties, denial of payment for new Medicare or Medicaid 

admissions, or termination from the Medicare and Medicaid programs. 

State surveys that miss serious deficiencies or cite deficiencies at too low 

a scope and severity level have enforcement implications because a 

nursing home may escape sanctions intended to discourage repeated 

noncompliance. For example, facilities that receive at least one G through 

L level deficiency on successive standard surveys or complaint 

investigations must be referred for immediate sanctions.9

 In addition, CMS 

guidance calls for higher fines when a home has a poor compliance history 

and requires that state survey teams revisit a home to verify that serious 

deficiencies have actually been corrected (such revisits are not required 

for most deficiencies cited below the actual harm level—A through F).10

CMS Oversight of State 

Surveys 

Statutorily required federal monitoring surveys, which are conducted 

annually in at least 5 percent of state-surveyed Medicare and Medicaid 

nursing homes in each state, are a key CMS oversight tool in ensuring the 

adequacy of state surveys.11 CMS headquarters—specifically, CMS’s Survey 

and Certification Group—is responsible for the management of the federal 

monitoring survey database and for oversight of the 10 CMS regional 

offices’ implementation of the federal monitoring survey program.12

Federal surveyors located in regional offices conduct federal monitoring 

surveys. The surveys can be either comparative or observational, with 

each offering unique advantages and disadvantages as an oversight tool. 

For example, an advantage of comparative surveys is that they are an

independent evaluation of a nursing home recently surveyed by a state 

 9

See GAO, Nursing Homes: Efforts to Strengthen Federal Enforcement Have Not Deterred

Some Homes from Repeatedly Harming Residents, GAO-07-241 (Washington, D.C.:

Mar. 26, 2007). 

10A revisit is required for F level deficiencies that are cited in one of three areas: Quality of 

Care, which can include deficiencies such as inadequate treatment or prevention of

pressure sores; Quality of Life, which can include deficiencies such as a failure to

accommodate the needs and preferences of residents; and Resident Behavior and Facility 

Practices, which can include deficiencies such as a failure to protect residents from abuse. 

11In addition, CMS also conducts annual state performance reviews, which include an

examination of the quality of state survey agency investigations and decision making and 

the timeliness and quality of complaint investigations. 

12Throughout this report, when we refer to CMS headquarters or CMS headquarters 

officials, we mean the Survey and Certification Group and its officials. 

Page 8 GAO-08-517 Federal Monitoring Surveys for Nursing Homes
survey agency team. A disadvantage is that the time lag between the two 

surveys can make analysis of differences difficult.

• Comparative survey. A federal survey team conducts an independent

survey of a home recently surveyed by a state survey agency in order to 

compare and contrast the findings. This comparison takes place after 

completion of the federal survey. When federal surveyors identify a 

deficiency not cited by state surveyors, they assess whether the deficiency

existed at the time of the state survey and should have been cited by 

entering either yes or no to the question, “Based on the evidence available 

to the [state], should the [state survey] team have cited this [deficiency]?” 

This assessment is critical in determining whether understatement 

occurred because some deficiencies cited by federal surveyors may not 

have existed at the time of the state survey. For example, a deficiency

identified during a federal survey could involve a resident who was not in 

the nursing home at the time of the earlier state survey. By statute, 

comparative surveys must be conducted within 2 months of the 

completion of the state survey. However, differences in timing, resident 

sample selection, and staffing can make analysis of differences between 

the state and federal comparative surveys difficult. On the basis of our 

prior recommendations, CMS has taken several steps to ensure that 

comparative surveys more accurately capture conditions at the time of the 

state survey.13 For example, CMS now calls for the length of time between 

the state and federal surveys to be between 10 and 30 working days and 

requires federal surveyors conducting a comparative survey in a nursing 

home to include at least half of the state survey’s sample of residents from 

that nursing home in the comparative survey sample, making it easier to 

determine whether state surveyors missed a deficiency.14 Furthermore, 

federal comparative survey teams are expected to mimic the number of 

staff assigned to the state survey. CMS also issued guidance in October 

2002 defining the criteria for federal surveyors to consider when selecting 

 13See GAO/HEHS-00-6. 

14In December 2001, CMS requested that regional offices conduct comparative surveys 

between 2 weeks and 1 month after the state survey. In October 2002, CMS relaxed this 

standard by changing the requirement to between 10 and 30 working days. In general, it is

easier for federal surveyors to determine whether state surveyors should have identified 

deficiencies when conditions during the comparative survey are as close as possible to 

those existing during the state survey. Reducing the time between state and federal surveys 

and requiring a review of the quality of care provided to as many of the same nursing home

residents as possible enhances the similarities between state and federal surveys. 

Page 9 GAO-08-517 Federal Monitoring Surveys for Nursing Homes
facilities for comparative surveys.15 These selection criteria can generally

be categorized as state survey team performance and facility 

characteristics. Regional offices were given latitude in their use of these 

criteria and may supplement them with other selection factors unique to

their regions. For example, some regions use statistics on the prevalence 

of pressure sores in a nursing home’s resident population as a comparative 

survey selection factor. 

• Observational survey. Federal surveyors accompany a state survey team 

to a nursing home to evaluate the team’s on-site survey performance and 

ability to document survey deficiencies. State teams are evaluated in six 

areas—Concern Identification, Sample Selection, General Investigation,

Food-Borne Illness Investigation, Medication Investigations, and 

Deficiency Determination—and are rated in one of five categories for each 

of the six measures. The rating categories—from highest to lowest—are 

extremely effective, very effective, satisfactory, less than satisfactory, and 

much less than satisfactory. CMS annual state performance reviews 

require that state survey teams achieve an average rating of satisfactory. 

Observational surveys allow federal surveyors to provide more immediate 

feedback to state surveyors and to identify state surveyor training needs. 

However, observational surveys are not independent evaluations of the 

state survey. Because state surveyors may perform their survey tasks more 

attentively than they would if federal surveyors were not present, 

observational surveys may not provide an accurate picture of state 

surveyors’ typical performance. Since 2001, CMS has also taken steps to 

strengthen observational surveys. For example, the agency issued written 

guidance defining a standard process for resolving disagreements and a 

new manual to increase consistency across observational surveys. 

The 976 federal comparative surveys conducted from fiscal year 2002 

through 2007 ranged from as few as 10 in Vermont, which has about 

40 facilities, to as many as 49 in California, which has about 1,300 facilities.

Of the 4,023 federal observational surveys conducted during the same 

period, the number ranged from 16 in New Hampshire to 346 in California.

 15In 1999, we reported that there was little consistency across CMS regional offices in the 

criteria used to select homes for comparative surveys. For example, some regions were 

selecting homes that had no serious deficiencies, while others were focusing on homes 

with serious deficiencies. We noted that federal surveyors were less likely to find missed 

deficiencies at homes where state surveyors found serious care problems. See 

GAO/HEHS-00-6. 

Page 10 GAO-08-517 Federal Monitoring Surveys for Nursing Homes
The results of federal monitoring surveys, including information on the 

corresponding state surveys, are entered in the federal monitoring survey 

database. In fiscal year 2002, CMS began including information on 

comparative surveys in the database, and the agency began requiring 

federal surveyors to determine whether a deficiency cited by federal but 

not state surveyors had been missed by determining whether state 

surveyors should have cited the deficiency. 

Although comparative surveys and the wide variability across states in the 

proportion of homes with deficiencies at the actual harm and immediate 

jeopardy levels indicate that state surveyors miss some serious 

deficiencies, our prior work has also indicated that state surveyors 

sometimes understate the scope and severity of a deficiency. In 2003, we 

found widespread understatement of actual harm deficiencies in a sample 

of surveys from homes with a history of harming residents.16 Overall, 

39 percent of the 76 state surveys we reviewed had documented problems 

that should have been classified as actual harm instead of as lower-level 

deficiencies.

A substantial proportion of federal comparative surveys identify missed 

deficiencies at the potential for more than minimal harm level or above. 

From fiscal year 2002 through 2007, about 15 percent of federal 

comparative surveys nationwide identified state surveys that failed to cite 

at least one deficiency at the most serious levels of noncompliance—the 

actual harm and immediate jeopardy levels (G through L). There was wide 

variation across states in the proportion of comparative surveys that found 

at least one missed serious deficiency, from more than 25 percent in nine 

states to none in seven others. In contrast to missed serious deficiencies,

missed deficiencies at the potential for more than minimal harm level 

(D through F) were considerably more widespread, with such missed 

deficiencies greater than 40 percent in all but five states. Every state had at

least one comparative survey with missed D through F level deficiencies. 

At both levels of noncompliance, the most frequently missed deficiencies 

involved Quality of Care standards. Federal observational survey results 

and prior GAO reports have highlighted several factors that may 

contribute to the understatement of deficiencies.

Understatement of 

Deficiency Scope and 

Severity Level 

Substantial 

Proportion of Federal 

Comparative Surveys 

Identify Missed 

Deficiencies 

 16See GAO-03-561. 

Page 11 GAO-08-517 Federal Monitoring Surveys for Nursing Homes
About 15 percent (142) of the 976 comparative surveys conducted from 

fiscal year 2002 through 2007 identified state surveys that missed at least 

one deficiency at the actual harm or immediate jeopardy level (G through 

L), the most serious levels of noncompliance. This proportion fluctuated 

from a high of 17.5 percent in fiscal year 2003 to a low of 11.1 percent in 

fiscal year 2004, but it has remained relatively constant at about 15 percent 

for the last several fiscal years (see fig. 1). This proportion is small, but 

CMS maintains that any missed serious deficiencies are unacceptable.

Figure 1: Percentage of Comparative Surveys Nationwide Citing at Least One 

Missed Deficiency at the Actual Harm or Immediate Jeopardy Level, Fiscal Years 

2002 through 2007

From fiscal year 2002 through 2007, federal surveyors identified missed 

serious deficiencies in 25 percent or more of their comparative surveys in

nine states. The proportion of missed serious deficiencies in these nine 

states ranged from 26.3 percent in Tennessee to 33.3 percent in New 

Mexico, South Carolina, South Dakota, and Wyoming (see table 2).17 The 

Small but Unacceptably 

High Proportion of Federal 

Comparative Surveys 

Found That State Survey 

Teams Missed Serious 

Deficiencies 

Example of a Missed Deficiency at the 

Actual Harm Level

On a fiscal year 2006 comparative survey, 

federal surveyors found a G-level deficiency 

that the state survey team had not cited at 

any level. After verifying that the affected 

resident was part of the state survey sample 

and the evidence used by federal surveyors

was available at the time of the state survey, 

the federal surveyors commented that “The 

resident had an avoidable, unplanned weight 

loss of 11 [percent] in [6] months from June 

to November 9, 2005. There was no 

nutritional assessment that addressed the

11 percent weight loss for November 2005 

and the physician was not aware of the 

resident’s weight loss.” 

0

2

4

6

8

10

12

14

16

18

20

2002 2003 2004 2005 2006 2007

Percentage of comparative surveys

Source: GAO analysis of federal monitoring survey data.

Fiscal year

13.8

17.5

11.1

14.3

15.3 15.3

 17We examined missed deficiencies by state for each fiscal year from 2002 through 2007 and

found that for most states the failure to cite deficiencies at the actual harm and immediate 

jeopardy levels was not isolated to a single year during the 6 fiscal years we examined, and 

it continued to be a problem for many states in fiscal year 2007. 

Page 12 GAO-08-517 Federal Monitoring Surveys for Nursing Homes
total number of missed deficiencies at the G through L levels also varied 

across these nine states, from a low of 4 in South Dakota to a high of 19 in

South Carolina. Federal surveyors identified no missed serious 

deficiencies in seven states (see app. II for complete state results).18

Table 2: States with 25 Percent or More of Comparative Surveys Identifying Missed

Deficiencies at the Actual Harm or Immediate Jeopardy Levels, Fiscal Years 2002 

through 2007 

State 

Number 

of homes

in fiscal 

year 2007

Total

comparative 

surveys

Total

comparative 

surveys with

at least one

missed G-L

deficiency 

Percentage

of total 

comparative 

surveys with

at least one

missed G-L

deficiency

Total

number

missed G-L

deficiencies

New Mexico 72 12 4 33.3 9

South Carolina 176 18 6 33.3 19

South Dakota 112 12 4 33.3 4

Wyoming 39 12 4 33.3 5

Oklahoma 348 20 6 30.0 11

Missouri 530 28 8 28.6 14

Alabama 233 18 5 27.8 13

Arizona 137 15 4 26.7 6

Tennessee 332 19 5 26.3 10

Source: GAO analysis of federal monitoring survey data. 

 18Alaska, Idaho, Maine, North Dakota, Oregon, Vermont, and West Virginia had no missed 

serious deficiencies. 

Page 13 GAO-08-517 Federal Monitoring Surveys for Nursing Homes
In contrast to missed serious deficiencies, missed deficiencies at the 

potential for more than minimal harm level (D through F) were 

considerably more widespread on comparative surveys conducted during 

fiscal years 2002 through 2007. Approximately 70 percent of comparative

surveys conducted nationwide identified state surveys that missed at least 

one deficiency at the potential for more than minimal harm level (D 

through F), with such missed deficiencies identified on greater than 

40 percent of comparative surveys in all but five states—Alaska, Ohio, 

Vermont, West Virginia, and Wisconsin.19 On average, state surveys 

selected for comparative surveys failed to identify 2.5 D through F level 

deficiencies per survey. Undetected care problems at the D through F level 

are of concern because they could become more serious over time if 

nursing homes are not required to take corrective actions. Missed 

deficiencies at the potential for more than minimal harm level were not 

isolated to a single year during the 6 fiscal years we examined and 

continued to be a problem for states in fiscal year 2007. Nationally, the 

proportion of comparative surveys identifying at least one missed D 

through F level deficiency in fiscal year 2007 was about 74 percent (see 

fig. 2). For results by state, see appendix III. 

Missed Deficiencies at the 

Potential for More Than 

Minimal Harm Level Were 

Widespread on Federal 

Comparative Surveys 

Example of a Missed Deficiency at the 

Potential for More Than Minimal Harm 

Level

On a fiscal year 2007 comparative survey, 

federal surveyors found an F-level 

deficiency—that is, a deficiency widespread 

throughout the facility—that the state survey 

team had not cited at any level. After verifying 

that the evidence used by federal surveyors

was available at the time of the state survey, 

the federal surveyors commented that 

“Deficiency [is] based on a systemic lack of 

monthly drug reviews dating back to June 

2006. [State surveyors] would have been 

expected to cite this deficient practice.” 

 19This finding was consistent with the overall prevalence of D through F level deficiencies

cited by state survey teams during annual standard surveys. Approximately 84 percent of

all deficiencies identified during these surveys in 2006 were at the D through F level. In 

contrast, only about 5 percent of deficiencies cited on state surveys were at the actual 

harm and immediate jeopardy (G through L) levels. 

Page 14 GAO-08-517 Federal Monitoring Surveys for Nursing Homes
Figure 2: National Percentage of Comparative Surveys Citing at Least One Missed

Deficiency at the Potential for More Than Minimal Harm Level, Fiscal Years 2002 

through 2007 

Our analysis found that the most frequently missed deficiencies at both the 

potential for more than minimal harm (D through F) and the actual harm 

or immediate jeopardy (G through L) levels occurred in quality standards 

under CMS’s Quality of Care category. Missed deficiencies in this category 

involved residents’ receipt of the necessary care and services to attain and 

maintain the highest practicable physical, mental, and psychosocial wellbeing—such as prevention of pressure sores, nutrition and hydration, 

accident prevention, and assistance with bathing and grooming. 

From fiscal year 2002 through 2007, 11.9 percent of federal comparative 

surveys (116) cited at least one Quality of Care deficiency at the actual 

harm or immediate jeopardy level that state survey teams failed to cite. 

These 116 surveys contained a total of 143 missed serious Quality of Care

deficiencies. The category with the next highest frequency of missed 

serious deficiencies was Resident Behavior and Facility Practices, with 

0

10

20

30

40

50

60

70

80

90

2002 2003 2004 2005 2006 2007

Percentage of comparative surveys

Source: GAO analysis of federal monitoring survey data.

Fiscal year

55.0

66.3 66.0

76.4

72.0 74.1

Most Frequently Missed 

Deficiencies Involved 

Quality of Care 

Page 15 GAO-08-517 Federal Monitoring Surveys for Nursing Homes
only 2.2 percent of total federal comparative surveys.20 At the potential for

more than minimal harm level, Quality of Care was one of two categories 

with the highest frequency of missed deficiencies—31.7 percent.21 For the 

percentage of missed deficiencies in each of the CMS quality standard 

categories, see appendix IV. 

Both federal observational surveys and our prior reports have identified 

factors that may contribute to the understatement of deficiencies by state 

survey teams. From fiscal year 2002 through 2007, 80 percent of the 4,999 

federal monitoring surveys were observational. Our review of 

observational survey data—which are collected during direct observation 

of state survey teams—found that some of the lowest state survey team 

ratings nationwide were in the General Investigation and Deficiency 

Determination areas. Together, these two areas directly affect the 

appropriate identification and citation of deficiencies. 

Federal Observational 

Surveys and Prior GAO 

Reports Identified Factors 

That May Contribute to 

Deficiency 

Understatement by State 

Survey Teams 

• The General Investigation segment of an observational survey evaluates 

the effectiveness with which the state survey team collected information 

to determine how the facility’s environment and care of residents affect 

residents’ quality of life, health and safety, and ability to reach their 

highest practicable physical, mental, and psychosocial well-being. This 

segment includes observations of state survey team actions such as 

collection of information, discussion of survey observations, interviews 

with facility residents, and implementation of CMS investigative protocols.

• The Deficiency Determination segment of an observational survey 

evaluates the skill with which the state survey teams (1) integrate and 

analyze all information collected and (2) use the guidance to surveyors 

and regulatory requirements to make accurate compliance determinations. 

This segment includes observations of state survey team actions such as 

reviews of regulatory requirements, team participation in deficiency

discussions, presentation of complete information, accurate decision 

making, and accurate citation of deficiencies. 

 20Examples of deficiencies related to Resident Behavior and Facility Practices include 

resident abuse and the misuse of restraints. 

21In addition, 31.7 percent of total comparative surveys found at least one missed potential 

for more than minimal harm level deficiency in the Resident Assessment category. An 

example of a deficiency related to Resident Assessment is the failure to develop a 

comprehensive care plan that meets a resident’s physical, mental, and psychosocial needs. 

Page 16 GAO-08-517 Federal Monitoring Surveys for Nursing Homes
Nationwide, 7.7 percent of the state survey teams observed by federal 

surveyors received below satisfactory ratings on the General Investigation 

measure from fiscal year 2002 through 2007.22 During the same 6 fiscal 

years, 9.2 percent, or about 1 in 11, of the state survey teams observed by 

federal surveyors received below satisfactory ratings on the Deficiency

Determination measure. Our analysis found variation across states in 

survey team performance in General Investigation and Deficiency 

Determination. Sixteen states had more teams than the national average 

receive below satisfactory ratings for both measures, while 28 states had 

fewer teams than the national average receive below satisfactory ratings 

(see app. V).23

Poor performance on these observational survey measures may be a 

contributing factor to the understatement of deficiencies by state survey 

teams. For example, of the nine states in table 2 with the highest 

percentage of missed serious deficiencies on comparative surveys, six had 

more teams than the national average receive below satisfactory ratings 

for both General Investigation and Deficiency Determination (see 

table 3).24

 22Federal observational surveys use a five-point rating scale to evaluate state survey teams. 

Our analysis collapsed the ratings in the lowest two categories—much less than 

satisfactory and less than satisfactory—into a single category of below satisfactory results. 

23An additional seven states had mixed performance on these two measures—performing

above the national average for one measure and below the national average for the other.

24Later in this report, we observe that no Wyoming and South Dakota survey teams 

received below satisfactory ratings on observational surveys. 

Page 17 GAO-08-517 Federal Monitoring Surveys for Nursing Homes
Table 3: States with 25 Percent or More of Comparative Surveys with Missed

Deficiencies and Percentage of Their Observational Surveys with Less Than

Satisfactory Ratings on General Investigation and Deficiency Determination, Fiscal 

Years 2002 through 2007 

Percentage of state observational 

surveys with below satisfactory 

ratings

State 

General 

investigation

Deficiency 

determination

Percentage of

comparative surveys that 

found at least one 

missed G-L deficiency

Alabama 20.0 22.7 27.8

Arizona 7.4 15.4 26.7

Missouri 17.6 22.1 28.6

New Mexico 26.3 31.6 33.3

Oklahoma 12.1 16.5 30.0

South Carolina 14.3 22.9 33.3

South Dakota 0.0 0.0 33.3

Tennessee 14.6 20.7 26.3

Wyoming 0.0 0.0 33.3

Nation 7.7 9.2 14.5

Source: GAO analysis of federal monitoring survey data. 

Our prior reports have described some other factors that may contribute 

to survey inconsistency and the understatement of deficiencies by state 

survey teams: (1) weaknesses in CMS’s survey methodology, such as poor 

documentation of deficiencies;25 (2) confusion about the definition of 

actual harm;26 (3) predictability of surveys, which allows homes to conceal 

problems if they so desire;

27 (4) inadequate quality assurance processes at 

the state level to help detect understatement in the scope and severity of

 25See GAO-03-561 and GAO-07-794T. In response to our recommendation to finalize the 

development, testing, and implementation of a more rigorous survey methodology, CMS 

developed and is currently evaluating a revised survey methodology. 

26See GAO-06-117. 

27See GAO-03-561. Our analysis of survey predictability considered surveys to be 

predictable if (1) homes were surveyed within 15 days of the 1-year anniversary of the prior 

survey or (2) homes were surveyed within 1 month of the maximum 15-month interval 

between standard surveys. We used this rationale because homes know the maximum 

allowable interval between surveys, and those whose prior surveys were conducted 14 or

15 months earlier are aware that they are likely to be surveyed soon. 

Page 18 GAO-08-517 Federal Monitoring Surveys for Nursing Homes
deficiencies;28 and (5) inexperienced state surveyors as a result of 

retention problems.29 In ongoing work, we are investigating the factors that 

contribute to understatement. 

CMS has taken steps to improve the federal monitoring survey program, 

but weaknesses remain in program management and oversight. For 

example, CMS has improved processes to ensure that comparative surveys 

more accurately reflect conditions at the time of the state survey, has 

switched control of the federal monitoring survey database to the office 

responsible for ensuring the effectiveness of state surveys, and has begun 

examining how to use monitoring survey data to improve oversight. 

Despite this progress, the management and oversight potential of the 

program has not been fully realized. In particular, CMS (1) has only begun

exploring options for identifying understatement that occurs in cases 

where state surveys cite deficiencies at too low a level, for possible 

implementation in fiscal year 2009, and (2) is not effectively managing the 

federal monitoring survey database or using the database to oversee 

consistent implementation of the federal monitoring survey program by its 

regional offices. 

CMS has taken steps in three areas—time between surveys, resident 

sample, and survey resources—to ensure that comparative surveys more 

accurately capture the conditions at the time of the state survey. 

CMS Has Taken Steps 

to Improve the 

Federal Monitoring 

Survey Program, but 

Weaknesses in 

Management and 

Oversight Remain 

CMS Policy Changes Have 

Improved Federal 

Monitoring Surveys 

• Time between surveys. In fiscal year 2002, CMS initiated a policy that

shortened the length of time between state and comparative surveys from 

2 months to 1 month. CMS relaxed the 1 month standard by changing the 

requirement to 30 working days in fiscal year 2003. As a result of 

shortening the time between the two surveys, the conditions at the time of 

the comparative survey are more likely to reflect those at the time of the 

state survey; for example, the same residents are still likely to be in the 

nursing home. Comparative surveys during fiscal year 2007 took place on 

average 21.4 working days (30.9 calendar days) after state surveys. 

 28See GAO-03-561. 

29See GAO-03-561. 

Page 19 GAO-08-517 Federal Monitoring Surveys for Nursing Homes
• Resident sample. Beginning in fiscal year 2003, CMS policy required that

comparative surveys include at least half of the residents from state survey 

investigative samples. Officials from several regional offices said that 

examining the same resident allows for more clear-cut determinations of 

whether the state should have cited a deficiency. Since the policy change, 

about 78 percent of comparative surveys from fiscal year 2003 through 

2007 included at least half the residents from state surveys’ investigative 

samples. By comparison, only 13 percent of comparative surveys met that

50 percent threshold in fiscal year 2002, the year before the policy went 

into effect. 

• Survey resources. Beginning in fiscal year 2003, CMS initiated a policy 

that each comparative survey should have the same number of federal 

surveyors as its corresponding state survey, again to more closely mirror 

the conditions under which the state survey was conducted.30 We found 

that in fiscal year 2007, the average state survey team (3.4 surveyors) was 

larger than the average federal survey team (3.0 surveyors). However, on 

average, federal surveyors remained on-site longer than state surveyors—

4.3 days for federal surveyors compared with 3.7 days for state surveyors. 

When the number of surveyors and time on-site are taken together, state 

surveys averaged 12.6 surveyor-days and federal comparative surveys 

averaged 12.9 surveyor-days.31

Given these improvements, we asked the regional offices how receptive

state survey teams were to feedback that they had missed deficiencies. 

Most regional office officials told us that in general the feedback session 

with state surveyors on missed deficiencies was not contentious and that

state surveyors generally accepted the feedback provided. However, CMS 

established a formal dispute resolution process for comparative surveys in

October 2007. The process is similar to the process already in place for 

resolving disagreements about observational survey results.32

 30On a comparative survey, CMS does not evaluate the adequacy of state survey team 

staffing. 

31Surveyor-days are calculated as the total number of days on-site times the number of 

surveyors who worked full-time on that nursing home survey. If three surveyors were onsite and the survey took 3 days, then the survey would have used 9 surveyor-days. 

32Because the establishment of the dispute resolution process for comparative surveys is 

relatively recent, we did not assess how often states challenge comparative findings. 

Page 20 GAO-08-517 Federal Monitoring Surveys for Nursing Homes
While CMS requires federal surveyors to determine whether a deficiency

cited on a comparative but not a state survey was missed by state 

surveyors, there is no comparable requirement for deficiencies that are 

cited at different scope and severity levels. As a result, comparative 

surveys do not effectively capture the extent of the understatement of 

serious deficiencies by state surveyors. As with missed deficiencies, a

discrepancy between federal and state survey results does not 

automatically indicate understatement. For example, the deficiency could 

have worsened by the time of the federal survey. 

Although CMS does not require federal surveyors to evaluate scope and 

severity differences between the two sets of surveys, we found that some 

regional offices used the validation question for missed deficiencies—

”based on the evidence available to the [state], should the [state survey] 

team have cited this [deficiency]?”—to make such a determination.33 Using 

the validation question to make these determinations is contrary to CMS 

guidance issued in October 2003, which instructed comparative survey 

teams to only answer this question when the state failed to cite the 

deficiency altogether. 

To assess whether differences in scope and severity levels were actually 

understated—rather than deficiencies that worsened between the state 

and federal surveys—we first identified all 71 deficiencies on comparative 

surveys conducted from fiscal year 2002 through 2007 where federal 

survey teams cited actual harm or immediate jeopardy deficiencies that 

state survey teams cited at a lower scope and severity level.34 We then 

examined the comment fields in the federal monitoring survey database 

associated with those deficiencies. Our analysis identified 27 deficiencies

(38 percent) in which federal survey teams determined that a state’s scope 

and severity citation was too low. For another 22 deficiencies (31 percent), 

federal survey teams found that the state’s lower scope and severity 

Federal Monitoring Survey 

Database Has Not Been 

Used to Capture 

Understatement of Scope 

and Severity Levels 

 33Five of the 10 CMS regions made determinations on multiple occasions about whether the 

state “should have cited” a deficiency, even though the only discrepancy was the scope and 

severity level. 

34We did not examine the 388 instances where federal survey teams cited potential for more

than minimal harm deficiencies and state survey teams cited the same deficiencies at a 

lower scope and severity level. 

Page 21 GAO-08-517 Federal Monitoring Surveys for Nursing Homes
determination was appropriate, given the circumstances at the time of the

state survey. The remaining 22 deficiencies (31 percent) did not have 

comments or contained remarks that were inconclusive about whether the 

state deficiency citation was too low. When the confirmed scope and 

severity understatement was included with understatement caused by 

missed deficiencies, the total percentage of comparative surveys with 

understatement of serious deficiencies increased by an average of about 

1 percentage point over the 6 fiscal years we analyzed (see fig. 3).35

 35Overall, federal surveyors identified understated scope and severity levels on 2.4 percent 

of state surveys during fiscal years 2002 through 2007. The 1 percent average increase in

the total percentage of comparative surveys is lower because comparative surveys with 

both types of understatement—failing to cite a deficiency or citing a deficiency at too low a 

scope and severity level—were counted only once. 

Page 22 GAO-08-517 Federal Monitoring Surveys for Nursing Homes
Figure 3: Percentage of Comparative Surveys Nationwide with Understatement of 

Actual Harm and Immediate Jeopardy Deficiencies, with Scope and Severity 

Differences Included, Fiscal Years 2002 through 2007 

a

The inclusion of scope and severity understatement is based on our analysis of 71 deficiencies that 

federal survey teams cited as actual harm or immediate jeopardy deficiencies that state survey teams 

cited at a lower scope and severity level. 

While CMS headquarters does not require federal surveyors to determine 

whether a deficiency cited by state survey teams was cited at too low a 

scope and severity level, some regional offices have developed their own 

procedures to track this information and use it to provide feedback to 

state survey agencies. For example, in one regional office an individual 

reviews all comment fields for a year’s worth of comparative surveys, 

makes a hand count of scope and severity differences that states should 

have cited, and then shares this with the state survey agencies during their 

annual performance reviews. Because the federal monitoring survey 

database does not automatically collect data on scope and severity 

determinations, CMS headquarters does not have access to the data 

analyses the regions have independently conducted. Some of the regional

offices told us that they would like to have a specific way that the federal

0

5

10

15

20

25

2002 2003 2004 2005 2006 2007

Percentage of comparative surveys

Source: GAO analysis of federal monitoring survey data.

Fiscal year

Percentage of comparative surveys with missed deficiencies at the actual harm or immediate 

jeopardy level

Percentage of comparative surveys with missed deficiencies and scope and severity understatement 

at the actual harm or immediate jeopardy levela

13.8

17.5

11.1

14.3

15.3 15.3

19.3

11.7

17.4

15.9 16.5

Page 23 GAO-08-517 Federal Monitoring Surveys for Nursing Homes
monitoring survey database could track scope and severity 

understatement that is similar to how deficiencies missed by state 

surveyors are tracked. 

In January 2008, CMS officials told us that they had initiated a pilot 

program in October 2007 to test the collection of data on understatement 

of scope and severity differences.36 According to CMS, the pilot, which will 

run through 2008 for possible fiscal year 2009 implementation, is 

necessary because the agency needs to determine which scope and 

severity understatement differences should be captured. For example, 

CMS is uncertain whether regions should only focus on differences that 

would raise the scope and severity level to actual harm or immediate 

jeopardy and not assess differences for understatement that occurs at 

lower scope and severity levels. 

Our analysis found that CMS headquarters was not effectively managing 

the federal monitoring survey database or using the database to oversee 

consistent implementation of the federal monitoring survey program by 

regional offices.37 While CMS uses data from comparative and 

observational surveys to provide feedback to state survey agencies during 

state performance reviews, CMS officials told us that they recognized the 

need to improve their management and use of the database for better 

oversight of the agency’s 10 regional offices. 

We identified two problems in CMS’s management of the federal 

monitoring survey database. CMS was not aware that (1) the results of a 

considerable number of comparative surveys were missing from the 

database and (2) the validation question for missed deficiencies was being 

used by some regional offices to identify scope and severity differences, 

contrary to CMS guidance. 

CMS Could More 

Effectively Manage the 

Federal Monitoring Survey 

Database and Use It for 

Regional Office Oversight 

 36CMS became aware in October 2007 that some regional offices were using the validation 

question designed to identify missed deficiencies when our preliminary analysis identified

missing data in several regions. 

37Officials from CMS’s Survey and Certification Group—the component responsible for

ensuring the effectiveness of state survey activities—assumed control of the database in 

January 2007 from CMS’s Division of National Systems. While officials from the Survey and

Certification Group are still familiarizing themselves with the database, they stated that this 

change in control was necessary to move it closer to the component responsible for 

managing the federal monitoring survey program. 

Page 24 GAO-08-517 Federal Monitoring Surveys for Nursing Homes
• Missing data. In October 2007, we identified missing comparative surveys 

for two regional offices dating back to 2005 and asked CMS to follow up 

with officials in those regions. At least one of the regions had completed 

the surveys but had failed to upload them into the national database.38 We 

also found that CMS had not included data in the federal monitoring 

survey database from 162 contractor-led comparative surveys conducted 

between fiscal years 2004 and 2007.

39

• Use of validation question contrary to CMS guidance. Some regional 

offices were using the missed deficiency validation question to make 

determinations about whether scope and severity differences constituted 

understatement, making it difficult to distinguish between missed 

deficiencies and scope and severity understatement. In addition, we found 

that the regional office answer to the validation question was not always 

consistent with the information recorded in the comment box. 

Similarly, we identified weaknesses in CMS’s use of the database for 

regional office oversight. For example, CMS was not (1) examining 

comparative survey data to ensure that regional offices comply with CMS 

guidance intended to ensure that comparative surveys more accurately

capture the conditions at the time of the state survey and (2) using the 

database to identify inconsistencies between comparative and 

observational survey results. 

• Ensuring regional office compliance. While CMS has provided 

guidance to its regional offices to help ensure that comparative surveys 

more accurately capture the conditions at the time of the state survey, the 

agency is not fully using available data to ensure that the regional offices 

implement the agency’s guidance. For example, we found that the length

of time between state and comparative surveys varied broadly by CMS 

region. In 2007, the average time gap ranged from a low of 15.4 working 

days (22.5 calendar days) in the Boston region to a high of 38.5 working 

days (54.4 calendar days) in the New York region. Furthermore, while 

78 percent of comparative surveys from fiscal year 2003 through 2007 

 38Subsequently, CMS informed us that all comparative surveys from the two regions were 

now accessible in the database. 

39CMS indicated that the results of contractor-led comparative health surveys—which 

began in fiscal year 2004—are not included in the federal monitoring survey database 

because those surveys are in addition to the federal monitoring surveys required by statute

and that inclusion of the contractor-led data would hinder CMS’s ability to collect and

analyze data about CMS staff resources that are devoted to comparative surveys. 

Page 25 GAO-08-517 Federal Monitoring Surveys for Nursing Homes
followed CMS’s guidance to include at least half of the residents from state 

surveys’ investigative samples, 22 percent of comparative surveys did not 

meet this threshold. Finally, when we contacted officials in CMS 

headquarters to ask clarifying questions about the data variables needed to 

conduct these analyses, the headquarters officials were not familiar with a 

number of the variables and referred us to a CMS staff person in one of the 

regional offices. Together, these three examples suggest that CMS is not 

effectively using the data to hold regional offices accountable for 

implementing guidance. 

• Identify inconsistencies between comparative and observational 

results. CMS officials told us that they have begun to explore regional

office differences in less than satisfactory ratings for state survey teams on 

observational surveys.40 However, CMS officials told us that they do not 

plan to use the database to identify inconsistencies between comparative 

and observational surveys that may warrant follow-up to ensure that 

regional offices are adhering to CMS guidance and consistently assessing 

state surveyor performance.41 For example, some states that performed 

below the national average in identifying serious deficiencies on 

comparative surveys received above-average marks on observational 

survey measures for Deficiency Determination and General Investigation. 

Wyoming’s 33.3 percent rate for surveys with missed serious deficiencies 

was more than double the national average of about 15 percent for surveys 

conducted during fiscal years 2002 and 2007. Yet Wyoming never received

a below satisfactory rating on its General Investigation or Deficiency

Determination measures during 18 observational surveys over that same 

6-year period. We found similar inconsistencies in the results of federal 

monitoring surveys for South Dakota and a few other states. Although 

inconsistencies between comparative and observational surveys may not 

necessarily indicate a problem, they may warrant investigation. For 

example, in a small state like Wyoming it is likely that comparative and 

observational surveys have evaluated the same group of state surveyors. 

Further, Wyoming and South Dakota are two of six states whose federal 

monitoring surveys are conducted by CMS’s Denver regional office. Of the 

140 observational surveys conducted from fiscal year 2002 through 2007, 

federal surveyors from the Denver regional office gave one below 

 40However, CMS officials told us that they have been unable to identify whether region-toregion differences were the result of inconsistencies in state survey agency performance or 

regional variation among nursing homes.

41Comparative and observational surveys each measure some of the same skills required 

for effective surveying, particularly state survey team general investigative techniques and 

ability to accurately identify deficiencies. 

Page 26 GAO-08-517 Federal Monitoring Surveys for Nursing Homes
satisfactory rating on the Deficiency Determination measure. That 

0.7 percent rate of below satisfactory performance was more than four 

times lower than the regional office with the next-lowest percentage—the 

Chicago regional office—which awarded below satisfactory ratings to 

3.3 percent of state survey teams it observed. 

With about 1 in 6 comparative surveys concluding that state survey teams 

had missed a serious deficiency or understated its scope and severity level, 

it is evident that state survey agency performance limits the federal 

government’s ability to obtain an accurate picture of how often nursing 

home residents face actual harm or are at risk of serious injury or death.

These missed serious deficiencies most frequently involved Quality of 

Care, reflecting shortcomings in fundamental provider responsibilities 

such as ensuring proper nutrition and hydration, accident prevention, and

preventing pressure sores. Observational survey results also underscore 

problems state surveyors may face in identifying facility deficiencies; 

about 1 in 11 state survey teams nationwide were rated as below 

satisfactory by CMS surveyors on the Deficiency Determination measure. 

We found that comparative survey data may mask the true extent of 

understatement because CMS’s current protocol does not require regional

offices to track in the federal monitoring survey database when state 

surveyors cite lower-than-appropriate scope and severity levels. As we 

conducted our work, CMS officials recognized this problem and in 

October 2007 began to experiment with a pilot program to measure 

understated scope and severity. However, at the conclusion of the pilot, 

scheduled for fiscal year 2008, CMS may decide not to implement a 

validation question for all scope and severity differences. We believe it is 

important to assess differences for understatement that occurs at the 

D through L levels—potential for more than minimal harm, actual harm, 

and immediate jeopardy. 

We also found that CMS was not effectively managing the federal 

monitoring survey database to ensure that regional offices were entering 

data in a timely and consistent fashion. Lack of accurate and reliable data 

hinders effective oversight. For example, we found that the database was 

missing a considerable number of comparative surveys. Further, CMS has 

not used the federal monitoring survey database to its full potential as an

oversight tool. For example, CMS is not fully using data on comparative 

surveys to ensure that regional offices are implementing guidance 

intended to improve federal monitoring surveys. Although CMS’s Survey 

and Certification Group assumed control of the database in January 2007, 

Conclusions 

Page 27 GAO-08-517 Federal Monitoring Surveys for Nursing Homes
headquarters staff often referred us to CMS regional office staff to answer 

specific database questions, suggesting a lack of familiarity with the 

organization and content of the database. In addition, agency officials told

us that they do not plan to follow up on inconsistencies between 

comparative and observational survey results that could indicate 

weaknesses in how regional offices evaluate state surveyors’ performance. 

Identifying and following up on such inconsistencies could help ensure 

database reliability and hold regional office officials accountable for their 

implementation of the federal monitoring survey program, a program 

required by statute. 

To address weaknesses in CMS’s management of the federal monitoring 

survey database that also affect the agency’s ability to effectively track 

understatement, we recommend that the Administrator of CMS take the 

following two actions: 

• Require regional offices to determine if there was understatement when 

state surveyors cite a deficiency at a lower scope and severity level than 

federal surveyors do and to track this information in the federal 

monitoring survey database. 

• Establish quality controls to improve the accuracy and reliability of 

information entered into the federal monitoring survey database. 

To address weaknesses that affect CMS’s ability to oversee regional office 

implementation of the federal monitoring survey program, we recommend 

that the Administrator of CMS take the following two actions: 

• Routinely examine comparative survey data and hold regional offices 

accountable for implementing CMS guidance that is intended to ensure 

that comparative surveys more accurately capture the conditions at the 

time of the state survey. 

• Regularly analyze and compare federal comparative and observational 

survey results. 

In written comments on our draft report, HHS indicated that it fully 

endorsed and would implement our four recommendations intended to 

strengthen management and oversight of the federal monitoring survey

program. The comments generally outlined CMS’s implementation plan 

through 2009 and indicated that some steps, such as improved 

Recommendations for 

Executive Action 

Agency Comments 

Page 28 GAO-08-517 Federal Monitoring Surveys for Nursing Homes
management of the federal monitoring survey database, are already under 

way. HHS’s comments are reproduced in appendix VI. 

The majority of HHS’s comments focused on its strategic approach to 

improving oversight: (1) ensuring that all nursing homes are surveyed at 

least once every 15 months, (2) improving surveyor understanding of 

federal quality requirements through improved guidance and training, 

(3) increasing the consistency of state surveys through the introduction of

a new nursing home survey methodology, and (4) improving the use of 

data generated by federal monitoring surveys. Many of these strategies aim 

to address the underlying causes of understatement, the topic of a 

forthcoming GAO report. HHS also noted that limitations in the Medicare 

survey and certification budget underscore the agency’s need to target 

resources effectively to maximize results. For example, HHS indicated that 

the implementation of the new survey methodology will be dependent on 

the level of funding in the overall survey and certification budget through 

fiscal year 2014. Survey and Certification funding is the subject of another 

forthcoming GAO report. 

Two of HHS’s observations merit further discussion. First, HHS noted that 

understatement that arises from a lack of understanding or confusion 

about federal requirements would generally not be detected through 

federal monitoring surveys because both federal and state surveyors 

would be affected by the same limitation. We believe that the consistency 

with which federal surveys have identified serious deficiencies missed by 

state surveyors from fiscal year 2002 through 2007—about 15 percent, on 

average—suggests that federal surveyors have a better understanding of 

CMS quality requirements than do state surveyors. We have previously 

reported that the limited experience level of state surveyors because of the 

high turnover rate was a contributing factor to deficiency 

understatement.42

Second, HHS questioned our use of “one missed deficiency per survey” as 

a measure of understatement. We believe that this standard is appropriate 

for serious deficiencies that result in harm or immediate jeopardy 

(G through L level) because the goal of state surveys should be to identify

and require nursing homes to address all such deficiencies. CMS itself uses 

this standard during annual state performance reviews. We also used this 

standard to describe the proportion of comparative surveys that identified 

 42See GAO-03-561 and GAO-06-117. 

Page 29 GAO-08-517 Federal Monitoring Surveys for Nursing Homes
missed deficiencies at the potential for more than minimal harm level 

(D through F). Identifying and requiring nursing homes to correct such 

deficiencies is important because if uncorrected they have the potential to 

become more serious. Compared to missed serious deficiencies, we found 

that understatement of potential for more than minimal harm deficiencies 

was more widespread—about 70 percent of comparative surveys identified 

at least one state survey with such missed deficiencies. The number of 

state surveys with missed deficiencies at the D through F level was greater 

than 40 percent in all but five states, and state surveys selected for 

comparative surveys failed to identify an average of 2.5 deficiencies in this

range per survey. In short, the magnitude of understatement at the 

potential for more than minimal harm level should be a cause for concern. 

HHS also provided technical comments, which we incorporated as 

appropriate.

As arranged with your offices, unless you publicly announce its contents 

earlier, we plan no further distribution of this report until 30 days after its 

issue date. At that time, we will send copies to the Administrator of the 

Centers for Medicare & Medicaid Services and appropriate congressional 

committees. We will also make copies available to others upon request. In

addition, the report will be available at no charge on GAO’s Web site at 

http://www.gao.gov. 

If you or your staffs have any questions about this report, please contact 

me at (202) 512-7114 or dickenj@gao.gov. Contact points for our Offices of 

Congressional Relations and Public Affairs may be found on the last page 

of this report. GAO staff who made major contributions to this report are 

listed in appendix VII. 

John E. Dicken 

Director, Health Care 

Page 30 GAO-08-517 Federal Monitoring Surveys for Nursing Homes
Appendix I: Percentage of Nursing Homes

Cited for Actual Harm or Immediate Jeopardy 

during Standard Surveys 

In order to identify trends in the percentage of nursing homes cited with 

actual harm or immediate jeopardy deficiencies, we analyzed data from 

the Centers for Medicare & Medicaid Service’s (CMS) On-Line Survey, 

Certification, and Reporting system (OSCAR) database for fiscal years 

2002 through 2007 (see table 4). Because homes must be surveyed at least 

every 15 months, with a required 12-month statewide average, it is possible 

that a home was surveyed more than once in any fiscal year. To avoid 

double counting homes, we included only a home’s most recent survey 

from each fiscal year. Because CMS conducts a relatively small number of 

comparative surveys, it is not possible to compare the results of 

comparative surveys to the results of all state surveys. 

Table 4: Percentage of Nursing Homes Cited for Actual Harm or Immediate 

Jeopardy, by State, Fiscal Years 2002 through 2007 

Fiscal year

State 

Number of homes 

in fiscal year 2007 2002 2003 2004 2005 2006 2007

Alabama 233 12.7 18.1 16.6 24.3 23.6 16.3

Alaska 15 26.7 0.0 0.0 0.0 26.7 13.3

Arizona 137 7.3 6.6 9.4 9.9 25.4 26.4

Arkansas 248 22.3 24.8 21.5 17.6 13.6 14.2

California 1,285 5.1 3.7 6.5 8.0 13.9 12.0

Colorado 212 32.7 20.9 25.9 40.4 43.8 42.4

Connecticut 244 45.8 43.1 55.4 44.2 46.5 38.3

Delaware 45 11.1 5.3 15.0 35.7 21.1 33.3

District of Columbia 20 30.0 41.2 40.0 30.0 25.0 36.8

Florida 683 14.9 10.2 8.6 4.2 8.0 8.3

Georgia 362 23.7 24.6 17.9 19.3 15.0 14.9

Hawaii 48 21.2 12.1 22.9 2.8 2.1 7.3

Idaho 79 39.2 31.9 27.3 40.5 44.9 27.1

Illinois 810 15.3 18.4 16.3 16.7 20.3 25.1

Indiana 520 23.2 19.7 24.5 29.6 30.6 35.2

Iowa 463 8.0 9.2 12.4 11.7 9.6 16.3

Kansas 358 32.9 26.5 31.9 36.6 37.7 29.4

Kentucky 293 23.2 26.1 15.0 8.0 10.6 7.6

Louisiana 299 21.8 16.2 13.8 16.3 16.1 11.7

Maine 113 6.6 11.1 12.8 7.0 9.8 7.2

Maryland 234 26.1 15.4 18.3 7.6 7.6 17.1

Massachusetts 452 24.6 25.9 17.4 22.9 20.6 16.6

Appendix I: Percentage of Nursing Homes 

Cited for Actual Harm or Immediate 

Jeopardy during Standard Surveys 

Page 31 GAO-08-517 Federal Monitoring Surveys for Nursing Homes
Appendix I: Percentage of Nursing Homes

Cited for Actual Harm or Immediate Jeopardy 

during Standard Surveys 

Fiscal year

State 

Number of homes 

in fiscal year 2007 2002 2003 2004 2005 2006 2007

Michigan 439 29.7 26.9 23.6 23.1 27.9 29.8

Minnesota 398 22.3 18.3 14.3 14.4 17.5 16.0

Mississippi 206 18.8 16.0 19.5 18.7 8.4 9.1

Missouri 530 15.6 12.5 12.5 15.8 15.2 15.5

Montana 96 12.0 20.0 18.0 17.9 16.7 23.9

Nebraska 228 20.1 14.8 15.8 14.4 25.0 26.4

Nevada 48 11.9 9.1 17.5 21.7 21.3 12.5

New Hampshire 81 29.4 24.4 25.6 26.3 22.9 14.7

New Jersey 368 18.8 10.5 12.9 18.2 14.6 16.5

New Mexico 72 14.9 21.3 25.7 32.4 25.0 28.6

New York 657 34.2 15.2 11.0 14.4 17.5 17.2

North Carolina 422 25.6 29.0 22.1 18.8 16.7 10.7

North Dakota 83 17.9 12.4 13.6 17.7 20.5 15.6

Ohio 984 25.4 19.1 11.8 14.4 13.4 14.4

Oklahoma 348 22.2 26.4 17.1 26.5 18.7 18.7

Oregon 139 23.7 20.3 16.7 18.9 17.8 17.6

Pennsylvania 724 13.5 17.2 19.9 15.5 12.0 11.9

Rhode Island 87 5.6 8.1 9.3 9.5 4.4 1.3

South Carolina 176 19.8 29.6 33.3 25.5 16.5 8.8

South Dakota 112 26.8 32.1 23.4 12.8 18.4 22.6

Tennessee 332 20.7 21.8 24.1 17.6 11.5 16.2

Texas 1,195 22.4 18.0 12.8 16.7 18.1 14.7

Utah 94 25.6 19.0 12.2 8.4 16.7 7.5

Vermont 41 15.0 10.0 19.5 23.7 13.5 12.1

Virginia 281 11.7 13.7 10.2 16.3 15.2 18.3

Washington 247 37.0 30.9 28.5 27.6 23.5 23.7

West Virginia 133 20.4 12.7 11.5 17.7 10.5 21.7

Wisconsin 401 11.2 10.9 13.1 18.8 21.6 29.8

Wyoming 39 25.0 22.9 17.1 11.8 16.2 24.2

Nation 16,114 20.2 17.8 16.4 17.4 17.9 17.8a

Source: GAO analysis of OSCAR data. 

Note: Data presented in this table may be slightly different from those presented in earlier reports 

because we (1) used final year-end OSCAR files when they were available or (2) the OSCAR extracts

contained surveys that had not previously been entered in the database. 

a

Fiscal year 2007 results are incomplete and use all available information as of January 2008. 

Page 32 GAO-08-517 Federal Monitoring Surveys for Nursing Homes
Appendix II: Percentage of Comparative 

Surveys Identifying Missed Deficiencies at 

Actual Harm or Immediate Jeopardy Level 

State 

Number of 

homes in 

fiscal year 2007

Total

comparative 

surveys

Total

comparative 

surveys with at 

least one missed

G-L deficiency

Percentage of

total comparative 

surveys with at 

least one missed

G-L Deficiency

Total

number missed

G-L deficiencies

Alabama 233 18 5 27.8 13

Alaska 15 11 0 0.0 0

Arizona 137 15 4 26.7 6

Arkansas 248 18 1 5.6 1

California 1,285 49 5 10.2 6

Colorado 212 22 3 13.6 4

Connecticut 244 16 1 6.3 1

Delaware 45 13 2 15.4 2

District of Columbia 20 12 1 8.3 1

Florida 683 26 3 11.5 6

Georgia 362 18 3 16.7 4

Hawaii 48 12 1 8.3 1

Idaho 79 12 0 0.0 0

Illinois 810 32 7 21.9 12

Indiana 520 25 3 12.0 4

Iowa 463 19 3 15.8 4

Kansas 358 24 4 16.7 8

Kentucky 293 18 2 11.1 2

Louisiana 299 17 3 17.6 6

Maine 113 12 0 0.0 0

Maryland 234 19 2 10.5 2

Massachusetts 452 17 1 5.9 1

Michigan 439 25 5 20.0 5

Minnesota 398 21 2 9.5 2

Mississippi 206 18 4 22.2 8

Missouri 530 28 8 28.6 14

Montana 96 12 2 16.7 2

Nebraska 228 18 1 5.6 1

Nevada 48 12 1 8.3 2

New Hampshire 81 14 2 14.3 2

New Jersey 368 24 5 20.8 16

New Mexico 72 12 4 33.3 9

Appendix II: Percentage of Comparative 

Surveys Identifying Missed Deficiencies at 

Actual Harm or Immediate Jeopardy Level 

Page 33 GAO-08-517 Federal Monitoring Surveys for Nursing Homes
Appendix II: Percentage of Comparative 

Surveys Identifying Missed Deficiencies at 

Actual Harm or Immediate Jeopardy Level 

State 

Number of 

homes in 

fiscal year 2007

Total

comparative 

surveys

Total

comparative 

surveys with at 

least one missed

G-L deficiency

Percentage of

total comparative 

surveys with at 

least one missed

G-L Deficiency

Total

number missed

G-L deficiencies

New York 657 27 6 22.2 12

North Carolina 422 21 3 14.3 3

North Dakota 83 12 0 0.0 0

Ohio 984 31 1 3.2 1

Oklahoma 348 20 6 30.0 11

Oregon 139 18 0 0.0 0

Pennsylvania 724 37 6 16.2 6

Rhode Island 87 12 2 16.7 3

South Carolina 176 18 6 33.3 19

South Dakota 112 12 4 33.3 4

Tennessee 332 19 5 26.3 10

Texas 1,195 38 5 13.2 7

Utah 94 11 1 9.1 1

Vermont 41 10 0 0.0 0

Virginia 281 17 1 5.9 1

Washington 247 18 2 11.1 2

West Virginia 133 13 0 0.0 0

Wisconsin 401 21 2 9.5 2

Wyoming 39 12 4 33.3 5

Nation 16,114 976 142 14.5 232

Source: GAO analysis of CMSs OSCAR data for the number of homes and federal monitoring survey data for all other data presente d. 

Page 34 GAO-08-517 Federal Monitoring Surveys for Nursing Homes
Appendix III: Percentage of Comparative 

Surveys Identifying Missed Deficiencies with 

Potential for More Than Minimal Harm

State 

Number of 

homes in 

fiscal year 2007 

Total

comparative 

surveys

Total comparative 

surveys with at least 

one missed

D-F deficiency

Percentage of total 

comparative surveys with

at least one missed

D-F deficiency

Total number of

missed

D-F deficiencies

Alabama 233 18 17 94.4 61

Alaska 15 11 4 36.4 8

Arizona 137 15 12 80.0 73

Arkansas 248 18 13 72.2 50

California 1,285 49 36 73.5 104

Colorado 212 22 21 95.5 108

Connecticut 244 16 8 50.0 28

Delaware 45 13 9 69.2 29

District of Columbia 20 12 10 83.3 26

Florida 683 26 18 69.2 53

Georgia 362 18 13 72.2 48

Hawaii 48 12 7 58.3 22

Idaho 79 12 7 58.3 18

Illinois 810 32 17 53.1 63

Indiana 520 25 12 48.0 31

Iowa 463 19 13 68.4 36

Kansas 358 24 19 79.2 66

Kentucky 293 18 11 61.1 33

Louisiana 299 17 13 76.5 73

Maine 113 12 6 50.0 25

Maryland 234 19 9 47.4 16

Massachusetts 452 17 8 47.1 22

Michigan 439 25 18 72.0 37

Minnesota 398 21 15 71.4 29

Mississippi 206 18 15 83.3 57

Missouri 530 28 22 78.6 146

Montana 96 12 12 100.0 52

Nebraska 228 18 13 72.2 42

Nevada 48 12 11 91.7 33

New Hampshire 81 14 9 64.3 42

New Jersey 368 24 14 58.3 50

New Mexico 72 12 9 75.0 27

New York 657 27 15 55.6 77

Appendix III: Percentage of Comparative 

Surveys Identifying Missed Deficiencies with 

Potential for More Than Minimal Harm 

Page 35 GAO-08-517 Federal Monitoring Surveys for Nursing Homes
Appendix III: Percentage of Comparative 

Surveys Identifying Missed Deficiencies with 

Potential for More Than Minimal Harm

State 

Number of 

homes in 

fiscal year 2007 

Total

comparative 

surveys

Total comparative 

surveys with at least 

one missed

D-F deficiency

Percentage of total 

comparative surveys with

at least one missed

D-F deficiency

Total number of

missed

D-F deficiencies

North Carolina 422 21 17 81.0 48

North Dakota 83 12 11 91.7 32

Ohio 984 31 12 38.7 20

Oklahoma 348 20 15 75.0 96

Oregon 139 18 12 66.7 30

Pennsylvania 724 37 23 62.2 66

Rhode Island 87 12 9 75.0 14

South Carolina 176 18 15 83.3 59

South Dakota 112 12 12 100.0 44

Tennessee 332 19 16 84.2 50

Texas 1,195 38 29 76.3 119

Utah 94 11 11 100.0 94

Vermont 41 10 4 40.0 18

Virginia 281 17 12 70.6 31

Washington 247 18 10 55.6 20

West Virginia 133 13 3 23.1 3

Wisconsin 401 21 8 38.1 19

Wyoming 39 12 12 100.0 83

Nation 16,114 976 667 68.3 2,431

Source: GAO analysis of CMS’s OSCAR data for the number of homes and federal monitoring survey data for all other data presented. 

Page 36 GAO-08-517 Federal Monitoring Surveys for Nursing Homes
Appendix IV: Percentage of Comparative 

Surveys with at Least One Missed Deficiency, Appendix IV by Federal Quality Standard Category : Percentage of Comparative 

Surveys with at Least One Missed Deficiency, 

by Federal Quality Standard Category 

Percentage of total comparative surveys 

citing at least one missed deficiency 

Federal quality standard category 

At the D through 

F level 

At the G through 

L level

Quality of Care 31.7 11.9

Resident Behavior and Facility Practices 17.7 2.2

Resident Assessment 31.7 1.6

Administration 13.3 1.4

Resident Rights 10.8 0.5

Quality of Life 18.9 0.5

Nursing Services 1.3 0.4

Other 0.2 0.2

Dietary Services 18.5 0.1

Physician Services 3.0 0.1

Dental Services 0.2 0.1

Pharmacy Services 9.7 0.1

Infection Control 9.8 0.1

Physical Environment 14.5 0.1

Admission, Transfer, and Discharge Rights 0.3 0.0

Specialized Rehabilitative 0.3 0.0

National averagea 68.3 14.5

Source: GAO analysis of federal monitoring survey data. 

a

Percentages for both the D through F and G through L levels do not total the national average 

because some surveys cited missed deficiencies in multiple categories. 

Page 37 GAO-08-517 Federal Monitoring Surveys for Nursing Homes
Appendix V: Percentage of Below Satisfactory 

State Survey Ratings for General

Investigation and Deficiency Determination 

 Percentage of state surveys

receiving below satisfactory ratingsa

State 

Percentage of total 

comparative surveys 

with at least one

missed G-L deficiency

General 

Investigation

Deficiency 

Determination

Alabama 27.8 20.0 22.7

Alaska 0.0 5.3 5.3

Arizona 26.7 7.4 15.4

Arkansas 5.6 3.5 7.0

California 10.2 10.7 6.1

Colorado 13.6 4.7 0.0

Connecticut 6.3 7.4 7.4

Delaware 15.4 0.0 11.8

District of Columbia 8.3 5.6 5.6

Florida 11.5 16.6 19.5

Georgia 16.7 12.5 21.6

Hawaii 8.3 5.6 11.1

Idaho 0.0 5.6 0.0

Illinois 21.9 4.7 2.4

Indiana 12.0 2.3 3.9

Iowa 15.8 1.7 1.7

Kansas 16.7 2.3 4.7

Kentucky 11.1 11.4 21.4

Louisiana 17.6 14.9 19.4

Maine 0.0 12.5 4.2

Maryland 10.5 2.0 6.0

Massachusetts 5.9 0.8 0.8

Michigan 20.0 2.9 6.8

Minnesota 9.5 5.9 3.0

Mississippi 22.2 16.7 21.4

Missouri 28.6 17.6 22.1

Montana 16.7 10.5 0.0

Nebraska 5.6 4.1 2.0

Nevada 8.3 11.1 11.1

New Hampshire 14.3 25.0 18.8

New Jersey 20.8 3.3 1.1

Appendix V: Percentage of Below 

Satisfactory State Survey Ratings for General 

Investigation and Deficiency Determination 

Page 38 GAO-08-517 Federal Monitoring Surveys for Nursing Homes
Appendix V: Percentage of Below Satisfactory 

State Survey Ratings for General

Investigation and Deficiency Determination 

 Percentage of state surveys

receiving below satisfactory ratingsa

State 

Percentage of total 

comparative surveys 

with at least one

missed G-L deficiency

General 

Investigation

Deficiency 

Determination

New Mexico 33.3 26.3 31.6

New York 22.2 14.4 14.5

North Carolina 14.3 12.3 23.6

North Dakota 0.0 0.0 0.0

Ohio 3.2 0.8 2.7

Oklahoma 30.0 12.1 16.5

Oregon 0.0 4.3 8.7

Pennsylvania 16.2 4.3 5.3

Rhode Island 16.7 5.6 11.1

South Carolina 33.3 14.3 22.9

South Dakota 33.3 0.0 0.0

Tennessee 26.3 14.6 20.7

Texas 13.2 8.0 12.0

Utah 9.1 0.0 5.6

Vermont 0.0 0.0 0.0

Virginia 5.9 4.8 6.3

Washington 11.1 5.2 3.4

West Virginia 0.0 3.6 7.1

Wisconsin 9.5 6.1 3.0

Wyoming 33.3 0.0 0.0

Nation 14.5 7.7 9.2

Source: GAO analysis of federal monitoring survey data. 

a

Federal observational surveys use a five-point rating scale to evaluate state survey teams. Our 

analysis collapsed the ratings in the lowest two categories—much less than satisfactory and less than

satisfactory—into a single category of below satisfactory results. 

Page 39 GAO-08-517 Federal Monitoring Surveys for Nursing Homes
Appendix VI: Comments from the Department Appendix VI: Comments from the of Health and Human Services 

Department of Health & Human Services 

Page 40 GAO-08-517 Federal Monitoring Surveys for Nursing Homes
Appendix VI: Comments from the Department 

of Health and Human Services 

Page 41 GAO-08-517 Federal Monitoring Surveys for Nursing Homes
Appendix VI: Comments from the Department 

of Health and Human Services 

Page 42 GAO-08-517 Federal Monitoring Surveys for Nursing Homes
Appendix VI: Comments from the Department 

of Health and Human Services 

Page 43 GAO-08-517 Federal Monitoring Surveys for Nursing Homes
Appendix VI: Comments from the Department 

of Health and Human Services 

Page 44 GAO-08-517 Federal Monitoring Surveys for Nursing Homes
Appendix VI: Comments from the Department 

of Health and Human Services 

Page 45 GAO-08-517 Federal Monitoring Surveys for Nursing Homes
Appendix VI: Comments from the Department 

of Health and Human Services 

Page 46 GAO-08-517 Federal Monitoring Surveys for Nursing Homes
Appendix VI: Comments from the Department 

of Health and Human Services 

Page 47 GAO-08-517 Federal Monitoring Surveys for Nursing Homes
Appendix VI: Comments from the Department 

of Health and Human Services 

Page 48 GAO-08-517 Federal Monitoring Surveys for Nursing Homes
Appendix VII: 

A

GAO Contact and Staff 

cknowledgments 

Page 49 GAO-08-517

Appendix VII: GAO Contact and Staff 

Acknowledgments 

GAO Contact John E. Dicken, (202) 512-7114 or dickenj@gao.gov 

In addition to the contact named above, Walter Ochinko, Assistant 

Director; Katherine Nicole Laubacher; Dan Lee; Elizabeth T. Morrison; 

Steve Robblee; Karin Wallestad; and Rachael Wojnowicz made key 

contributions to this report. 

Acknowledgments 

Federal Monitoring Surveys for Nursing Homes
Related GAO Products Related GAO Products 

Nursing Home Reform: Continued Attention Is Needed to Improve 

Quality of Care in Small but Significant Share of Homes. GAO-07-794T. 

Washington, D.C.: May 2, 2007. 

Nursing Homes: Efforts to Strengthen Federal Enforcement Have Not 

Deterred Some Homes from Repeatedly Harming Residents. GAO-07-241. 

Washington, D.C.: March 26, 2007. 

Nursing Homes: Despite Increased Oversight, Challenges Remain in 

Ensuring High-Quality Care and Resident Safety. GAO-06-117. 

Washington, D.C.: December 28, 2005. 

Nursing Home Deaths: Arkansas Coroner Referrals Confirm Weaknesses 

in State and Federal Oversight of Quality of Care. GAO-05-78. 

Washington, D.C.: November 12, 2004. 

Nursing Home Fire Safety: Recent Fires Highlight Weaknesses in 

Federal Standards and Oversight. GAO-04-660. Washington D.C.: July 16, 

2004. 

Nursing Home Quality: Prevalence of Serious Problems, While 

Declining, Reinforces Importance of Enhanced Oversight. GAO-03-561. 

Washington, D.C.: July 15, 2003. 

Nursing Homes: Public Reporting of Quality Indicators Has Merit, but 

National Implementation Is Premature. GAO-03-187. Washington, D.C.: 

October 31, 2002. 

Nursing Homes: Quality of Care More Related to Staffing than Spending. 

GAO-02-431R. Washington, D.C.: June 13, 2002. 

Nursing Homes: More Can Be Done to Protect Residents from Abuse. 

GAO-02-312. Washington, D.C.: March 1, 2002. 

Nursing Homes: Federal Efforts to Monitor Resident Assessment Data 

Should Complement State Activities. GAO-02-279. Washington, D.C.: 

February 15, 2002. 

Nursing Homes: Sustained Efforts Are Essential to Realize Potential of 

the Quality Initiatives. GAO/HEHS-00-197. Washington, D.C.: 

September 28, 2000. 

Page 50 GAO-08-517 Federal Monitoring Surveys for Nursing Homes
Related GAO Products 

Nursing Home Care: Enhanced HCFA Oversight of State Programs 

Would Better Ensure Quality. GAO/HEHS-00-6. Washington, D.C.: 

November 4, 1999. 

Nursing Home Oversight: Industry Examples Do Not Demonstrate That 

Regulatory Actions Were Unreasonable. GAO/HEHS-99-154R. Washington, 

D.C.: August 13, 1999. 

Nursing Homes: Proposal to Enhance Oversight of Poorly Performing 

Homes Has Merit. GAO/HEHS-99-157. Washington, D.C.: June 30, 1999. 

Nursing Homes: Complaint Investigation Processes Often Inadequate to 

Protect Residents. GAO/HEHS-99-80. Washington, D.C.: March 22, 1999. 

Nursing Homes: Additional Steps Needed to Strengthen Enforcement of 

Federal Quality Standards. GAO/HEHS-99-46. Washington, D.C.: 

March 18, 1999. 

California Nursing Homes: Care Problems Persist Despite Federal and

State Oversight. GAO/HEHS-98-202. Washington, D.C.: July 27, 1998. 

(290676) Page 51 GAO-08-517 Federal Monitoring Surveys for Nursing Homes
GAO’s Mission The Government Accountability Office, the audit, evaluation, and 

investigative arm of Congress, exists to support Congress in meeting its 

constitutional responsibilities and to help improve the performance and 

accountability of the federal government for the American people. GAO 

examines the use of public funds; evaluates federal programs and policies; 

and provides analyses, recommendations, and other assistance to help

Congress make informed oversight, policy, and funding decisions. GAO’s 

commitment to good government is reflected in its core values of 

accountability, integrity, and reliability. 

The fastest and easiest way to obtain copies of GAO documents at no cost

is through GAO’s Web site (www.gao.gov). Each weekday, GAO posts 

newly released reports, testimony, and correspondence on its Web site. To

have GAO e-mail you a list of newly posted products every afternoon, go 

to www.gao.gov and select “E-mail Updates.” 

The first copy of each printed report is free. Additional copies are $2 each. 

A check or money order should be made out to the Superintendent of 

Documents. GAO also accepts VISA and Mastercard. Orders for 100 or 

more copies mailed to a single address are discounted 25 percent. Orders 

should be sent to: 

U.S. Government Accountability Office 

441 G Street NW, Room LM 

Washington, DC 20548 

To order by Phone: Voice: (202) 512-6000 

TDD: (202) 512-2537 

Fax: (202) 512-6061 

Contact: 

Web site: www.gao.gov/fraudnet/fraudnet.htm

E-mail: fraudnet@gao.gov

Automated answering system: (800) 424-5454 or (202) 512-7470 

Ralph Dawn, Managing Director, dawnr@gao.gov, (202) 512-4400 

U.S. Government Accountability Office, 441 G Street NW, Room 7125 

Washington, DC 20548 

Chuck Young, Managing Director, youngc1@gao.gov, (202) 512-4800 

U.S. Government Accountability Office, 441 G Street NW, Room 7149 

Washington, DC 20548 

Obtaining Copies of 

GAO Reports and 

Testimony

Order by Mail or Phone 

To Report Fraud,

Waste, and Abuse in 

Federal Programs 

Congressional 

Relations 

Public Affairs 

PRINTED ON RECYCLED PAPER