Department of Health and Human Services 

OFFICE OF 

INSPECTOR GENERAL 

Inspector General 

June 2004 

OEI-01-03-00130 

N 

INSPECTION RESULTS ON 

NURSING HOME COMPARE: 

COMPLETENESS AND ACCURACY 
Office of Inspector General
 http://oig.hhs.gov 

The mission of the Office of Inspector General (OIG), as mandated by Public Law 95-452, 

as amended, is to protect the integrity of the Department of Health and Human Services 

(HHS) programs, as well as the health and welfare of beneficiaries served by those 

programs. This statutory mission is carried out through a nationwide network of audits, 

investigations, and inspections conducted by the following operating components: 

Office of Audit Services 

The OIG's Office of Audit Services (OAS) provides all auditing services for HHS, either by 

conducting audits with its own audit resources or by overseeing audit work done by others. 

Audits examine the performance of HHS programs and/or its grantees and contractors 

in carrying out their respective responsibilities and are intended to provide independent 

assessments of HHS programs and operations in order to reduce waste, abuse, and 

mismanagement and to promote economy and efficiency throughout the department. 

Office of Evaluation and Inspections 

The OIG's Office of Evaluation and Inspections (OEI) conducts short-term management 

and program evaluations (called inspections) that focus on issues of concern to the 

department, the Congress, and the public. The findings and recommendations contained 

in the inspections reports generate rapid, accurate, and up-to-date information on the 

efficiency, vulnerability, and effectiveness of departmental programs. The OEI also 

oversees state Medicaid fraud control units, which investigate and prosecute fraud and 

patient abuse in the Medicaid program. 

Office of Investigations 

The OIG's Office of Investigations (OI) conducts criminal, civil, and administrative 

investigations of allegations of wrongdoing in HHS programs or to HHS beneficiaries 

and of unjust enrichment by providers. The investigative efforts of OI lead to criminal 

convictions, administrative sanctions, or civil monetary penalties. 

Office of Counsel to the Inspector General 

The Office of Counsel to the Inspector General (OCIG) provides general legal services to 

OIG, rendering advice and opinions on HHS programs and operations and providing all 

legal support in OIG's internal operations. The OCIG imposes program exclusions and 

civil monetary penalties on health care providers and litigates those actions within the 

department. The OCIG also represents OIG in the global settlement of cases arising 

under the Civil False Claims Act, develops and monitors corporate integrity agreements, 

develops model compliance plans, renders advisory opinions on OIG sanctions to the 

health care community, and issues fraud alerts and other industry guidance. 


� ABSTRACT 
 

Nursing Home Compare is a Web site maintained by the Centers for 

Medicare & Medicaid Services (CMS) that provides consumers with 

information on nursing home quality. Consumers use the information 

on the site to help them choose the nursing home that best fits their 

needs. Therefore, it is essential that the site accurately portray all 

Medicare- and Medicaid-certified nursing homes. Using CMS databases 

and State survey documentation, we determined whether the site 

contained all Medicare- and Medicaid-certified nursing homes, and for a 

random sample of nursing homes, determined the accuracy of the 

inspection results provided on the site. We found that Nursing Home 

Compare contains nearly all Medicare- and Medicaid-certified nursing 

homes. However, one or more surveys were missing from 19 percent of 

nursing homes, leaving consumers with incomplete information about 

those homes' survey and complaint histories. Inspection results on 

Nursing Home Compare are largely accurate, but one or more 

deficiencies were missing from 11 percent of nursing homes’ inspection 

results, and Nursing Home Compare presents deficiencies not found on 

State survey documentation for 15 percent of nursing homes. 

Inaccuracies may be due to late data entry by State survey agencies, no 

tracking of inaccuracy reports by CMS, and failure of State survey 

agencies to transmit data on amended deficiencies. We recommend that 

CMS require State agencies to verify that the most recent inspection 

results are in CMS databases and establish a single point of contact for 

reporting discrepancies on the Web site. 

OEI - 01 - 03 - 00130 INSPECTION R ESULTS ON N URSING H OME C OMPARE : COMPLETENESS AND A CCURACY i 
� EXECUTIVE SUMMARY 
 

OBJECTIVES 

To determine if Nursing Home Compare includes all Medicare- and 

Medicaid-certified nursing homes. 

To determine whether Nursing Home Compare includes all intended 

inspection results. 

To determine the accuracy of nursing home inspection results provided 

on the Nursing Home Compare Web site. 

BACKGROUND 

In 1998, the Department of Health and Human Services (HHS) 

launched the Nursing Home Compare Web site. The site is maintained 

by the Centers for Medicare & Medicaid Services (CMS). Nursing Home 

Compare provides information about the past performance of all 

Medicare- and Medicaid-certified nursing homes in the country. The 

site also serves as a source of public information on nursing home 

quality. Consumers can use the information provided on the site to help 

them choose the nursing home that best fits their needs. Therefore, it is 

important that the site accurately portray all Medicare- and Medicaidcertified nursing homes. 

This report focuses on the inspection results section of Nursing Home 

Compare. This section provides results from a nursing home's three 

most recent Medicare certification surveys (referred to as health surveys 

on Nursing Home Compare and in this report) and any complaint 

investigations. The information in the inspection results section 

includes type of deficiency, survey date, date the deficiency was 

corrected, and scope and severity of the deficiency. 

We base this inspection on the Nursing Home Compare database as of 

April 1, 2003. To determine completeness, we matched the nursing 

homes on the site with those on a CMS database that contains 

information on Medicare- and Medicaid-certified nursing homes. To 

determine accuracy, we matched Nursing Home Compare inspection 

results from a simple random sample of 150 nursing homes with State 

survey documentation. The State survey documentation includes 

statements of deficiency from health surveys and complaint 

investigations, as well as informal dispute resolution outcomes. 

OEI - 01 - 03 - 00130 INSPECTION R ESULTS ON N URSING H OME C OMPARE : COMPLETENESS AND A CCURACY ii 
EXECUTIVE SUMMARY 

FINDINGS 

Nursing Home Compare contains almost all Medicare- and Medicaidcertified nursing homes. Less than 1 percent of nursing homes did not 

match to either Nursing Home Compare or the Minimum Data Set 

(MDS) facility file, a demographic database of nursing homes 

maintained by CMS. Nursing Home Compare excluded 55 nursing 

homes (0.2 percent) included in the MDS facility file. Nursing Home 

Compare included 119 nursing homes (0.7 percent) that were not found 

in the MDS facility file over a 5-month period. 

Nineteen percent of nursing homes had one or more surveys 

missing on Nursing Home Compare. Nursing Home Compare lacked 

the most current health survey for 7 percent of nursing homes, meaning 

that it instead provided older information. It also lacked either a 

complaint investigation or the second or third most recent health survey 

for 12 percent of nnursing homes. Furthermore, when Nursing Home 

Compare does not include deficiencies from complaint investigations, 

consumers have no way of determining that. 

Inspection results on Nursing Home Compare are largely accurate, 

but some areas of concern exist. Sixty-eight percent of nursing 

homes had Nursing Home Compare inspection results that exactly 

matched State survey documentation. Eleven percent of nursing homes 

had one or more deficiencies missing from their inspection results on 

Nursing Home Compare. Furthermore, Nursing Home Compare 

presents deficiencies not found on State survey documentation for 15 

percent of nursing homes. These inaccuracies may be due to late data 

entry by State survey agencies, no tracking of inaccuracy reports by 

CMS, and/or failure of State survey agencies to transmit data on 

amended deficiencies. 

RECOMMENDATIONS 

Overall, Nursing Home Compare's inspection results provide consumers 

with a valuable resource on the past performance of Medicare- and 

Medicaid- certified nursing homes. However, we found some 

inaccuracies between Nursing Home Compare and State survey reports 

that deserve attention, and suggest that CMS could do more to ensure 

that consumers receive the most accurate data possible when choosing a 

nursing home. 

OEI - 01 - 03 - 00130 INSPECTION R ESULTS ON N URSING H OME C OMPARE : COMPLETENESS AND A CCURACY iii 
EXECUTIVE SUMMARY 

CMS should work to enhance Nursing Home Compare by: 

Requiring State agencies to verify that the most recent inspection 

results are in CMS databases. Timely and accurate inspection results 

on Nursing Home Compare depend on State surveyors transmitting 

data into the Online Survey Certification and Reporting system 

(OSCAR), the CMS database containing survey information. To ensure 

that data have been promptly transmitted into OSCAR, State agencies 

should verify the most recent inspection results in OSCAR with those in 

the Automated Survey Processing Environment (ASPEN) system, the 

application used for survey data entry and transmission to OSCAR. 

Establishing a single point of contact for reporting Nursing Home 

Compare inaccuracies. By having a single point of contact within 

CMS for reporting inaccuracies, CMS would be able to track and 

conduct follow-up on information that is reported to be inaccurate. 

Agency Comments 

CMS agreed with our first recommendation to require State agencies to 

verify that the most recent inspection results are in Nursing Home 

Compare, citing it has performance standards under development. 

CMS disagreed with our second recommendation to establish a single 

point of contact for reporting Nursing Home Compare inaccuracies. The 

agency commented that “multiple CMS components are involved in 

providing Nursing Home Compare data on the Web site”, making a 

single point of contact impractical. CMS, however, will consider adding 

regional office contact information to Nursing Home Compare to 

facilitate corrections to the Web site. We continue to believe that a 

single point of contact for reporting inaccuracies would be the most 

effective solution. However, we appreciate CMS’s consideration of 

adding regional office contact information to Nursing Home Compare, 

and urge CMS to follow through on this idea. 

OEI - 01 - 03 - 00130 INSPECTION R ESULTS ON N URSING H OME C OMPARE : COMPLETENESS AND A CCURACY iv 
� TABLE OF CONTENTS 
 

ABSTRACT ................................ ....................... i
 

EXECUTIVE SUMMARY ................................ ......... ii
 

INTRODUCTION ................................ .................. 1
 

FINDINGS ................................ ........................ 6
 

Nursing Home Compare contains almost all nursing homes ............ 6
 

Nursing Home Compare does not contain all surveys....................... 7
 

Inspection results are largely accurate, but areas of concern exist... 8
 

RECOMMENDATIONS ................................ ........... 12
 

APPENDICES ................................ .................... 14
 

A: Definitions of scope and severity of deficiencies ........................ 14
 

B: Methodology................................................................................ 15
 

C: Agency Comments ...................................................................... 19
 

ACKNOWLEDGMENTS ................................ .......... 22
 

E ND NOTES ................................ ..................... 23
 
� INTRODUCTION 
 

OBJECTIVES 

To determine if Nursing Home Compare includes all Medicare- and 

Medicaid-certified nursing homes. 

To determine whether Nursing Home Compare includes all intended 

inspection results. 

To determine the accuracy of nursing home inspection results provided 

on the Nursing Home Compare Web site. 

BACKGROUND 

Nursing Home Compare 

In 1998, the Department of Health and Human Services (HHS) 

launched the Nursing Home Compare Web site. The purposes of the 

site are "to provide detailed information about the past performance of 

every Medicare- and Medicaid-certified nursing home in the country" 

and to serve as a source of public information on nursing home quality.1 

The site allows Medicare beneficiaries and their caregivers access to 

individual nursing home quality information to help them choose the 

nursing home that best fits their needs. Consumers access Nursing 

Home Compare through the main consumer Medicare Web site, 

www.medicare.gov. 

The Centers for Medicare & Medicaid Services (CMS) operates Nursing 

Home Compare. CMS intends the information on the site to be used 

with other sources of information, such as a visit to a nursing home. 

Currently, the site provides information on about 17,000 nursing homes. 

Between March 2002 and July 2003, Nursing Home Compare received 

an average of 134,000 visits per month. Nursing Home Compare 

presents four categories of information: 

•	 inspection results, including deficiencies from Medicare 

certification surveys and complaint investigations 

•	 facility characteristics, such as number of beds and type of 

ownership 

• nursing home staffing levels 

•	 quality measures, which provide information on clinical and 

physical characteristics of a nursing home's residents 

OEI - 01 - 03 - 00130 INSPECTION R ESULTS ON N URSING H OME C OMPARE : COMPLETENESS AND A CCURACY 1 
INTRODUCTION 

Nursing Home Compare Inspection Results 

The inspection results section on Nursing Home Compare lists 

deficiencies from the three most recent Medicare certification surveys 

(referred to as health surveys on Nursing Home Compare and in this 

report), as well as those from substantiated complaint investigations 

conducted in the past 3 years for each nursing home. State survey 

agencies conduct nursing home surveys on behalf of CMS. Health 

surveys, which evaluate the quality of care and the services provided by 

the nursing home, occur on an average of every 12 months but no less 

frequently than every 15 months. State surveyors also conduct 

complaint investigations, as necessary, outside of the cycle of health 

surveys and cite deficiencies, as appropriate. 

Nursing Home Compare clusters deficiencies from each of the three 

health surveys with deficiencies from complaint investigations for an 

approximate 1-year period. The site describes each deficiency, provides 

the date that the deficiency was cited and corrected, and lists the scope 

and severity of the deficiency (see Figure 1). Severity refers to the level 

of harm, and scope refers to the number of affected residents. Nursing 

Home Compare also provides definitions for scope and severity (see 

Appendix A). 

FIGURE 1 

Deficiency Information 

Presented on Nursing 

Home Compare 

Deficiency and 

date cited 

Date deficiency 

was rectified 

Severity of 

deficiency 

Scope of 

deficiency 

Source: Nursing Home Compare Web site, Centers for Medicare & Medicaid Services, August 2003 

Nursing Home Compare Inspection Results and OSCAR 

Inspection results on Nursing Home Compare come from CMS's Online 

Survey Certification and Reporting (OSCAR) system, a database 

containing survey information for all Medicare- and Medicaid-certified 

nursing homes. After completion of a health survey or complaint 

investigation, State surveyors enter inspection results into the 

OEI - 01 - 03 - 00130 INSPECTION R ESULTS ON N URSING H OME C OMPARE : COMPLETENESS AND A CCURACY 2 
INTRODUCTION 

Automated Survey Processing Environment (ASPEN) system. 

Surveyors use ASPEN to electronically transmit inspection results into 

OSCAR. According to CMS's State Survey Agency Performance 

Standards for Fiscal Year 2003, the average time for transmitting data 

from health surveys and complaint investigations into OSCAR must not 

exceed 70 calendar days from the last day of the survey. 

Nursing Home Compare does not include information for deficiencies 

that are under dispute and notes this in a disclaimer on the site. 

Nursing homes can appeal deficiencies from health surveys and 

complaint investigations through the State survey agency with informal 

dispute resolution. Any deficiency undergoing this process is not 

transmitted into OSCAR. When informal dispute resolution concludes, 

State survey agencies transmit any deficiencies that remain into 

OSCAR. These deficiencies will then appear on Nursing Home 

Compare. 

CMS updates the inspection results on Nursing Home Compare 

monthly. At the beginning of each month, data analysts in CMS's 

Center for Medicaid and State Operations extract data from OSCAR. 

CMS's Center for Beneficiary Choices, which is responsible for the 

appearance and content of Nursing Home Compare, then provides the 

data to its contractor who maintains the Nursing Home Compare Web 

site. The contractor incorporates the new data into Nursing Home 

Compare by the end of each month. 

Prominence of Nursing Home Compare 

According to CMS officials, about 20 percent of visitors to the main 

Medicare Web site access Nursing Home Compare, making Nursing 

Home Compare one of the most frequently visited sections of the 

Medicare Web site.2 Nursing Home Compare's popularity likely can be 

attributed to nationwide promotion of the Web site. In November 2002, 

CMS placed advertisements in 71 newspapers across all 50 States. 

CMS also promoted Nursing Home Compare in national television 

advertisements. The media campaign aimed to increase public 

awareness of the site as a principal source of nursing home quality 

information. Prior to the media campaign and launch of the quality 

measures, Nursing Home Compare received fewer than 100,000 visits 

per month. In November 2002, Nursing Home Compare received about 

400,000 visits. 

CMS has used Nursing Home Compare as a model for reporting quality 

information to consumers on other types of Medicare-certified facilities 

OEI - 01 - 03 - 00130 INSPECTION R ESULTS ON N URSING H OME C OMPARE : COMPLETENESS AND A CCURACY 3 
INTRODUCTION 

and services. Currently, consumers can access Home Health Compare 

and Dialysis Facility Compare from the Medicare Web site. An 

initiative to report quality information for hospitals is in progress. 

Concerns about Nursing Home Compare 

In February 2002, the House of Representatives' Committee on 

Government Reform released the report, HHS 'Nursing Home Compare' 

Website Has Major Flaws. The report found that Nursing Home 

Compare excluded deficiencies from complaint investigations, thus not 

presenting a complete picture of the nursing home. In May 2002, CMS 

added deficiencies from complaint investigations to Nursing Home 

Compare's inspection results section. 

In October 2002, the General Accounting Office released the report, 

Nursing Homes: Public Reporting of Quality Indicators Has Merit, but 

National Implementation is Premature. The report raised concerns 

about the validity and usefulness of the quality measures posted on 

Nursing Home Compare and concluded that CMS was prematurely 

reporting the measures. Furthermore, the report stated that CMS was 

not adequately prepared to address questions raised by the public 

concerning the site. In its comments on the report, CMS affirmed its 

decision to release the quality measures as a means of providing 

consumer information and public accountability. 

This Inspection 

Considering the efforts to increase public awareness of Nursing Home 

Compare as a tool for consumers, it is essential that the site accurately 

portray all Medicare- and Medicaid-certified nursing homes. This 

inspection focuses on the completeness of the Nursing Home Compare 

Web site, as well as the accuracy of the inspection results section of the 

site. Determining the completeness of Nursing Home Compare shows 

whether the site contains all Medicare- and Medicaid-certified nursing 

homes. Assessing the accuracy of inspection results shows whether 

Nursing Home Compare contains the inspection results it claims to 

present, as well as whether it presents consumers with correct and 

reliable information. 

This inspection does not assess the accuracy of the other three 

categories of information on Nursing Home Compare: facility 

characteristics, staffing levels, and quality measures. 

OEI - 01 - 03 - 00130 INSPECTION R ESULTS ON N URSING H OME C OMPARE : COMPLETENESS AND A CCURACY 4 
INTRODUCTION 

METHODOLOGY 

This inspection is based on the Nursing Home Compare database as of 

April 1, 2003. On this date, Nursing Home Compare included 16,446 

Medicare- and Medicaid-certified nursing homes. 

To determine the completeness of Nursing Home Compare, we matched 

provider numbers and addresses of all nursing homes included in 

Nursing Home Compare with nursing homes included in the Minimum 

Data Set (MDS) facility file, a demographic data file of nursing homes 

that submit resident assessments. 

To determine whether Nursing Home Compare includes all intended 

inspection results, as well as the accuracy of the inspection results 

provided by Nursing Home Compare, we selected a national, simple 

random sample of 150 nursing homes from Nursing Home Compare. 

We then requested survey documentation from State survey agencies. 

Survey documentation consisted of statements of deficiency for health 

surveys and complaint investigations as well as outcomes from informal 

dispute resolution, if relevant, for the sampled nursing homes. Our 

response rate from the State survey agencies was 100 percent. We 

matched survey documentation to the information presented on Nursing 

Home Compare for each nursing home. With this information, we 

assessed whether any health or complaint surveys were missing from 

Nursing Home Compare. The information we matched for accuracy 

included dates of surveys, number of deficiencies, and the scope and 

severity of deficiencies. 

We interviewed staff in CMS central and regional offices who work with 

Nursing Home Compare and survey and certification of nursing homes. 

We also reviewed documentation relevant to Nursing Home Compare's 

inspection results, including CMS's State Operations Manual and State 

Survey Agency Performance Standards for Fiscal Year 2003. Appendix 

B contains a detailed description of our methodology. 

We conducted this inspection in accordance with the Quality Standards 

for Inspections issued by the President's Council on Integrity and 

Efficiency. 

OEI - 01 - 03 - 00130 INSPECTION R ESULTS ON N URSING H OME C OMPARE : COMPLETENESS AND A CCURACY 5 
�FINDINGS FINDINGS 

Nursing Home Compare contains almost all In order to fully provide consumers 

Medicare- and Medicaid- certified nursing homes. with all available options for 

nursing homes, Nursing Home 

Compare must contain all Medicare- and Medicaid-certified nursing 

homes. Our match of the 16,446 nursing homes in Nursing Home 

Compare and the MDS facility file showed that Nursing Home Compare 

included almost all nursing homes (see Figure 2). 

FIGURE 2 

Match of Nursing Home 

Compare database with 

the MDS facility file 

Nursing Homes in 

Nursing Home Compare 

NOT MDS Facility File 

0.7 percent (n=119) 

Nursing Homes in 

MDS Facility File 

NOT Nursing Home Compare 

0.2 percent (n=55) 

Nursing Homes in 

Nursing Home Compare 

and MDS Facility File 

Source: OIG analysis of nursing homes in Nursing Home Compare and the MDS facility file, 2003 

In all, less than 1 percent of nursing homes did not appear in both 

Nursing Home Compare and the MDS facility file. Nursing Home 

Compare included 119 nursing homes (0.7 percent) that were not found 

in the MDS facility file over a 5-month period. Nursing Home Compare 

did not include 55 nursing homes (0.2 percent) included in the MDS 

facility file. These 55 nursing homes submitted MDS data, yet no 

evidence of their operation exists in Nursing Home Compare. 

A closer look at the 119 nursing homes from Nursing Home Compare 

not found in the MDS facility file showed that Nursing Home Compare 

may erroneously include many of these nursing homes. For example, 14 

of the 119 nursing homes had not had a health survey in 2 years, 

though CMS requires nursing homes to be surveyed every 15 months 

OEI - 01 - 03 - 00130 INSPECTION R ESULTS ON N URSING H OME C OMPARE : COMPLETENESS AND A CCURACY 6 
FINDINGS 

(107 of the 119 were surveyed within the 2-year period). To determine 

whether some of the 119 nursing homes might be closed, we contacted a 

survey agency in 1 State to confirm the operating status of its 9 nursing 

homes that were in Nursing Home Compare but not the MDS facility 

file. The State survey agency informed us that seven of the nine 

nursing homes were closed. The October 2003 update of Nursing Home 

Compare still included all seven of the closed nursing homes. In 

addition to the potentially closed nursing homes, six nursing homes had 

obtained new provider numbers and were listed twice in Nursing Home 

Compare with both the new and old provider numbers. 

Nineteen percent of nursing homes had one or more Nursing Home Compare informs 

surveys missing on Nursing Home Compare.	 consumers that the inspection 

results section includes 

deficiencies from the three most recent health surveys as well as from 

recent substantiated complaint investigations. 3 State survey agencies 

have 70 days after the last day of a survey to transmit survey results 

into OSCAR. However, Nursing Home Compare lacked one or more of 

the three most recent health surveys and/or complaint investigations 

that should have been there for 19 percent of nursing homes.4 These 

surveys and investigations included 206 deficiencies unaccounted for on 

Nursing Home Compare. Nursing home Compare lacked the most 

recent health survey for 7 percent of nursing homes, meaning that it 

instead provided older information. 

If Nursing Home Compare does not reflect the most recent inspection 

results, consumers may be misled by the information presented. For 

example, the 2 most recent surveys occurring in 2002 and 2000 for 1 

nursing home yielded 40 and 14 deficiencies, respectively. However, 

Nursing Home Compare's inspection results show three older surveys 

for this nursing home. The three surveys on Nursing Home Compare 

occurred in 1999, 1998, and 1997, and yielded 18, 9, and 2 deficiencies, 

respectively. 

Furthermore, Nursing Home Compare lacked complaint investigations 

for 8 percent of nursing homes. A complaint investigation on Nursing 

Home Compare provides an additional level of detail about the 

performance of a nursing home, but when it is missing, the nursing 

home erroneously appears more favorable to the consumer. 

Late data entry by State survey agencies may explain most of the 

missing health surveys. As the majority of the missing health surveys 

OEI - 01 - 03 - 00130 INSPECTION R ESULTS ON N URSING H OME C OMPARE : COMPLETENESS AND A CCURACY 7 
FINDINGS 

and complaint investigations could be found in Nursing Home Compare 

by the latter part of 2003, State survey agencies apparently exceeded 

the 70-day time period required to enter results into OSCAR. However, 

only the recent missing health surveys and complaint investigations can 

be attributed to late data entry, as older health surveys and complaint 

investigations were still missing. CMS evaluates the accuracy and 

timeliness of data entry by State survey agencies semiannually.5 

However, this evaluation is a retrospective review, so missing surveys 

could go undetected for as long as 6 months. 

Inspection results on Nursing Home Compare are If State survey documentation 

largely accurate, but some areas of concern exist.	 exactly matched the inspection 

results on Nursing Home 

Compare for our sample of 150 nursing homes, we considered that 

information on the site to be accurate. Five different types of 

inaccuracies existed between Nursing Home Compare inspection results 

and State survey documentation. Table 1 shows the number and 

percentage of each type of inaccuracy. As some nursing homes had 

more than one inaccuracy between Nursing Home Compare and State 

survey documentation, percentages listed do not total 100 percent. 

Below, we discuss Nursing Home Compare’s accurate inspection results, 

as well as the the two most significant types of inaccuracies. 

TABLE 1 

Number and Percentage 

of Nursing Homes with 

Inaccuracies between 

Nursing Home Compare 

and State Survey 

Documentation (n=150) 

Type of Inaccuracy Number of 

Nursing 

Homes with 

Inaccuracies 

Percent of 

Nursing Homes 

with 

Inaccuracies 

Deficiencies incorrectly posted on Nursing Home 

Compare 

23 15% 

Deficiencies missing from Nursing Home Compare 16 11% 

Incorrect scope and/or severity score 12 8% 

Incorrect deficiency date 6 4% 

Incorrect deficiency description 4 3% 

Source: OIG analysis of Nursing Home Compare inspection results and State survey documentation, 2003 

OEI - 01 - 03 - 00130 INSPECTION R ESULTS ON N URSING H OME C OMPARE : COMPLETENESS AND A CCURACY 8 
FINDINGS 

Sixty-eight percent of nursing homes had Nursing Home Compare 

inspection results that exactly matched State survey documentation. 

Accurate reporting of this information allows the consumer to 

confidently gather detailed information about the type of deficiency, the 

level of severity, and scope of the deficiency. It gives a picture of the 

timing of surveys and substantiated complaint investigations, and the 

deficiencies from each. For 68 percent of nursing homes, Nursing Home 

Compare accurately reported inspection results, based on a match to 

State survey documentation (see Figure 3). 

FIGURE 3
 

Percent of nursing 
 

homes by number of 
 

inaccuracies
 

68% 

15% 

7% 

1% 

9% 

None 

1 

2 

3 

4 or more 

Source: OIG analysis of Nursing Home Compare inspection results and State survey documentation, 2003 

Nursing homes with a low number of deficiencies tended to be 

accurately presented on Nursing Home Compare. Nursing homes with 

no inaccuracies between State survey documentation and Nursing 

Home Compare had significantly fewer deficiencies than nursing homes 

with inaccuracies. Across all 3 survey periods on Nursing Home 

Compare, nursing homes without inaccuracies had an average of 17 

deficiencies. Nursing homes with inaccuracies averaged 35 

deficiencies. 6 

OEI - 01 - 03 - 00130 INSPECTION R ESULTS ON N URSING H OME C OMPARE : COMPLETENESS AND A CCURACY 9 
FINDINGS 

Eleven percent of nursing homes had one or more deficiencies missing 

from their inspection results on Nursing Home Compare. 

These represent instances where Nursing Home Compare excluded one 

or more deficiencies from a nursing home's health survey or complaint 

investigation, based on State survey documentation. On average, 

Nursing Home Compare lacked two deficiencies for each of these 

nursing homes. Thus, 11 percent of nursing homes appeared to 

consumers to have fewer deficiencies than actually existed. 

Of the deficiencies missing on Nursing Home Compare, 43 percent had 

been unsuccessfully appealed through informal dispute resolution, thus 

were not eliminated by the process. While undergoing informal dispute 

resolution, a deficiency is withheld from OSCAR and not provided on 

Nursing Home Compare. If deficiencies remain after informal dispute 

resolution, it is the responsibility of the State survey agency to promptly 

update OSCAR. If the deficiency is not entered into OSCAR, the 

deficiency cannot be presented on Nursing Home Compare. 

For 1 nursing home, Nursing Home Compare shows 13 deficiencies for a 

nursing home's most recent health survey. However, the nursing 

home's survey results do not show an additional 11 deficiencies that 

were unsuccessfully appealed through informal dispute resolution. Two 

of the missing 11 deficiencies indicate citations for actual harm, which 

likely would be a caution to a consumer. CMS ranks deficiencies cited 

as actual harm the second-highest level of deficiency severity. 

CMS is unlikely to become aware of these inaccuracies unless nursing 

homes report them, which is also unlikely, since these inaccuracies 

portray nursing homes as better than they are. Even if CMS does 

become aware of these inaccuracies, it does not track reports of 

inaccuracies to determine whether they are recitified by State survey 

agencies. CMS may also find these inaccuracies through its semiannual 

evaluation of data accuracy and timeliness. However, CMS's review 

looks at only a random sample of surveys conducted during the fiscal 

year and permits an overall accuracy rate of 85 percent.7 

Nursing Home Compare presents deficiencies not found on State survey 

documentation for 15 percent of nursing homes. 

Deficiencies appeared on Nursing Home Compare that State survey 

documentation did not substantiate for 23 nursing homes we reviewed. 

These false positive deficiencies make these nursing homes appear to 

consumers to have more deficiencies than actually exist. In all, 63 false 

positive deficiencies exist, with about half coming from just 4 nursing 

OEI - 01 - 03 - 00130 INSPECTION R ESULTS ON N URSING H OME C OMPARE : COMPLETENESS AND A CCURACY 10 
FINDINGS 

homes. For 1 of the 4 nursing homes, 10 deficiencies from a health 

survey are duplicated on Nursing Home Compare for a date 2 months 

after the original survey. Survey documentation exists for these 

deficiencies for just one of the dates. Thus, Nursing Home Compare's 

inspection results for this nursing home appear worse than the nursing 

home actually is. 

Of the 23 nursing homes with false positives, 8 had 1 deficiency 

appearing on Nursing Home Compare that had been successfully 

appealed through informal dispute resolution. For these eight nursing 

homes, State survey agencies had incorrectly transmitted deficiencies 

under dispute into OSCAR, and then did not remove the deficiencies 

when resolved by informal dispute resolution. 

If nursing homes detect inaccuracies within their inspection results, 

Nursing Home Compare directs them to contact their respective State 

survey agency to correct the information in OSCAR. CMS does not 

track the number of such reports, the types of inaccuracies reported, or 

whether the State survey agency rectified the inaccuracies. 

OEI - 01 - 03 - 00130 INSPECTION R ESULTS ON N URSING H OME C OMPARE : COMPLETENESS AND A CCURACY 11 
0 RECOMMENDATIONS RECOMMENDATIONS 

Overall, Nursing Home Compare's inspection results provide consumers 

with a valuable resource on the past performance of Medicare- and 

Medicaid- certified nursing homes. However, we found some 

inaccuracies between Nursing Home Compare and State survey reports 

that deserve attention, and suggest that CMS could do more to ensure 

that consumers receive the most accurate data possible when choosing a 

nursing home. 

CMS should enhance Nursing Home Compare by: 

Requiring State agencies to verify that the most recent inspection results 

are in CMS databases. Timely and accurate inspection results on 

Nursing Home Compare depend on the transmission of survey data by 

State surveyors into OSCAR, the CMS database containing survey 

information. However, Nursing Home Compare did not include some 

cited deficiencies as well as entire surveys. In some cases, Nursing 

Home Compare did not reflect changes in deficiencies amended through 

informal dispute resolution. If State survey agencies do not transmit 

data into OSCAR, then changes will not be reflected on Nursing Home 

Compare. 

To assure that data have been promptly transmitted into OSCAR, State 

agencies should verify the most recent inspection results in OSCAR 

with those in ASPEN, the system used for survey data entry and 

transmission to OSCAR. This verification can be included as an offsite 

survey preparation task. Currently, CMS's offsite survey preparation 

requires state surveyors to review an OSCAR report of the nursing 

home, which contains deficiencies from the past four health surveys and 

complaint investigations. Surveyors can use the same OSCAR report to 

compare deficiencies from the most recent health survey and complaint 

investigations with those on ASPEN. 

To target this verification, CMS could direct State survey agencies to 

begin this activity with nursing homes that have the most deficiencies. 

As we found, the more deficiencies a nursing home had, the more likely 

there were inaccuracies in its inspection results on Nursing Home 

Compare. 

Establishing a single point of contact for reporting Nursing Home 

Compare inaccuracies. Given our finding that Nursing Home Compare 

presents deficiencies not supported by the State survey report for 15 

percent of nursing homes, these nursing homes may be incorrectly 

OEI - 01 - 03 - 00130 INSPECTION R ESULTS ON N URSING H OME C OMPARE : COMPLETENESS AND A CCURACY 12 
RECOMMENDATIONS 

portrayed as having deficiences not found to exist. Nursing Home 

Compare advises nursing homes that discover incorrect information to 

report it to their State survey agencies. However, CMS does not track 

these reports of inaccuracies. Hence, it is unable to assess the number 

of inaccuracies reported, or if the State survey agencies made the 

corrections. By having a single point of contact within CMS for 

reporting inaccuracies, CMS would be able to track and follow up on 

information that is reported to be inaccurate. Contact information for 

this individual (either a phone number or an electronic mail address) 

should be included on the Nursing Home Compare Web site. 

Agency Comments 

CMS agreed with our first recommendation to require State agencies to 

verify that the most recent inspection results are in Nursing Home 

Compare, citing it has performance standards under development. 

CMS disagreed with our second recommendation to establish a single 

point of contact for reporting Nursing Home Compare inaccuracies. The 

agency commented that “multiple CMS components are involved in 

providing Nursing Home Compare data on the Web site”, making a 

single point of contact impractical. CMS, however, will consider adding 

regional office contact information to Nursing Home Compare to 

facilitate corrections to the Web site. We continue to believe that a 

single point of contact for reporting inaccuracies would be the most 

effective solution. However, we appreciate CMS’s consideration of 

adding regional office contact information to Nursing Home Compare, 

and urge CMS to follow through on this idea. 

CMS also made a technical comment concerning its policy for handling 

duplicate deficiencies. CMS noted that its Web site contractor 

suppresses duplicate deficiencies if a deficiency is cited on both a 

complaint investigation and a health survey within 15 days of each 

other. CMS officials informed us of this policy during this inspection; 

thus, we did not count duplicate deficiencies within this time frame as 

inaccuracies. For the full text of CMS’s comments, see Appendix C. 

OEI - 01 - 03 - 00130 INSPECTION R ESULTS ON N URSING H OME C OMPARE : COMPLETENESS AND A CCURACY 13 
� APPENDIX ~ A 
 

Nursing Home Compare Definitions of Scope and Severity of Deficiencies 

Table 1: Nursing Home Compare Severity of Deficiency Scale 

Level of Harm Severity of Deficiency 

1 Potential for minimal harm - causing no more than a minor 

negative impact on the resident. 

2 

Minimal harm or potential for actual harm - results in 

minimal discomfort to the resident or has the potential (not yet 

realized) to negatively affect the resident's ability to achieve 

his/her highest functional status. 

3 

Actual harm - results in a negative outcome that has 

negatively affected the resident's ability to achieve his/her 

highest functional status. 

4 

Immediate jeopardy - places the resident in immediate 

jeopardy as it has caused (or is likely to cause) serious injury, 

harm, impairment, or death to a resident receiving care in the 

nursing home. Immediate corrective action is necessary when 

this deficiency is identified. 

Source: Nursing Home Compare Web site, Centers for Medicare & Medicaid Services, August 2003 

Table 2: Nursing Home Compare Scope of Deficiency Scale 

Residents affected 

Few 

(Isolated) 

Scope of Deficiency 

Deficiency affects one or the fewest number of residents, staff, 

or occurrences. 

Some (Pattern) Deficiency affects more than a limited number of residents, 

staff, or occurrences. 

Many (Widespread) 

Deficiency is found to be widespread throughout the facility 

and/or has the potential to affect a large portion or all the 

residents. 

Source: Nursing Home Compare Web site, Centers for Medicare & Medicaid Services, August 2003 

OEI - 01 - 03 - 00130 INSPECTION R ESULTS ON N URSING H OME C OMPARE : COMPLETENESS AND A CCURACY 14 
� APPENDIX ~ B 
 

METHODOLOGY 

For this report, we used inspection results data for nursing homes 

included in Nursing Home Compare as of April 1, 2003. Nursing Home 

Compare included 16,446 nursing homes on this date. 

Nursing Home Compare Completeness 

To determine the extent to which Nursing Home Compare includes all 

Medicare- and Medicaid-certified nursing homes, we matched the Nursing 

Home Compare database with the March 2003 Minimum Data Set (MDS) 

facility file, a demographic data file of nursing homes that submit 

resident assessments. We used the MDS facility file to match with 

Nursing Home Compare since CMS requires all Medicare- and Medicaidcertified nursing homes to report resident assessments. Using SAS® 

statistical software, we matched provider numbers, names, and addresses 

from both data sets to determine whether nursing homes in Nursing 

Home Compare were also located in the MDS facility file. We also 

reversed our match to determine whether nursing homes located in the 

MDS facility file were found on Nursing Home Compare. 

Since CMS composes the MDS facility file monthly, it was possible that 

some of these nursing homes were in stages of closure or termination from 

Medicare. Therefore, these nursing homes might not be included in the 

MDS facility file for other months. For this reason, we matched nursing 

homes not found in the March 2003 MDS facility file with those in the 

February, April, May, and June 2003 MDS facility files. We also checked 

Nursing Home Compare in September 2003 to determine whether the site 

still included the unmatched nursing homes. 

To further investigate whether nursing homes not found in both 

databases remained in operation, we used various methods. These 

included contacting one State survey agency, rechecking nursing homes in 

the May, September, and October Nursing Home Compare database, 

Internet searches, and phone calls to nursing homes. 

Nursing Home Compare Accuracy 

To determine whether Nursing Home Compare includes all intended 

inspection results, as well as to determine the accuracy of nursing home 

inspection results provided on Nursing Home Compare, we matched 

inspection results for a sample of nursing homes in Nursing Home 

Compare with survey documentation supplied by State survey agencies. 

We selected a simple random sample of 150 nursing homes from the 

16,446 included in Nursing Home Compare as of April 1, 2003. The 

OEI - 01 - 03 - 00130 INSPECTION R ESULTS ON N URSING H OME C OMPARE : COMPLETENESS AND A CCURACY 15 
APPENDIX~B 

sample included nursing homes from 39 States, with a range of 1 to 20 

nursing homes per State. Our sample produces an estimate of the 

proportion of cases in which there is an inaccuracy between State survey 

documentation and Nursing Home Compare. This estimate is within 8 

percentage points of the actual inaccuracy rate at the 95 percent 

confidence level. 

For each State represented in our sample, we sent a letter to the State's 

director of nursing home certification requesting survey documentation 

for each sampled nursing home. Before sending the letter, we solicited 

comments from CMS central office officials, CMS regional office 

surveyors, and a State surveyor on its content and clarity. We requested 

the following survey documentation for each sampled nursing home in the 

State. 

•	 statements of deficiencies (CMS-2567 forms) from certification 

(health) surveys, conducted between January 1, 1999 and March 31, 

2003, as well as documentation of the completed corresponding life 

safety code survey 

•	 statements of deficiencies (CMS-2567 forms) from complaint 

investigations, conducted between December 1, 1999 and February 

28, 2003 

•	 copies of letters from nursing homes requesting informal dispute 

resolution for any deficiencies cited from the health surveys and 

complaint investigations listed above, as well as documentation of 

the informal dispute resolution outcome 

All States in our sample sent requested documentation for a 100 percent 

response rate. However, two States could not locate some information 

needed for our analysis for a total of four nursing homes. This 

information included one health survey each for two nursing homes, three 

complaint investigations for one nursing home, and two health surveys for 

one nursing home. Two of the missing health surveys were from 1998, 

and one was from 1997. In the instances where States could not locate 

information, we did not score Nursing Home Compare as inaccurate. 

We allowed State survey agencies to either mail paper copies of 

documentation or electronically mail files with the documentation. Of the 

39 States represented in our sample, 22 mailed paper copies of all 

documentation, 10 mailed files electronically, and 7 sent us both paper 

copies and electronic files. When necessary, we contacted States by 

OEI - 01 - 03 - 00130 INSPECTION R ESULTS ON N URSING H OME C OMPARE : COMPLETENESS AND A CCURACY 16 
APPENDIX~B 

telephone or electronic mail and requested clarifications or additional 

information. 

To perform the comparison between the State survey documentation and 

Nursing Home Compare, we downloaded and printed the complete 

inspection information section from Nursing Home Compare in April 2003 

and matched Nursing Home Compare's information with the State 

documentation we received. We verified the following information: 

•	 inclusion of the three most recent health surveys and all 

substantiated complaint investigations occurring between 

December 1, 1999 and February 28, 2003 

• dates of health surveys and complaint investigations 

•	 number of deficiencies per health survey and complaint 

investigation 

•	 descriptions of deficiencies found during each survey or complaint 

investigation 

• description of severity for each deficiency 

• description of affected residents (scope) for each deficiency. 

We also reviewed informal dispute resolution documentation to determine 

whether relevant deficiencies presented on Nursing Home Compare 

reflected the outcome of informal dispute resolution. 

To account for the time between the end of a survey and the transmission 

of its data into Nursing Home Compare, we employed criteria from the 

Fiscal Year 2003 CMS State Performance Guidelines. These guidelines 

require State agencies to enter results in the Online Survey Certification 

and Reporting (OSCAR) system within 70 days of the last day of the 

health survey or complaint investigation. Hence, we do not count any 

health survey or complaint investigation occurring after January 10, 2003 

as missing from Nursing Home Compare. 

When information from State survey documentation was not accurately 

reflected on Nursing Home Compare, we recorded the inaccuracies on 

paper forms and used Microsoft Access and SAS® statistical software to 

analyze the data. 

OEI - 01 - 03 - 00130 INSPECTION R ESULTS ON N URSING H OME C OMPARE : COMPLETENESS AND A CCURACY 17 
APPENDIX~B 

The table below presents point estimates and confidence intervals at the 

95 percent confidence level for key figures presented in the findings 

section. 

Point 

Estimate 

95 percent 

confidence 

interval 

Percent of nursing homes with no 

inaccuracies 68% +/- 7.5% 

Percent of nursing homes missing one or 

more surveys on Nursing Home Compare 19% +/- 6.3% 

Percent of nursing homes with deficiencies 

incorrectly posted on Nursing Home 

Compare 

15% +/- 5.7% 

Percent of nursing homes with deficiencies 

missing from Nursing Home Compare 11% +/- 5.0% 

Percent of nursing homes with an incorrect 

scope and/or severity score 8% +/- 4.3% 

Percent of nursing homes with an incorrect 

deficiency date 4% +/- 3.1% 

Percent of nursing homes with an incorrect 

deficiency description 3% +/- 2.7% 

Interviews with CMS staff 

We interviewed CMS officials to learn about the process by which 

information from OSCAR is transmitted onto Nursing Home Compare, 

and the data systems involved. These interviews included staff from the 

Center for Beneficiary Choices, and Center for Medicaid and State 

Operations in CMS's central office, as well as CMS regional office staff. 

Document reviews 

We reviewed CMS documents that provide procedures or guidelines 

relevant to Nursing Home Compare. These documents include the State 

Operations Manual and the State Survey Agency Performance Standards 

for Fiscal Year 2003. 

OEI - 01 - 03 - 00130 INSPECTION R ESULTS ON N URSING H OME C OMPARE : COMPLETENESS AND A CCURACY 18 
� APPENDIX ~ C 
 

Agency Comments 

OEI - 01 - 03 - 00130 INSPECTION R ESULTS ON N URSING H OME C OMPARE : COMPLETENESS AND A CCURACY 19
 
APPENDIX~C 

OEI - 01 - 03 - 00130 INSPECTION R ESULTS ON N URSING H OME C OMPARE : COMPLETENESS AND A CCURACY 20
 
APPENDIX~C 

OEI - 01 - 03 - 00130 INSPECTION R ESULTS ON N URSING H OME C OMPARE : COMPLETENESS AND A CCURACY 21
 
� ACKNOWLEDGEMENTS 
 

This report was prepared under the direction of Mark R. Yessian, Ph.D., 

Regional Inspector General for Evaluation and Inspections in the Boston 

Regional Office, and Joyce M. Greenleaf, MBA, Assistant Regional 

Inspector General. Other principal Office of Evaluation and Inspections 

staff who contributed include: 

Maria E. Maddaloni, Project Leader 

China D. Eng, Program Analyst
 

Baltimore:
 

Sandy Khoury, Program Specialist
 

Scott Horning, Program Analyst
 

Barbara Tedesco, Mathematical Statistician
 

OEI - 01 - 03 - 00130 INSPECTION R ESULTS ON N URSING H OME C OMPARE : COMPLETENESS AND A CCURACY 22 
� END NOTES 
 

1 The Nursing Home Compare website can be accessed at 

http://www.medicare.gov/NHCompare/home.asp. 

2 The numbers presented show overall visits to Nursing Home 

Compare. CMS does not collect information that could be used to 

identify a visitor to its Web sites. Hence, CMS is not able to 

determine the number of unique visitors to Nursing Home Compare. 

3 Nursing Home Compare typically presents deficiencies from 

complaint investigations for about a 2-year period. For the data used 

in this inspection, Nursing Home Compare included deficiencies from 

complaint investigations between December 1, 1999 and February 28, 

2003. 

4 Fiscal Year 2003 State Performance Review Protocol Guidance, 

Centers for Medicare & Medicaid Services, May 2003, pp. 41-48. 

5 Fiscal Year 2003 State Performance Review Protocol Guidance, 

Centers for Medicare & Medicaid Services, May 2003, pp. 41-48. 

6 A T-test found a statistically significant difference between the 

mean number of deficiencies for nursing homes with inaccuracies and 

for nursing homes with no inaccuracies at the 95 percent confidence 

level. 

Average number of deficiencies 

Sample (n=150) 22 

Nursing homes with no inaccuracies on Nursing Home 

Compare (n=103) 16 

Nursing homes with inaccuracies on Nursing Home Compare 

(n=47) 35 

T= -4.94 Degrees of freedom=57.6 

7 Fiscal Year 2003 State Performance Review Protocol Guidance, 

Centers for Medicare & Medicaid Services, May 2003, pp. 44-45. 

OEI - 01 - 03 - 00130 INSPECTION R ESULTS ON N URSING H OME C OMPARE : COMPLETENESS AND A C C URACY 23