Department of Health and Human Services 
OFFICE OF 
INSPECTOR GENERAL 
N 
I R
NSPECTION  ESULTS ON 
N H C : 
URSING  OME  OMPARE
C A
OMPLETENESS AND  CCURACY 
Inspector General 
June 2004 
OEI-01-03-00130  Office of Inspector General
 
http://oig.hhs.gov 
The mission of the Office of Inspector General (OIG), as mandated by Public Law 95-452, 
as amended, is to protect the integrity of the Department of Health and Human Services 
(HHS) programs, as well as the health and welfare of beneficiaries served by those 
programs. This statutory mission is carried out through a nationwide network of audits, 
investigations, and inspections conducted by the following operating components: 
Office of Audit Services 
The OIG's Office of Audit Services (OAS) provides all auditing services for HHS, either by 
conducting audits with its own audit resources or by overseeing audit work done by others. 
Audits examine the performance of HHS programs and/or its grantees and contractors 
in carrying out their respective responsibilities and are intended to provide independent 
assessments of HHS programs and operations in order to reduce waste, abuse, and 
mismanagement and to promote economy and efficiency throughout the department. 
Office of Evaluation and Inspections 
The OIG's Office of Evaluation and Inspections (OEI) conducts short-term management 
and program evaluations (called inspections) that focus on issues of concern to the 
department, the Congress, and the public. The findings and recommendations contained 
in the inspections reports generate rapid, accurate, and up-to-date information on the 
efficiency, vulnerability, and effectiveness of departmental programs.  The OEI also 
oversees state Medicaid fraud control units, which investigate and prosecute fraud and 
patient abuse in the Medicaid program. 
Office of Investigations 
The OIG's Office of Investigations (OI) conducts criminal, civil, and administrative 
investigations of allegations of wrongdoing in HHS programs or to HHS beneficiaries 
and of unjust enrichment by providers. The investigative efforts of OI lead to criminal 
convictions, administrative sanctions, or civil monetary penalties. 
Office of Counsel to the Inspector General 
The Office of Counsel to the Inspector General (OCIG) provides general legal services to 
OIG, rendering advice and opinions on HHS programs and operations and providing all 
legal support in OIG's internal operations. The OCIG imposes program exclusions and 
civil monetary penalties on health care providers and litigates those actions within the 
department. The OCIG also represents OIG in the global settlement of cases arising 
under the Civil False Claims Act, develops and monitors corporate integrity agreements, 
develops model compliance plans, renders advisory opinions on OIG sanctions to the 
health care community, and issues fraud alerts and other industry guidance.      A B S T R A C T 
  
Nursing Home Compare is a Web site maintained by the Centers for 
Medicare & Medicaid Services (CMS) that provides consumers with 
information on nursing home quality. Consumers use the information 
on the site to help them choose the nursing home that best fits their 
needs. Therefore, it is essential that the site accurately portray all 
Medicare- and Medicaid-certified nursing homes. Using CMS databases 
and State survey documentation, we determined whether the site 
contained all Medicare- and Medicaid-certified nursing homes, and for a 
random sample of nursing homes, determined the accuracy of the 
inspection results provided on the site. We found that Nursing Home 
Compare contains nearly all Medicare- and Medicaid-certified nursing 
homes. However, one or more surveys were missing from 19 percent of 
nursing homes, leaving consumers with incomplete information about 
those homes' survey and complaint histories. Inspection results on 
Nursing Home Compare are largely accurate, but one or more 
deficiencies were missing from 11 percent of nursing homes inspection 
results, and Nursing Home Compare presents deficiencies not found on 
State survey documentation for 15 percent of nursing homes. 
Inaccuracies may be due to late data entry by State survey agencies, no 
tracking of inaccuracy reports by CMS, and failure of State survey 
agencies to transmit data on amended deficiencies. We recommend that 
CMS require State agencies to verify that the most recent inspection 
results are in CMS databases and establish a single point of contact for 
reporting discrepancies on the Web site. 
OEI-01-03-00130  INSPECTION RESULTS ON NURSING HOME COMPARE: COMPLETENESS AND ACCURACY  i    E X E C U T I V E   S U M M A R Y  
 
OBJECTIVES 
To determine if Nursing Home Compare includes all Medicare- and 
Medicaid-certified nursing homes. 
To determine whether Nursing Home Compare includes all intended 
inspection results. 
To determine the accuracy of nursing home inspection results provided 
on the Nursing Home Compare Web site. 
BACKGROUND 
In 1998, the Department of Health and Human Services (HHS) 
launched the Nursing Home Compare Web site. The site is maintained 
by the Centers for Medicare & Medicaid Services (CMS). Nursing Home 
Compare provides information about the past performance of all 
Medicare- and Medicaid-certified nursing homes in the country. The 
site also serves as a source of public information on nursing home 
quality. Consumers can use the information provided on the site to help 
them choose the nursing home that best fits their needs. Therefore, it is 
important that the site accurately portray all Medicare- and Medicaid-
certified nursing homes. 
This report focuses on the inspection results section of Nursing Home 
Compare. This section provides results from a nursing home's three 
most recent Medicare certification surveys (referred to as health surveys 
on Nursing Home Compare and in this report) and any complaint 
investigations. The information in the inspection results section 
includes type of deficiency, survey date, date the deficiency was 
corrected, and scope and severity of the deficiency. 
We base this inspection on the Nursing Home Compare database as of 
April 1, 2003. To determine completeness, we matched the nursing 
homes on the site with those on a CMS database that contains 
information on Medicare- and Medicaid-certified nursing homes. To 
determine accuracy, we matched Nursing Home Compare inspection 
results from a simple random sample of 150 nursing homes with State 
survey documentation. The State survey documentation includes 
statements of deficiency from health surveys and complaint 
investigations, as well as informal dispute resolution outcomes. 
OEI-01-03-00130  INSPECTION RESULTS ON NURSING HOME COMPARE: COMPLETENESS AND ACCURACY  ii  E X E C U T I V E   S U M M A R Y  
FINDINGS 
Nursing Home Compare contains almost all Medicare-and Medicaid-
certified nursing homes. Less than 1 percent of nursing homes did not 
match to either Nursing Home Compare or the Minimum Data Set 
(MDS) facility file, a demographic database of nursing homes 
maintained by CMS. Nursing Home Compare excluded 55 nursing 
homes (0.2 percent) included in the MDS facility file. Nursing Home 
Compare included 119 nursing homes (0.7 percent) that were not found 
in the MDS facility file over a 5-month period. 
Nineteen percent of nursing homes had one or more surveys 
missing on Nursing Home Compare. Nursing Home Compare lacked 
the most current health survey for 7 percent of nursing homes, meaning 
that it instead provided older information. It also lacked either a 
complaint investigation or the second or third most recent health survey 
for 12 percent of nnursing homes. Furthermore, when Nursing Home 
Compare does not include deficiencies from complaint investigations, 
consumers have no way of determining that. 
Inspection results on Nursing Home Compare are largely accurate, 
but some areas of concern exist. Sixty-eight percent of nursing 
homes had Nursing Home Compare inspection results that exactly 
matched State survey documentation. Eleven percent of nursing homes 
had one or more deficiencies missing from their inspection results on 
Nursing Home Compare. Furthermore, Nursing Home Compare 
presents deficiencies not found on State survey documentation for 15 
percent of nursing homes. These inaccuracies may be due to late data 
entry by State survey agencies, no tracking of inaccuracy reports by 
CMS, and/or failure of State survey agencies to transmit data on 
amended deficiencies. 
RECOMMENDATIONS 
Overall, Nursing Home Compare's inspection results provide consumers 
with a valuable resource on the past performance of Medicare- and 
Medicaid- certified nursing homes. However, we found some 
inaccuracies between Nursing Home Compare and State survey reports 
that deserve attention, and suggest that CMS could do more to ensure 
that consumers receive the most accurate data possible when choosing a 
nursing home. 
OEI-01-03-00130  INSPECTION RESULTS ON NURSING HOME COMPARE: COMPLETENESS AND ACCURACY  iii  E X E C U T I V E   S U M M A R Y  
CMS should work to enhance Nursing Home Compare by: 
Requiring State agencies to verify that the most recent inspection 
results are in CMS databases. Timely and accurate inspection results 
on Nursing Home Compare depend on State surveyors transmitting 
data into the Online Survey Certification and Reporting system 
(OSCAR), the CMS database containing survey information. To ensure 
that data have been promptly transmitted into OSCAR, State agencies 
should verify the most recent inspection results in OSCAR with those in 
the Automated Survey Processing Environment (ASPEN) system, the 
application used for survey data entry and transmission to OSCAR. 
Establishing a single point of contact for reporting Nursing Home 
Compare inaccuracies. By having a single point of contact within 
CMS for reporting inaccuracies, CMS would be able to track and 
conduct follow-up on information that is reported to be inaccurate. 
Agency Comments 
CMS agreed with our first recommendation to require State agencies to 
verify that the most recent inspection results are in Nursing Home 
Compare, citing it has performance standards under development. 
CMS disagreed with our second recommendation to establish a single 
point of contact for reporting Nursing Home Compare inaccuracies. The 
agency commented that multiple CMS components are involved in 
providing Nursing Home Compare data on the Web site, making a 
single point of contact impractical. CMS, however, will consider adding 
regional office contact information to Nursing Home Compare to 
facilitate corrections to the Web site. We continue to believe that a 
single point of contact for reporting inaccuracies would be the most 
effective solution. However, we appreciate CMSs consideration of 
adding regional office contact information to Nursing Home Compare, 
and urge CMS to follow through on this idea. 
OEI-01-03-00130  INSPECTION RESULTS ON NURSING HOME COMPARE: COMPLETENESS AND ACCURACY  iv    T A B L E   O F   C O N T E N T S 
  
A B S T R A C T  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . .   i
 
E X E C U T I V E   S U M M A R Y  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . .  ii
 
I N T R O D U C T I O N . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . .  1
 
F I N D I N G S  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . .  6
 
Nursing Home Compare contains almost all nursing homes ............ 6
 
Nursing Home Compare does not contain all surveys....................... 7
 
Inspection results are largely accurate, but areas of concern exist... 8
 
R E C O M M E N D A T I O N S. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . .   12
 
A P P E N D I C E S. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . .   14
 
A: Definitions of scope and severity of deficiencies ........................ 14
 
B: Methodology................................................................................ 15
 
C: Agency Comments ...................................................................... 19
 
A C K N O W L E D G M E N T S  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . .  22
 
EN D   N O T E S  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . .  23
    I N T R O D U C T I O N  
 
OBJECTIVES 
To determine if Nursing Home Compare includes all Medicare- and 
Medicaid-certified nursing homes. 
To determine whether Nursing Home Compare includes all intended 
inspection results. 
To determine the accuracy of nursing home inspection results provided 
on the Nursing Home Compare Web site. 
BACKGROUND 
Nursing Home Compare 
In 1998, the Department of Health and Human Services (HHS) 
launched the Nursing Home Compare Web site. The purposes of the 
site are "to provide detailed information about the past performance of 
every Medicare- and Medicaid-certified nursing home in the country" 
and to serve as a source of public information on nursing home quality.1 
The site allows Medicare beneficiaries and their caregivers access to 
individual nursing home quality information to help them choose the 
nursing home that best fits their needs. Consumers access Nursing 
Home Compare through the main consumer Medicare Web site, 
www.medicare.gov. 
The Centers for Medicare & Medicaid Services (CMS) operates Nursing 
Home Compare. CMS intends the information on the site to be used 
with other sources of information, such as a visit to a nursing home. 
Currently, the site provides information on about 17,000 nursing homes. 
Between March 2002 and July 2003, Nursing Home Compare received 
an average of 134,000 visits per month. Nursing Home Compare 
presents four categories of information: 
	  inspection results, including deficiencies from Medicare 
certification surveys and complaint investigations 
	  facility characteristics, such as number of beds and type of 
ownership 
  nursing home staffing levels 
	  quality measures, which provide information on clinical and 
physical characteristics of a nursing home's residents 
OEI-01-03-00130  INSPECTION RESULTS ON NURSING HOME COMPARE: COMPLETENESS AND ACCURACY  1  I N T R O D U C T I O N  
Nursing Home Compare Inspection Results 
The inspection results section on Nursing Home Compare lists 
deficiencies from the three most recent Medicare certification surveys 
(referred to as health surveys on Nursing Home Compare and in this 
report), as well as those from substantiated complaint investigations 
conducted in the past 3 years for each nursing home. State survey 
agencies conduct nursing home surveys on behalf of CMS. Health 
surveys, which evaluate the quality of care and the services provided by 
the nursing home, occur on an average of every 12 months but no less 
frequently than every 15 months. State surveyors also conduct 
complaint investigations, as necessary, outside of the cycle of health 
surveys and cite deficiencies, as appropriate. 
Nursing Home Compare clusters deficiencies from each of the three 
health surveys with deficiencies from complaint investigations for an 
approximate 1-year period. The site describes each deficiency, provides 
the date that the deficiency was cited and corrected, and lists the scope 
and severity of the deficiency (see Figure 1). Severity refers to the level 
of harm, and scope refers to the number of affected residents. Nursing 
Home Compare also provides definitions for scope and severity (see 
Appendix A). 
FIGURE 1 
Deficiency Information 
Presented on Nursing 
Home Compare 
Deficiency and  Date deficiency  Severity of  Scope of 
date cited  was rectified  deficiency  deficiency 
Source: Nursing Home Compare Web site, Centers for Medicare & Medicaid Services, August 2003 
Nursing Home Compare Inspection Results and OSCAR 
Inspection results on Nursing Home Compare come from CMS's Online 
Survey Certification and Reporting (OSCAR) system, a database 
containing survey information for all Medicare- and Medicaid-certified 
nursing homes. After completion of a health survey or complaint 
investigation, State surveyors enter inspection results into the 
OEI-01-03-00130  INSPECTION RESULTS ON NURSING HOME COMPARE: COMPLETENESS AND ACCURACY  2  I N T R O D U C T I O N  
Automated Survey Processing Environment (ASPEN) system. 
Surveyors use ASPEN to electronically transmit inspection results into 
OSCAR. According to CMS's State Survey Agency Performance 
Standards for Fiscal Year 2003, the average time for transmitting data 
from health surveys and complaint investigations into OSCAR must not 
exceed 70 calendar days from the last day of the survey. 
Nursing Home Compare does not include information for deficiencies 
that are under dispute and notes this in a disclaimer on the site. 
Nursing homes can appeal deficiencies from health surveys and 
complaint investigations through the State survey agency with informal 
dispute resolution. Any deficiency undergoing this process is not 
transmitted into OSCAR. When informal dispute resolution concludes, 
State survey agencies transmit any deficiencies that remain into 
OSCAR. These deficiencies will then appear on Nursing Home 
Compare. 
CMS updates the inspection results on Nursing Home Compare 
monthly. At the beginning of each month, data analysts in CMS's 
Center for Medicaid and State Operations extract data from OSCAR. 
CMS's Center for Beneficiary Choices, which is responsible for the 
appearance and content of Nursing Home Compare, then provides the 
data to its contractor who maintains the Nursing Home Compare Web 
site. The contractor incorporates the new data into Nursing Home 
Compare by the end of each month. 
Prominence of Nursing Home Compare 
According to CMS officials, about 20 percent of visitors to the main 
Medicare Web site access Nursing Home Compare, making Nursing 
Home Compare one of the most frequently visited sections of the 
Medicare Web site.2  Nursing Home Compare's popularity likely can be 
attributed to nationwide promotion of the Web site. In November 2002, 
CMS placed advertisements in 71 newspapers across all 50 States. 
CMS also promoted Nursing Home Compare in national television 
advertisements. The media campaign aimed to increase public 
awareness of the site as a principal source of nursing home quality 
information. Prior to the media campaign and launch of the quality 
measures, Nursing Home Compare received fewer than 100,000 visits 
per month. In November 2002, Nursing Home Compare received about 
400,000 visits. 
CMS has used Nursing Home Compare as a model for reporting quality 
information to consumers on other types of Medicare-certified facilities 
OEI-01-03-00130  INSPECTION RESULTS ON NURSING HOME COMPARE: COMPLETENESS AND ACCURACY  3  I N T R O D U C T I O N  
and services. Currently, consumers can access Home Health Compare 
and Dialysis Facility Compare from the Medicare Web site. An 
initiative to report quality information for hospitals is in progress. 
Concerns about Nursing Home Compare 
In February 2002, the House of Representatives' Committee on 
Government Reform released the report, HHS 'Nursing Home Compare' 
Website Has Major Flaws.  The report found that Nursing Home 
Compare excluded deficiencies from complaint investigations, thus not 
presenting a complete picture of the nursing home. In May 2002, CMS 
added deficiencies from complaint investigations to Nursing Home 
Compare's inspection results section. 
In October 2002, the General Accounting Office released the report, 
Nursing Homes: Public Reporting of Quality Indicators Has Merit, but 
National Implementation is Premature. The report raised concerns 
about the validity and usefulness of the quality measures posted on 
Nursing Home Compare and concluded that CMS was prematurely 
reporting the measures. Furthermore, the report stated that CMS was 
not adequately prepared to address questions raised by the public 
concerning the site. In its comments on the report, CMS affirmed its 
decision to release the quality measures as a means of providing 
consumer information and public accountability. 
This Inspection 
Considering the efforts to increase public awareness of Nursing Home 
Compare as a tool for consumers, it is essential that the site accurately 
portray all Medicare- and Medicaid-certified nursing homes. This 
inspection focuses on the completeness of the Nursing Home Compare 
Web site, as well as the accuracy of the inspection results section of the 
site. Determining the completeness of Nursing Home Compare shows 
whether the site contains all Medicare- and Medicaid-certified nursing 
homes. Assessing the accuracy of inspection results shows whether 
Nursing Home Compare contains the inspection results it claims to 
present, as well as whether it presents consumers with correct and 
reliable information. 
This inspection does not assess the accuracy of the other three 
categories of information on Nursing Home Compare: facility 
characteristics, staffing levels, and quality measures. 
OEI-01-03-00130  INSPECTION RESULTS ON NURSING HOME COMPARE: COMPLETENESS AND ACCURACY  4  I N T R O D U C T I O N  
METHODOLOGY 
This inspection is based on the Nursing Home Compare database as of 
April 1, 2003. On this date, Nursing Home Compare included 16,446 
Medicare- and Medicaid-certified nursing homes. 
To determine the completeness of Nursing Home Compare, we matched 
provider numbers and addresses of all nursing homes included in 
Nursing Home Compare with nursing homes included in the Minimum 
Data Set (MDS) facility file, a demographic data file of nursing homes 
that submit resident assessments. 
To determine whether Nursing Home Compare includes all intended 
inspection results, as well as the accuracy of the inspection results 
provided by Nursing Home Compare, we selected a national, simple 
random sample of 150 nursing homes from Nursing Home Compare. 
We then requested survey documentation from State survey agencies. 
Survey documentation consisted of statements of deficiency for health 
surveys and complaint investigations as well as outcomes from informal 
dispute resolution, if relevant, for the sampled nursing homes. Our 
response rate from the State survey agencies was 100 percent. We 
matched survey documentation to the information presented on Nursing 
Home Compare for each nursing home. With this information, we 
assessed whether any health or complaint surveys were missing from 
Nursing Home Compare. The information we matched for accuracy 
included dates of surveys, number of deficiencies, and the scope and 
severity of deficiencies. 
We interviewed staff in CMS central and regional offices who work with 
Nursing Home Compare and survey and certification of nursing homes. 
We also reviewed documentation relevant to Nursing Home Compare's 
inspection results, including CMS's State Operations Manual and State 
Survey Agency Performance Standards for Fiscal Year 2003. Appendix 
B contains a detailed description of our methodology. 
We conducted this inspection in accordance with the Quality Standards 
for Inspections issued by the President's Council on Integrity and 
Efficiency. 
OEI-01-03-00130  INSPECTION RESULTS ON NURSING HOME COMPARE: COMPLETENESS AND ACCURACY  5  F I  NF DI IN ND GI SN G S  
Nursing Home Compare contains almost all  In order to fully provide consumers 
Medicare- and Medicaid- certified nursing homes.	 with all available options for 
nursing homes, Nursing Home 
Compare must contain all Medicare- and Medicaid-certified nursing 
homes. Our match of the 16,446 nursing homes in Nursing Home 
Compare and the MDS facility file showed that Nursing Home Compare 
included almost all nursing homes (see Figure 2). 
FIGURE 2  Nursing Homes in 
Match of Nursing Home  MDS Facility File 
Compare database with  NOT Nursing Home Compare 
the MDS facility file  0.2 percent (n=55) 
Nursing Homes in 
Nursing Home Compare 
and MDS Facility File 
Nursing Homes in 
Nursing Home Compare 
NOT MDS Facility File 
0.7 percent (n=119) 
Source: OIG analysis of nursing homes in Nursing Home Compare and the MDS facility file, 2003 
In all, less than 1 percent of nursing homes did not appear in both 
Nursing Home Compare and the MDS facility file. Nursing Home 
Compare included 119 nursing homes (0.7 percent) that were not found 
in the MDS facility file over a 5-month period. Nursing Home Compare 
did not include 55 nursing homes (0.2 percent) included in the MDS 
facility file. These 55 nursing homes submitted MDS data, yet no 
evidence of their operation exists in Nursing Home Compare. 
A closer look at the 119 nursing homes from Nursing Home Compare 
not found in the MDS facility file showed that Nursing Home Compare 
may erroneously include many of these nursing homes. For example, 14 
of the 119 nursing homes had not had a health survey in 2 years, 
though CMS requires nursing homes to be surveyed every 15 months 
OEI-01-03-00130  INSPECTION RESULTS ON NURSING HOME COMPARE: COMPLETENESS AND ACCURACY  6  F I N D I N G S  
(107 of the 119 were surveyed within the 2-year period). To determine 
whether some of the 119 nursing homes might be closed, we contacted a 
survey agency in 1 State to confirm the operating status of its 9 nursing 
homes that were in Nursing Home Compare but not the MDS facility 
file. The State survey agency informed us that seven of the nine 
nursing homes were closed. The October 2003 update of Nursing Home 
Compare still included all seven of the closed nursing homes. In 
addition to the potentially closed nursing homes, six nursing homes had 
obtained new provider numbers and were listed twice in Nursing Home 
Compare with both the new and old provider numbers. 
Nineteen percent of nursing homes had one or more  Nursing Home Compare informs 
surveys missing on Nursing Home Compare.	  consumers that the inspection 
results section includes 
deficiencies from the three most recent health surveys as well as from 
recent substantiated complaint investigations. 3  State survey agencies 
have 70 days after the last day of a survey to transmit survey results 
into OSCAR. However, Nursing Home Compare lacked one or more of 
the three most recent health surveys and/or complaint investigations 
that should have been there for 19 percent of nursing homes.4  These 
surveys and investigations included 206 deficiencies unaccounted for on 
Nursing Home Compare. Nursing home Compare lacked the most 
recent health survey for 7 percent of nursing homes, meaning that it 
instead provided older information. 
If Nursing Home Compare does not reflect the most recent inspection 
results, consumers may be misled by the information presented. For 
example, the 2 most recent surveys occurring in 2002 and 2000 for 1 
nursing home yielded 40 and 14 deficiencies, respectively. However, 
Nursing Home Compare's inspection results show three older surveys 
for this nursing home. The three surveys on Nursing Home Compare 
occurred in 1999, 1998, and 1997, and yielded 18, 9, and 2 deficiencies, 
respectively. 
Furthermore, Nursing Home Compare lacked complaint investigations 
for 8 percent of nursing homes. A complaint investigation on Nursing 
Home Compare provides an additional level of detail about the 
performance of a nursing home, but when it is missing, the nursing 
home erroneously appears more favorable to the consumer. 
Late data entry by State survey agencies may explain most of the 
missing health surveys. As the majority of the missing health surveys 
OEI-01-03-00130  INSPECTION RESULTS ON NURSING HOME COMPARE: COMPLETENESS AND ACCURACY  7  F I N D I N G S  
and complaint investigations could be found in Nursing Home Compare 
by the latter part of 2003, State survey agencies apparently exceeded 
the 70-day time period required to enter results into OSCAR. However, 
only the recent missing health surveys and complaint investigations can 
be attributed to late data entry, as older health surveys and complaint 
investigations were still missing. CMS evaluates the accuracy and 
timeliness of data entry by State survey agencies semiannually.5 
However, this evaluation is a retrospective review, so missing surveys 
could go undetected for as long as 6 months. 
Inspection results on Nursing Home Compare are  If State survey documentation 
exactly matched the inspection 
largely accurate, but some areas of concern exist.	 
results on Nursing Home 
Compare for our sample of 150 nursing homes, we considered that 
information on the site to be accurate. Five different types of 
inaccuracies existed between Nursing Home Compare inspection results 
and State survey documentation. Table 1 shows the number and 
percentage of each type of inaccuracy. As some nursing homes had 
more than one inaccuracy between Nursing Home Compare and State 
survey documentation, percentages listed do not total 100 percent. 
Below, we discuss Nursing Home Compares accurate inspection results, 
as well as the the two most significant types of inaccuracies. 
TABLE 1  Type of Inaccuracy  Number of  Percent of 
Number and Percentage  Nursing  Nursing Homes 
Homes with  with 
of Nursing Homes with 
Inaccuracies  Inaccuracies 
Inaccuracies between 
Nursing Home Compare 
Deficiencies incorrectly posted on Nursing Home  23  15% 
and State Survey  Compare 
Documentation (n=150) 
Deficiencies missing from Nursing Home Compare  16  11% 
Incorrect scope and/or severity score  12  8% 
Incorrect deficiency date  6  4% 
Incorrect deficiency description  4  3% 
Source: OIG analysis of Nursing Home Compare inspection results and State survey documentation, 2003 
OEI-01-03-00130  INSPECTION RESULTS ON NURSING HOME COMPARE: COMPLETENESS AND ACCURACY  8  F I N D I N G S  
Sixty-eight percent of nursing homes had Nursing Home Compare 
inspection results that exactly matched State survey documentation. 
Accurate reporting of this information allows the consumer to 
confidently gather detailed information about the type of deficiency, the 
level of severity, and scope of the deficiency. It gives a picture of the 
timing of surveys and substantiated complaint investigations, and the 
deficiencies from each. For 68 percent of nursing homes, Nursing Home 
Compare accurately reported inspection results, based on a match to 
State survey documentation (see Figure 3). 
FIGURE 3
 
Percent of nursing 
 
9% 
homes by number of 
 
1% 
inaccuracies
 
7% 
15% 
68%  None 
1 
2 
3 
4 or more 
Source: OIG analysis of Nursing Home Compare inspection results and State survey documentation, 2003 
Nursing homes with a low number of deficiencies tended to be 
accurately presented on Nursing Home Compare. Nursing homes with 
no inaccuracies between State survey documentation and Nursing 
Home Compare had significantly fewer deficiencies than nursing homes 
with inaccuracies. Across all 3 survey periods on Nursing Home 
Compare, nursing homes without inaccuracies had an average of 17 
deficiencies. Nursing homes with inaccuracies averaged 35 
deficiencies. 6 
OEI-01-03-00130  INSPECTION RESULTS ON NURSING HOME COMPARE: COMPLETENESS AND ACCURACY  9  F I N D I N G S  
Eleven percent of nursing homes had one or more deficiencies missing 
from their inspection results on Nursing Home Compare. 
These represent instances where Nursing Home Compare excluded one 
or more deficiencies from a nursing home's health survey or complaint 
investigation, based on State survey documentation. On average, 
Nursing Home Compare lacked two deficiencies for each of these 
nursing homes. Thus, 11 percent of nursing homes appeared to 
consumers to have fewer deficiencies than actually existed. 
Of the deficiencies missing on Nursing Home Compare, 43 percent had 
been unsuccessfully appealed through informal dispute resolution, thus 
were not eliminated by the process. While undergoing informal dispute 
resolution, a deficiency is withheld from OSCAR and not provided on 
Nursing Home Compare. If deficiencies remain after informal dispute 
resolution, it is the responsibility of the State survey agency to promptly 
update OSCAR. If the deficiency is not entered into OSCAR, the 
deficiency cannot be presented on Nursing Home Compare. 
For 1 nursing home, Nursing Home Compare shows 13 deficiencies for a 
nursing home's most recent health survey. However, the nursing 
home's survey results do not show an additional 11 deficiencies that 
were unsuccessfully appealed through informal dispute resolution. Two 
of the missing 11 deficiencies indicate citations for actual harm, which 
likely would be a caution to a consumer. CMS ranks deficiencies cited 
as actual harm the second-highest level of deficiency severity. 
CMS is unlikely to become aware of these inaccuracies unless nursing 
homes report them, which is also unlikely, since these inaccuracies 
portray nursing homes as better than they are. Even if CMS does 
become aware of these inaccuracies, it does not track reports of 
inaccuracies to determine whether they are recitified by State survey 
agencies. CMS may also find these inaccuracies through its semiannual 
evaluation of data accuracy and timeliness. However, CMS's review 
looks at only a random sample of surveys conducted during the fiscal 
year and permits an overall accuracy rate of 85 percent.7 
Nursing Home Compare presents deficiencies not found on State survey 
documentation for 15 percent of nursing homes. 
Deficiencies appeared on Nursing Home Compare that State survey 
documentation did not substantiate for 23 nursing homes we reviewed. 
These false positive deficiencies make these nursing homes appear to 
consumers to have more deficiencies than actually exist. In all, 63 false 
positive deficiencies exist, with about half coming from just 4 nursing 
OEI-01-03-00130  INSPECTION RESULTS ON NURSING HOME COMPARE: COMPLETENESS AND ACCURACY  10  F I N D I N G S  
homes. For 1 of the 4 nursing homes, 10 deficiencies from a health 
survey are duplicated on Nursing Home Compare for a date 2 months 
after the original survey. Survey documentation exists for these 
deficiencies for just one of the dates. Thus, Nursing Home Compare's 
inspection results for this nursing home appear worse than the nursing 
home actually is. 
Of the 23 nursing homes with false positives, 8 had 1 deficiency 
appearing on Nursing Home Compare that had been successfully 
appealed through informal dispute resolution. For these eight nursing 
homes, State survey agencies had incorrectly transmitted deficiencies 
under dispute into OSCAR, and then did not remove the deficiencies 
when resolved by informal dispute resolution. 
If nursing homes detect inaccuracies within their inspection results, 
Nursing Home Compare directs them to contact their respective State 
survey agency to correct the information in OSCAR. CMS does not 
track the number of such reports, the types of inaccuracies reported, or 
whether the State survey agency rectified the inaccuracies. 
OEI-01-03-00130  INSPECTION RESULTS ON NURSING HOME COMPARE: COMPLETENESS AND ACCURACY  11  0R E CR OE MC MO EM NMDEANT DI AO TN IS O N S  
Overall, Nursing Home Compare's inspection results provide consumers 
with a valuable resource on the past performance of Medicare- and 
Medicaid- certified nursing homes.  However, we found some 
inaccuracies between Nursing Home Compare and State survey reports 
that deserve attention, and suggest that CMS could do more to ensure 
that consumers receive the most accurate data possible when choosing a 
nursing home. 
CMS should enhance Nursing Home Compare by: 
Requiring State agencies to verify that the most recent inspection results 
are in CMS databases.  Timely and accurate inspection results on 
Nursing Home Compare depend on the transmission of survey data by 
State surveyors into OSCAR, the CMS database containing survey 
information. However, Nursing Home Compare did not include some 
cited deficiencies as well as entire surveys. In some cases, Nursing 
Home Compare did not reflect changes in deficiencies amended through 
informal dispute resolution. If State survey agencies do not transmit 
data into OSCAR, then changes will not be reflected on Nursing Home 
Compare. 
To assure that data have been promptly transmitted into OSCAR, State 
agencies should verify the most recent inspection results in OSCAR 
with those in ASPEN, the system used for survey data entry and 
transmission to OSCAR. This verification can be included as an offsite 
survey preparation task. Currently, CMS's offsite survey preparation 
requires state surveyors to review an OSCAR report of the nursing 
home, which contains deficiencies from the past four health surveys and 
complaint investigations. Surveyors can use the same OSCAR report to 
compare deficiencies from the most recent health survey and complaint 
investigations with those on ASPEN. 
To target this verification, CMS could direct State survey agencies to 
begin this activity with nursing homes that have the most deficiencies. 
As we found, the more deficiencies a nursing home had, the more likely 
there were inaccuracies in its inspection results on Nursing Home 
Compare. 
Establishing a single point of contact for reporting Nursing Home 
Compare inaccuracies.  Given our finding that Nursing Home Compare 
presents deficiencies not supported by the State survey report for 15 
percent of nursing homes, these nursing homes may be incorrectly 
OEI-01-03-00130  INSPECTION RESULTS ON NURSING HOME COMPARE: COMPLETENESS AND ACCURACY  12  R E C O M M E N D A T I O N S  
portrayed as having deficiences not found to exist. Nursing Home 
Compare advises nursing homes that discover incorrect information to 
report it to their State survey agencies. However, CMS does not track 
these reports of inaccuracies. Hence, it is unable to assess the number 
of inaccuracies reported, or if the State survey agencies made the 
corrections. By having a single point of contact within CMS for 
reporting inaccuracies, CMS would be able to track and follow up on 
information that is reported to be inaccurate. Contact information for 
this individual (either a phone number or an electronic mail address) 
should be included on the Nursing Home Compare Web site. 
Agency Comments 
CMS agreed with our first recommendation to require State agencies to 
verify that the most recent inspection results are in Nursing Home 
Compare, citing it has performance standards under development. 
CMS disagreed with our second recommendation to establish a single 
point of contact for reporting Nursing Home Compare inaccuracies. The 
agency commented that multiple CMS components are involved in 
providing Nursing Home Compare data on the Web site, making a 
single point of contact impractical. CMS, however, will consider adding 
regional office contact information to Nursing Home Compare to 
facilitate corrections to the Web site. We continue to believe that a 
single point of contact for reporting inaccuracies would be the most 
effective solution. However, we appreciate CMSs consideration of 
adding regional office contact information to Nursing Home Compare, 
and urge CMS to follow through on this idea. 
CMS also made a technical comment concerning its policy for handling 
duplicate deficiencies. CMS noted that its Web site contractor 
suppresses duplicate deficiencies if a deficiency is cited on both a 
complaint investigation and a health survey within 15 days of each 
other. CMS officials informed us of this policy during this inspection; 
thus, we did not count duplicate deficiencies within this time frame as 
inaccuracies. For the full text of CMSs comments, see Appendix C. 
OEI-01-03-00130  INSPECTION RESULTS ON NURSING HOME COMPARE: COMPLETENESS AND ACCURACY  13    A P P E N D I X   ~   A  
 
Nursing Home Compare Definitions of Scope and Severity of Deficiencies 
Table 1: Nursing Home Compare Severity of Deficiency Scale 
Level of Harm  Severity of Deficiency 
Potential for minimal harm - causing no more than a minor 
1 
negative impact on the resident. 
Minimal harm or potential for actual harm -results in 
minimal discomfort to the resident or has the potential (not yet 
2 
realized) to negatively affect the resident's ability to achieve 
his/her highest functional status. 
Actual harm -results in a negative outcome that has 
3  negatively affected the resident's ability to achieve his/her 
highest functional status. 
Immediate jeopardy -places the resident in immediate 
jeopardy as it has caused (or is likely to cause) serious injury, 
4  harm, impairment, or death to a resident receiving care in the 
nursing home. Immediate corrective action is necessary when 
this deficiency is identified. 
Source: Nursing Home Compare Web site, Centers for Medicare & Medicaid Services, August 2003 
Table 2: Nursing Home Compare Scope of Deficiency Scale 
Residents affected  Scope of Deficiency 
Few 
Deficiency affects one or the fewest number of residents, staff, 
or occurrences. 
(Isolated) 
Deficiency affects more than a limited number of residents, 
Some (Pattern) 
staff, or occurrences. 
Deficiency is found to be widespread throughout the facility 
Many (Widespread)  and/or has the potential to affect a large portion or all the 
residents. 
Source: Nursing Home Compare Web site, Centers for Medicare & Medicaid Services, August 2003 
OEI-01-03-00130  INSPECTION RESULTS ON NURSING HOME COMPARE: COMPLETENESS AND ACCURACY  14    A P P E N D I X   ~   B  
 
METHODOLOGY 
For this report, we used inspection results data for nursing homes 
included in Nursing Home Compare as of April 1, 2003. Nursing Home 
Compare included 16,446 nursing homes on this date. 
Nursing Home Compare Completeness 
To determine the extent to which Nursing Home Compare includes all 
Medicare- and Medicaid-certified nursing homes, we matched the Nursing 
Home Compare database with the March 2003 Minimum Data Set (MDS) 
facility file, a demographic data file of nursing homes that submit 
resident assessments. We used the MDS facility file to match with 
Nursing Home Compare since CMS requires all Medicare- and Medicaid-
certified nursing homes to report resident assessments. Using SAS 
statistical software, we matched provider numbers, names, and addresses 
from both data sets to determine whether nursing homes in Nursing 
Home Compare were also located in the MDS facility file. We also 
reversed our match to determine whether nursing homes located in the 
MDS facility file were found on Nursing Home Compare. 
Since CMS composes the MDS facility file monthly, it was possible that 
some of these nursing homes were in stages of closure or termination from 
Medicare. Therefore, these nursing homes might not be included in the 
MDS facility file for other months. For this reason, we matched nursing 
homes not found in the March 2003 MDS facility file with those in the 
February, April, May, and June 2003 MDS facility files. We also checked 
Nursing Home Compare in September 2003 to determine whether the site 
still included the unmatched nursing homes. 
To further investigate whether nursing homes not found in both 
databases remained in operation, we used various methods. These 
included contacting one State survey agency, rechecking nursing homes in 
the May, September, and October Nursing Home Compare database, 
Internet searches, and phone calls to nursing homes. 
Nursing Home Compare Accuracy 
To determine whether Nursing Home Compare includes all intended 
inspection results, as well as to determine the accuracy of nursing home 
inspection results provided on Nursing Home Compare, we matched 
inspection results for a sample of nursing homes in Nursing Home 
Compare with survey documentation supplied by State survey agencies. 
We selected a simple random sample of 150 nursing homes from the 
16,446 included in Nursing Home Compare as of April 1, 2003. The 
OEI-01-03-00130  INSPECTION RESULTS ON NURSING HOME COMPARE: COMPLETENESS AND ACCURACY  15  A P P E N D I X ~ B  
sample included nursing homes from 39 States, with a range of 1 to 20 
nursing homes per State. Our sample produces an estimate of the 
proportion of cases in which there is an inaccuracy between State survey 
documentation and Nursing Home Compare. This estimate is within 8 
percentage points of the actual inaccuracy rate at the 95 percent 
confidence level. 
For each State represented in our sample, we sent a letter to the State's 
director of nursing home certification requesting survey documentation 
for each sampled nursing home. Before sending the letter, we solicited 
comments from CMS central office officials, CMS regional office 
surveyors, and a State surveyor on its content and clarity. We requested 
the following survey documentation for each sampled nursing home in the 
State. 
	  statements of deficiencies (CMS-2567 forms) from certification 
(health) surveys, conducted between January 1, 1999 and March 31, 
2003, as well as documentation of the completed corresponding life 
safety code survey 
	  statements of deficiencies (CMS-2567 forms) from complaint 
investigations, conducted between December 1, 1999 and February 
28, 2003 
	  copies of letters from nursing homes requesting informal dispute 
resolution for any deficiencies cited from the health surveys and 
complaint investigations listed above, as well as documentation of 
the informal dispute resolution outcome 
All States in our sample sent requested documentation for a 100 percent 
response rate. However, two States could not locate some information 
needed for our analysis for a total of four nursing homes. This 
information included one health survey each for two nursing homes, three 
complaint investigations for one nursing home, and two health surveys for 
one nursing home. Two of the missing health surveys were from 1998, 
and one was from 1997. In the instances where States could not locate 
information, we did not score Nursing Home Compare as inaccurate. 
We allowed State survey agencies to either mail paper copies of 
documentation or electronically mail files with the documentation. Of the 
39 States represented in our sample, 22 mailed paper copies of all 
documentation, 10 mailed files electronically, and 7 sent us both paper 
copies and electronic files. When necessary, we contacted States by 
OEI-01-03-00130  INSPECTION RESULTS ON NURSING HOME COMPARE: COMPLETENESS AND ACCURACY  16  A P P E N D I X ~ B  
telephone or electronic mail and requested clarifications or additional 
information. 
To perform the comparison between the State survey documentation and 
Nursing Home Compare, we downloaded and printed the complete 
inspection information section from Nursing Home Compare in April 2003 
and matched Nursing Home Compare's information with the State 
documentation we received. We verified the following information: 
	  inclusion of the three most recent health surveys and all 
substantiated complaint investigations occurring between 
December 1, 1999 and February 28, 2003 
  dates of health surveys and complaint investigations 
	  number of deficiencies per health survey and complaint 
investigation 
	  descriptions of deficiencies found during each survey or complaint 
investigation 
  description of severity for each deficiency 
  description of affected residents (scope) for each deficiency. 
We also reviewed informal dispute resolution documentation to determine 
whether relevant deficiencies presented on Nursing Home Compare 
reflected the outcome of informal dispute resolution. 
To account for the time between the end of a survey and the transmission 
of its data into Nursing Home Compare, we employed criteria from the 
Fiscal Year 2003 CMS State Performance Guidelines. These guidelines 
require State agencies to enter results in the Online Survey Certification 
and Reporting (OSCAR) system within 70 days of the last day of the 
health survey or complaint investigation. Hence, we do not count any 
health survey or complaint investigation occurring after January 10, 2003 
as missing from Nursing Home Compare. 
When information from State survey documentation was not accurately 
reflected on Nursing Home Compare, we recorded the inaccuracies on 
paper forms and used Microsoft Access and SAS statistical software to 
analyze the data. 
OEI-01-03-00130  INSPECTION RESULTS ON NURSING HOME COMPARE: COMPLETENESS AND ACCURACY  17  A P P E N D I X ~ B  
The table below presents point estimates and confidence intervals at the 
95 percent confidence level for key figures presented in the findings 
section. 
Point  95 percent 
Estimate  confidence 
interval 
Percent of nursing homes with no 
68%  +/- 7.5% 
inaccuracies 
Percent of nursing homes missing one or 
19%  +/- 6.3% 
more surveys on Nursing Home Compare 
Percent of nursing homes with deficiencies 
incorrectly posted on Nursing Home  15%  +/- 5.7% 
Compare 
Percent of nursing homes with deficiencies 
11%  +/- 5.0% 
missing from Nursing Home Compare 
Percent of nursing homes with an incorrect 
8%  +/- 4.3% 
scope and/or severity score 
Percent of nursing homes with an incorrect 
4%  +/- 3.1% 
deficiency date 
Percent of nursing homes with an incorrect 
3%  +/- 2.7% 
deficiency description 
Interviews with CMS staff 
We interviewed CMS officials to learn about the process by which 
information from OSCAR is transmitted onto Nursing Home Compare, 
and the data systems involved. These interviews included staff from the 
Center for Beneficiary Choices, and Center for Medicaid and State 
Operations in CMS's central office, as well as CMS regional office staff. 
Document reviews 
We reviewed CMS documents that provide procedures or guidelines 
relevant to Nursing Home Compare. These documents include the State 
Operations Manual and the State Survey Agency Performance Standards 
for Fiscal Year 2003. 
OEI-01-03-00130  INSPECTION RESULTS ON NURSING HOME COMPARE: COMPLETENESS AND ACCURACY  18    A P P E N D I X   ~   C  
 
Agency Comments 
OEI-01-03-00130  INSPECTION RESULTS ON NURSING HOME COMPARE: COMPLETENESS AND ACCURACY  19
  A P P E N D I X ~ C  
OEI-01-03-00130  INSPECTION RESULTS ON NURSING HOME COMPARE: COMPLETENESS AND ACCURACY  20
  A P P E N D I X ~ C  
OEI-01-03-00130  INSPECTION RESULTS ON NURSING HOME COMPARE: COMPLETENESS AND ACCURACY  21
    A C K N O W L E D G E M E N T S  
 
This report was prepared under the direction of Mark R. Yessian, Ph.D., 
Regional Inspector General for Evaluation and Inspections in the Boston 
Regional Office, and Joyce M. Greenleaf, MBA, Assistant Regional 
Inspector General. Other principal Office of Evaluation and Inspections 
staff who contributed include: 
Maria E. Maddaloni, Project Leader 
China D. Eng, Program Analyst
 
Baltimore:
 
Sandy Khoury, Program Specialist
 
Scott Horning, Program Analyst
 
Barbara Tedesco, Mathematical Statistician
 
OEI-01-03-00130  INSPECTION RESULTS ON NURSING HOME COMPARE: COMPLETENESS AND ACCURACY  22    E N D   N O T E S 
  
1 The Nursing Home Compare website can be accessed at 
http://www.medicare.gov/NHCompare/home.asp. 
2 The numbers presented show overall visits to Nursing Home 
Compare. CMS does not collect information that could be used to 
identify a visitor to its Web sites. Hence, CMS is not able to 
determine the number of unique visitors to Nursing Home Compare. 
3 Nursing Home Compare typically presents deficiencies from 
complaint investigations for about a 2-year period. For the data used 
in this inspection, Nursing Home Compare included deficiencies from 
complaint investigations between December 1, 1999 and February 28, 
2003. 
4 Fiscal Year 2003 State Performance Review Protocol Guidance, 
Centers for Medicare & Medicaid Services, May 2003, pp. 41-48. 
5 Fiscal Year 2003 State Performance Review Protocol Guidance, 
Centers for Medicare & Medicaid Services, May 2003, pp. 41-48. 
6 A T-test found a statistically significant difference between the 
mean number of deficiencies for nursing homes with inaccuracies and 
for nursing homes with no inaccuracies at the 95 percent confidence 
level. 
Average number of deficiencies 
Sample (n=150)  22 
Nursing homes with no inaccuracies on Nursing Home 
16 
Compare (n=103) 
Nursing homes with inaccuracies on Nursing Home Compare 
35 
(n=47) 
T= -4.94  Degrees of freedom=57.6 
7 Fiscal Year 2003 State Performance Review Protocol Guidance, 
Centers for Medicare & Medicaid Services, May 2003, pp. 44-45. 
OEI-01-03-00130  INSPECTION RESULTS ON NURSING HOME COMPARE: COMPLETENESS AND ACCURACY  23