United States Government Accountability Office

GAO Report to Congressional Committees

MEDICARE 

PAYMENT 

CMS Methodology 

Adequate to Estimate 

National Error Rate 

March 2006 

 

GAO-06-300 
What GAO Found

United States Government Accountability Office

Why GAO Did This Study

Highlights

Accountability Integrity Reliability

March 2006

MEDICARE PAYMENT

CMS Methodology Adequate to Estimate 

Highlights of GAO-06-300, a report to National Error Rate 

congressional committees 

The Centers for Medicare & 

Medicaid Services (CMS) estimated

that the Medicare program paid

approximately $20 billion (net) in 

error for fee-for-service (FFS) 

claims in fiscal year 2004. CMS 

established two programs—the

Comprehensive Error Rate Testing

(CERT) Program and the Hospital 

Payment Monitoring Program 

(HPMP)—to measure the accuracy

of claims paid. 

The Medicare Prescription Drug, 

Improvement, and Modernization 

Act of 2003 directed GAO to study 

the adequacy of the methodology 

that CMS used to estimate the 

Medicare FFS claims paid in error.

GAO reviewed the extent to which

CMS’s methodology for estimating

the fiscal year 2004 error rates was 

adequate by contractor type for 

(1) the CERT Program, (2) the 

HPMP, and (3) the combined

national error rate (including both 

the CERT Program and the HPMP). 

GAO reviewed relevant CMS 

documents and reports related to 

the CERT Program and the HPMP. 

In addition, GAO reviewed work

performed by the Department of

Health and Human Services (HHS)

Office of Inspector General (OIG) 

and its contractor that evaluated 

CMS’s fiscal year 2004 statistical 

methods and other aspects of the 

error rate estimation process. GAO 

also conducted interviews with 

officials from CMS, HHS’s OIG, and 

their contractors. 

The methodology used by CMS for the CERT Program was adequate to estimate the 

fiscal year 2004 error rates by contractor type—carrier, durable medical equipment 

regional carrier (DMERC), and fiscal intermediary (FI). Carriers pay claims 

submitted by physicians, diagnostic laboratories and facilities, and ambulance

service providers. DMERCs pay claims submitted by durable medical equipment 

suppliers. FIs pay claims submitted by hospitals, home health agencies, hospital

outpatient departments, skilled nursing facilities, and hospices. The methodology 

was adequate because CMS used a large sample—about 120,000 claims—and an

appropriate sample selection strategy. For these fiscal year 2004 error rate

estimates, CMS made improvements in the collection of medical records that 

supported the sampled claims. These medical records were appropriately reviewed 

to determine whether there were errors in payment. CMS used valid statistical 

methods to estimate the fiscal year 2004 error rates for the carrier, DMERC, and FI 

contractor types. 

The methodology used by CMS for the HPMP was adequate to estimate the fiscal 

year 2004 error rate by quality improvement organizations (QIO), which are

responsible for ascertaining the accuracy of coding and payment of Medicare FFS

paid claims for acute care inpatient hospital stays. CMS’s sampling methods were

adequate because the agency used a large sample, approximately 40,000 claims, that

was representative of the population from which it was drawn in terms of average 

dollar amount per claim. Also, the HPMP had adequate processes in place to ensure 

appropriate determinations of error. CMS used valid statistical methods to estimate 

the fiscal year 2004 error rate for the QIO contractor type. 

The fiscal year 2004 contractor-type error rate estimates for the CERT Program and 

the HPMP were appropriately combined to determine the national Medicare error 

rate through the use of a valid statistical method. CMS estimated the national 

Medicare error rate by averaging the carrier, DMERC, and FI contractor-type error 

rates in the CERT Program and the QIO contractor-type error rate in the HPMP, 

weighted by each contractor type’s share of total Medicare FFS payments. 

In written comments, HHS noted that GAO found CMS’s methodology adequate for 

estimating the fiscal year 2004 national Medicare FFS error rate. HHS also noted that

CMS is continually committed to refining the processes to estimate, as well as lower, 

the level of improper payments in the Medicare FFS program. 

Medicare Net FFS Error Rates and Dollars of Claims Paid in Error, Fiscal Year 2004 

CMS program Contractor type 

Error rate 

(percentage) 

Dollars paid in

error (in billions) 

Carrier 10.7 $6.5

DMERC 11.1 1.0

CERT Program 

FI 15.8 9.3 

HPMP QIO 3.6 3.1 

National Medicare FFS 

error rate 

All contractor types 9.3 $19.9 

Source: CMS. 

www.gao.gov/cgi-bin/getrpt?GAO-06-300.

To view the full product, including the scope 

and methodology, click on the link above. 

For more information, contact A. Bruce 

Steinwald at (202) 512-7101 or 

steinwalda@gao.gov. 
Contents 

Letter 1

Results in Brief 5 

Background 6

CMS Methodology Adequate for Estimating the Error Rates in the 

CERT Program 16 

CMS Methodology Adequate for Estimating the Error Rate in the 

HPMP 22

CMS Methodology Adequate for Estimating the National Error Rate 28 

Concluding Observations 29 

Agency Comments 30 

Appendix I Scope and Methodology 32 

Appendix II Fiscal Year 2004 Error Rate Information by 

Contractor Type—Carriers, DMERCs, FIs, and QIOs 35 

Appendix III Comments from the Department of Health and 

Human Services 40 

Appendix IV GAO Contact and Staff Acknowledgments 42 

Tables 

Table 1: Medicare FFS Error Rates and Dollars of Claims Paid in 

Error, Fiscal Year 2004 8 

Table 2: National Medicare FFS Error Rate by Category of Error, 

Fiscal Year 2004 14 

Figures 

Figure 1: Medicare FFS Error Rates Estimated through the CERT 

Program 9

Figure 2: Medicare FFS Error Rates Estimated through the HPMP 11 

Figure 3: Medicare FFS Error Rates That Produce the National 

Error Rate 13 

Page i GAO-06-300 Medicare Error Rate
Abbreviations 

CDAC Clinical Data Abstraction Center 

CERT Comprehensive Error Rate Testing 

CMS Centers for Medicare & Medicaid Services 

DMERC durable medical equipment regional carrier 

DRG diagnosis-related group 

FFS fee-for-service 

FI fiscal intermediary 

GPRA Government Performance and Results Act of 1993

HHS Department of Health and Human Services 

HPMP Hospital Payment Monitoring Program 

IPIA Improper Payments Information Act 

MAC Medicare administrative contractor 

OIG Office of Inspector General 

OMB Office of Management and Budget 

PPS prospective payment system 

QIO quality improvement organization 

This is a work of the U.S. government and is not subject to copyright protection in the 

United States. It may be reproduced and distributed in its entirety without further 

permission from GAO. However, because this work may contain copyrighted images or 

other material, permission from the copyright holder may be necessary if you wish to 

reproduce this material separately. 

Page ii GAO-06-300 Medicare Error Rate
United States Government Accountability Office

Washington, DC 20548 

March 24, 2006 

The Honorable Charles E. Grassley 

Chairman 

The Honorable Max Baucus 

Ranking Minority Member 

Committee on Finance 

United States Senate 

The Honorable Joe L. Barton 

Chairman 

The Honorable John D. Dingell 

Ranking Minority Member 

Committee on Energy and Commerce 

House of Representatives 

The Honorable William M. Thomas 

Chairman 

The Honorable Charles B. Rangel 

Ranking Minority Member 

Committee on Ways and Means 

House of Representatives 

The Centers for Medicare & Medicaid Services (CMS), the agency that 

administers the Medicare program, monitors the accuracy of claims paid 

for services provided to Medicare beneficiaries. Each fiscal year, CMS 

reports an estimate of the claims paid in error based on a sample of claims 

from previous years. In fiscal year 2004, CMS reported an error rate of 

9.3 percent, which represented approximately $20 billion in error out of 

the approximately $214 billion in fee-for-service (FFS) payments.1

 The 

fiscal year 2004 error rate estimated the percentage of FFS payments that 

did not comply with Medicare’s payment rules for a sample of claims that 

 1

Unless otherwise specified, dollars paid in error and error rates discussed in this report 

are net amounts. Net dollars paid in error were calculated by subtracting dollars paid in 

error that were due to underpayments from those that were due to overpayments. The net 

dollars paid in error were then used to estimate the error rate. CMS also reported gross

dollars paid in error and error rates in fiscal year 2004. Gross dollars paid in error were 

calculated by adding dollars paid in error that were due to underpayments to those that

were due to overpayments. 

Page 1 GAO-06-300 Medicare Error Rate
included inpatient discharges that occurred from July 1, 2002, through 

June 30, 2003, as well other services that were paid in 2003. The fiscal 

year 2004 Medicare FFS error rate was significantly higher than the goal of 

4.8 percent for that fiscal year, which CMS set under the Government 

Performance and Results Act of 1993 (GPRA).2

CMS uses several types of contractors to ensure the payment accuracy of 

Medicare claims,3

 including carriers,4 durable medical equipment regional 

carriers (DMERC),5

 fiscal intermediaries (FI),6 and quality improvement 

organizations (QIO).7

 Using contractor-specific error rate information, 

CMS estimates an error rate for each type of contractor; the agency

produces a national Medicare error rate by aggregating the four 

contractor-type error rates. In its fiscal year 2004 Medicare error rate 

report, CMS stated that it planned to use error rate information to help 

determine the underlying reasons for claim errors, such as incorrect 

coding, and implement corrective actions.8

 In a congressional testimony in 

July 2005, the Director of CMS’s Office of Financial Management stated 

 2

GPRA requires agencies to develop multiyear strategic plans, annual performance goals, 

and annual performance reports. See Pub. L. No. 103-62, 107 Stat. 285. 

3

In a few cases, program safeguard contractors are responsible for ensuring the payment 

accuracy of Medicare claims. Program safeguard contractors are Medicare contractors that 

conduct activities to address or prevent improper payments. 

4

Carriers are health insurers and pay claims submitted by physicians, diagnostic 

laboratories and facilities, and ambulance service providers. 

5

DMERCs are health insurers and pay claims submitted by durable medical equipment 

suppliers. In fiscal year 2004, a program safeguard contractor, TriCenturion, was 

responsible for medical review and for lowering the error rates in its region. 

6

FIs are almost exclusively health insurers and pay claims submitted by home health 

agencies, non-prospective payment system (PPS) hospitals, hospital outpatient 

departments, skilled nursing facilities, and hospices. PPS is a reimbursement method used 

by Medicare where the payment is made based on a predetermined rate and is unaffected 

by the provider’s actual costs. 

7

QIOs (formerly known as peer review organizations) are responsible for ascertaining the

accuracy of coding and payment of paid Medicare FFS claims for acute care inpatient

hospital stays—generally those that are covered by PPS—for Medicare beneficiaries in all 

50 states, the District of Columbia, and Puerto Rico. Unlike carriers, DMERCs, and FIs, 

however, QIOs do not process and pay claims. These activities are conducted by FIs. 

8

See Department of Health and Human Services, Centers for Medicare & Medicaid Services, 

Improper Medicare Fee-for-Service Payments Report Fiscal Year 2004 (Baltimore, Md.:

December 2004). 

Page 2 GAO-06-300 Medicare Error Rate
that CMS plans to create performance incentives for contractors.9

 CMS is 

also implementing a multiyear contractor reform initiative, which will 

reduce the number of contractors responsible for paying claims. 

To monitor the accuracy of Medicare FFS claims paid by contractors, CMS 

established two programs—the Comprehensive Error Rate Testing 

(CERT) Program and the Hospital Payment Monitoring Program (HPMP).10

Through the CERT Program, CMS monitors payment decisions made by 

three types of contractors—carriers, DMERCs, and FIs. It does this 

through a review of the claims and submitted medical record 

documentation to ensure that there is support for the payment based on 

the information reviewed for a sample of paid claims. CMS uses a similar 

process for the HPMP for a sample of claims that are reviewed by QIOs for 

accuracy of payment. 

The Department of Health and Human Services (HHS) Office of Inspector 

General (OIG)11 estimated the error rate for each fiscal year from 1996 

through 2002. CMS made significant changes to the methodology, 

including substantially increasing the size of the sample used to estimate 

the error rate, when it assumed responsibility for estimating the Medicare 

error rate in fiscal year 2003. CMS has continued to make changes to the 

methodology in subsequent years. 

The Medicare Prescription Drug, Improvement, and Modernization Act of 

2003 requires that we study the adequacy of the methodology that CMS 

used to estimate Medicare error rates and to make recommendations as 

deemed appropriate.12 Specifically, we report on the extent to which the 

methodology used by CMS to estimate the fiscal year 2004 Medicare error 

rates was adequate (1) by contractor type (carrier, DMERC, and FI) for the 

CERT Program, (2) by contractor type (QIO) for the HPMP, and (3) for the 

 9

See Senate Homeland Security and Governmental Affairs Subcommittee on Federal 

Financial Management, Government Information and International Security, Hearing on 

Medicare and Medicaid Improper Payments, Statement of the Director of Office of 

Financial Management, Centers for Medicare & Medicaid Services, 109th Congress, 

July 12, 2005. 

10Each program monitors the accuracy of paid claims that constitute approximately 

50 percent of Medicare’s FFS payments annually. 

11OIG regularly conducts audits, evaluations, and investigations pertaining to HHS 

programs. 

12Pub. L. No. 108-173, § 921(b)(3), 117 Stat. 2066, 2388-89. 

Page 3 GAO-06-300 Medicare Error Rate
combined national error rate (including the CERT Program and the 

HPMP). 

To conduct our analysis of the adequacy of the methodology that CMS 

used to estimate fiscal year 2004 Medicare error rates, we reviewed 

relevant documents, including CMS’s Medicare error rate reports for fiscal 

years 2003 and 2004, CERT and HPMP program documentation, and HHS 

OIG reports evaluating the fiscal year 2004 CERT Program and HPMP.13 In 

addition, we reviewed work performed by an OIG contractor that 

evaluated CMS’s statistical sampling and estimation methodology for the 

fiscal year 2004 Medicare error rate, including the contractor’s report and

supporting workpapers. We interviewed OIG officials; OIG contractor 

staff; CMS officials; and staff of the CERT subcontractor responsible for 

calculating the error rates for carriers, DMERCs, and FIs and the national

error rate for fiscal year 2004. Commenting on the adequacy of the 

methodology used in any other years was beyond the scope of our work. 

However, it is important to note that changes in the methodology may 

affect the estimation of the error rates and thus the comparability of these 

rates over time. 

As part of our assessment of the adequacy of the methodology that CMS 

used to estimate the Medicare error rates for fiscal year 2004, we reviewed 

the reliability of these estimates by examining the precision of the 

contractor-specific error rates, the contractor-type error rates, and the 

national error rate. Precision is the amount of variation between an 

estimate (such as the error rate for a sample of Medicare FFS paid claims) 

and the result that would be obtained from measuring the entire 

population (such as the error rate for all Medicare FFS paid claims). We 

examined precision of the error rate estimates by assessing relative 

precision, which is the standard error14 of the error rate estimate divided 

by the estimate itself. Estimates with lower relative precision are more 

reliable. For the purposes of this report, we established that a relative 

precision of no greater than 15 percent was within the acceptable 

statistical standard for precision.15 We chose relative precision because it 

 13Since the creation of the CERT Program and the HPMP in 2003, OIG has conducted 

annual reviews of the programs as part of its oversight of work performed for HHS by

contractors. 

14The standard error is a measure of variation around the estimate, in this case, the error 

rate. 

15See, for example, M.H. Hansen, W.N. Hurwitz, and W.G. Madow, Sample Survey Methods 

and Theory, vol. I (New York, N.Y.: John Wiley & Sons, Inc., 1953), 130. 

Page 4 GAO-06-300 Medicare Error Rate
allows for better comparison of the reliability of the range of error rates 

across contractors.16

During the course of our work, CMS published a report in November 2005 

that included its fiscal year 2005 error rates.17 While an evaluation of the 

methodology used to estimate the fiscal year 2005 error rates was outside 

the scope of our work, we reviewed the report and included references in 

this report where appropriate. 

For more information on our scope and methodology, see appendix I. We 

performed our work from April 2005 through March 2006 in accordance

with generally accepted government auditing standards. 

We found that the methodology used by CMS for the CERT Program was 

adequate to estimate the fiscal year 2004 error rates by contractor type 

(carrier, DMERC, and FI). CMS’s sample of 120,000 claims was sufficiently 

large to reliably estimate the error rate and was appropriately selected.

Further, CMS used systematic sampling with a random start, a method that 

is designed to ensure that the sample is representative of the population. 

CMS also had appropriate procedures in place to collect medical records 

from providers, such as physicians, durable medical equipment suppliers, 

and hospital outpatient departments, which supported the paid claims. 

Additionally, the processes used by CMS to identify and categorize 

payment errors were adequate because they ensured that the reviews 

conducted of the medical records supporting the paid claims were 

performed according to the established procedures for the CERT Program. 

This included adequate qualifications and training of those individuals 

conducting the medical record reviews. Further, CMS used valid statistical 

Results in Brief 

 16To provide illustration, consider that one contractor has an error rate of 11.9 percent with 

a standard error of 2.1 percent and a second contractor has an error rate of 20.4 percent

also with a standard error of 2.1 percent. The standard errors are the same, but relative 

precision, which is calculated by dividing the standard error by the error rate estimate, 

illustrates that the reliability of the estimates is different. Relative precision of the error 

rate estimate for the first contractor is 17.6 percent, while the relative precision of the error 

rate estimate for the second contractor is 10.3 percent. This indicates that the second 

contractor’s error rate estimate is more reliable. 

17See Department of Health and Human Services, Centers for Medicare & Medicaid 

Services, Improper Medicare FFS Payments Long Report (Web Version) for 

November 2005. 2005. 

https://www.cms.hhs.gov/apps/er_report/preview_er_report.asp?from=public&which=long

&reportID=3 (downloaded Jan. 26, 2006). 

Page 5 GAO-06-300 Medicare Error Rate
methods to estimate the fiscal year 2004 carrier, DMERC, and FI 

contractor-type error rates and standard errors. 

We found also that the methodology used by CMS for the HPMP to 

calculate the fiscal year 2004 contractor-type error rate for QIOs was 

adequate to reliably measure claims paid in error. We found the sampling 

methods to be adequate because CMS’s sample of approximately 40,000 

claims was sufficiently large to estimate the QIO contractor-type error 

rate. It was also representative of the population from which it was drawn 

in terms of average dollar amount per claim. Based on our review of 

oversight work of the HPMP conducted by OIG, we also found that the 

process used in the HPMP to collect the medical records that support the 

claims selected for review was adequate. Additionally, the processes CMS 

used to identify and categorize payment errors were adequate because 

they ensured that the reviews conducted of the medical records 

supporting the paid claims were performed according to established 

procedures for the HPMP. This included adequate qualifications and 

training of those individuals conducting the medical record reviews. CMS

also used valid statistical methods to estimate the QIO contractor-type 

error rate and standard error. 

The fiscal year 2004 error rates by contractor type (carrier, DMERC, FI, 

and QIO) were appropriately aggregated to determine the national 

Medicare error rate through the use of a valid statistical method. CMS 

estimated the national Medicare error rate by averaging the error rates of 

the four contractor types (carrier, DMERC, FI, and QIO), weighted by each 

contractor type’s proportion of total Medicare FFS payments. 

In written comments on a draft of this report, HHS noted that we found 

the CMS methodology adequate for estimating the fiscal year 2004 national 

Medicare FFS error rate. HHS also noted that CMS is continually 

committed to refining the processes to estimate, as well as lower, the level 

of improper payments in the Medicare FFS program. 

In fiscal year 2003, CMS assumed responsibility for estimating the national 

Medicare error rate, a responsibility that had previously been held by HHS 

OIG. OIG began estimating the national Medicare error rate in fiscal year 

Background 

Page 6 GAO-06-300 Medicare Error Rate
1996,

18 and continued doing so for each subsequent fiscal year through 

2002. The transfer of responsibilities for estimating the national Medicare 

error rate to CMS coincided with the implementation of the Improper 

Payments Information Act of 2002 (IPIA). The IPIA requires federal 

agencies to estimate and report annually on the extent of erroneous 

payments in their programs and activities.19 The IPIA defines an improper 

payment as any payment that should not have been made or that was 

made in an incorrect amount, including both under- and overpayments. All 

agencies that identify a program as susceptible to significant improper 

payments, defined by guidance from the Office of Management and Budget

(OMB) in 2003 as exceeding both 2.5 percent of total program payments 

and $10 million,20 are required to annually report to Congress and the 

President an estimate of improper payments and report on corrective 

actions. 

In addition to estimating the national Medicare error rate for purposes of 

compliance with the IPIA, CMS also began producing contractor-specific 

error rate estimates beginning in fiscal year 2003 to identify the underlying

causes of errors and to adjust action plans for carriers, DMERCs, FIs, and 

QIOs. To produce these contractor-specific error rate estimates for fiscal 

year 2004, CMS sampled approximately 160,000 claims. The contractorspecific error rate information was then aggregated by the four contractor 

types (carrier, DMERC, FI, and QIO), which were ultimately combined to 

estimate the national Medicare error rate. Under the methodology 

previously used by OIG to estimate the national Medicare error rate, 6,000 

claims were sampled. While the sample size used by OIG was sufficient to 

estimate the national Medicare error rate, it was not sufficient to reliably 

estimate the contractor-specific error rates. Additionally, the increased 

sample size improved precision of the national Medicare error rate 

estimate. 

 18According to OIG testimony in February 2000, OIG began estimating the national 

Medicare error rates in fiscal year 1996 as part of its audit of CMS’s financial statements. 

See House Committee on the Budget, Statement of Inspector General, Department of 

Health and Human Services, Hearing on Medicare and Medicaid: HHS High-Risk 

Programs, 106th Congress, February 17, 2000. 

19Pub. L. No. 107-300, 116 Stat. 2350 (codified at 31 U.S.C. § 3321 note). 

20OMB Mem. M-03-13 (2003). 

Page 7 GAO-06-300 Medicare Error Rate
The objective of the CERT Program and the HPMP is to measure the 

degree to which CMS, through its contractors, is accurately paying claims. 

Through the CERT Program, CMS monitors the accuracy of Medicare FFS

claims that are paid by carriers, DMERCs, and FIs. In fiscal year 2004, the 

Medicare error rates by contractor type as estimated through the CERT 

Program were 10.7 percent for the carrier contractor type, 11.1 percent for 

the DMERC contractor type, and 15.8 percent for the FI contractor type. 

(See table 1.) 

Table 1: Medicare FFS Error Rates and Dollars of Claims Paid in Error, Fiscal 

Year 2004 

CMS program Contractor type

Error rate 

(percentage)

Dollars paid in

error (in

billions)

CERT Program Carrier 10.7 $6.5 

DMERC 11.1 1.0

FI 15.8 9.3

HPMP QIO 3.6 3.1

National Medicare FFS 

error rate 

All contractor

types 

9.3 $19.9

Source: CMS. 

Notes: This table reflects net Medicare FFS error rates and dollars of claims paid in error. Based on 

data provided in CMS’s fiscal year 2005 error rate report, we calculated the net Medicare FFS error 

rates and net dollars paid in error for fiscal year 2005 by contractor type as follows: carriers— 

6.0 percent and $4.1 billion; DMERCs—8.6 percent and $0.8 billion; FIs—3.2 percent and $2.0 billion; 

and QIOs—3.8 percent and $3.5 billion. The national Medicare FFS error rate and dollars paid in 

error were 4.4 percent and $10.3 billion. See Department of Health and Human Services, Centers for 

Medicare & Medicaid Services, Improper Medicare FFS Payments Long Report (Web Version) for 

November 2005. 2005. 

https://www.cms.hhs.gov/apps/er_report/preview_er_report.asp?from=public&which=long&reportID=3

(downloaded Jan. 26, 2006). 

Through the HPMP, CMS monitors the accuracy of paid Medicare FFS 

claims for acute care inpatient hospital stays—generally those that are 

covered under the prospective payment system (PPS). For fiscal year 2004, 

the Medicare error rate for the QIO contractor type, as estimated through 

the HPMP, was 3.6 percent. (See table 1.) 

To estimate contractor-specific Medicare FFS error rates for the CERT 

Program, CMS reviews a sample of claims from each of the applicable

contractors, which included 25 carriers, 4 DMERCs, and 31 FIs for the 

fiscal year 2004 error rates. These error rates are then aggregated by 

contractor type. (See fig. 1.) For fiscal year 2004, CMS contracted with 

CMS Programs to Monitor 

the Payment Accuracy of 

Medicare FFS Claims 

CERT Program 

Page 8 GAO-06-300 Medicare Error Rate
AdvanceMed to administer the CERT Program. AdvanceMed sampled 

approximately 120,000 claims submitted from January 1, 2003, through 

December 31, 2003, to estimate the fiscal year 2004 contractor-specific and 

contractor-type error rates for the CERT Program. 

Figure 1: Medicare FFS Error Rates Estimated through the CERT Program

Source: GAO analysis of CMS data.

CERT CERT CERT

1 carrier contractor-type 

error rate

1 DMERC contractortype error rate1 FI contractor-type 

error rate

25 carrier-specific 

error rates

4 DMERC-specific

error rates

31 FI-specific 

error rates

Carrier-specific, 

DMERC-specific, and 

FI-specific error rates are 

collectively known as

contractor-specific

 error rates.

All contractor-specific rates

are aggregated into one 

contractor-type error rate.

21 22 23 24 25

1 2 3 4 5

6789 10

11 12 13 14 15

16 17 18 19 20

1 2 3 4 27 28 29 30 31

7 8 9 10 11

12 13 14 15 16

17 18 19 20 21

22 23 24 25 26

1

2 3 4 5 6

1 11

For each of the approximately 120,000 sampled claims, AdvanceMed 

requested the medical records from the provider that rendered the service

or from the contractor that processed the related claim, if the contractor 

previously performed a medical review on the claim. If a provider did not

respond to the initial request for medical records after 19 days, 

AdvanceMed initiated a series of follow-up procedures in an attempt to 

obtain the information. The follow-up procedures with nonresponding 

providers for fiscal year 2004 included three written letters and three 

contacts by telephone. Additionally, in fiscal year 2004, OIG followed up 

directly with nonresponders on claims over a certain dollar amount. If 

Page 9 GAO-06-300 Medicare Error Rate
medical records were not received within 55 days of the initial request, the 

entire amount of the claim was classified by AdvanceMed as an 

overpayment error. 

When medical records were received from the provider or from the 

contractor, CERT medical review staff reviewed the claim (which billed 

for the services provided) and the supporting medical records (which 

detailed the diagnosis and services provided) to assess whether the claim 

followed Medicare’s payment rules and national and local coverage

decisions.21 Claims that did not follow these rules were classified by 

AdvanceMed as being in error. Providers whose claims were reviewed 

were allowed to appeal these claims, and if the error determination for a 

claim was overturned through the appeals process, AdvanceMed adjusted 

the error rate accordingly. For the fiscal year 2004 error rate, AdvanceMed

notified individual carriers, DMERCs, and FIs of their respective payment 

errors.22

HPMP For the HPMP, CMS analyzes a sample of claims across QIOs to estimate 

Medicare error rates by state, because QIOs are organizations with statebased service areas. CMS estimated the QIO contractor-type error rate by

aggregating the QIO error rate estimates for each of the 50 states, the 

District of Columbia, and Puerto Rico. (See fig. 2.) Through the HPMP, 

CMS sampled approximately 40,000 claims for acute care inpatient 

hospital discharges that occurred from July 1, 2002, through June 30, 2003, 

to estimate the fiscal year 2004 state-specific and contractor-type error 

rates for QIOs. 

 21In Medicare, decisions about whether and under what circumstances new procedures or

devices are covered are made nationally by CMS or locally by Medicare contractors for 

beneficiaries in their service areas. 

22In the fiscal year 2005 error rate report, CMS reported that carriers, DMERCs, and FIs 

collected overpayments identified during the November 2005 error rate reporting period. 

Further, CMS reported that the agency will instruct carriers, DMERCs, and FIs to make 

payments to providers in underpayment cases identified for the November 2006 and later 

reports. See Department of Health and Human Services, Centers for Medicare & Medicaid 

Services, Improper Medicare FFS Payments Long Report (Web Version) for 

November 2005. 

Page 10 GAO-06-300 Medicare Error Rate
Figure 2: Medicare FFS Error Rates Estimated through the HPMP 

Source: GAO analysis of CMS data.

The 52 QIO error rates

are also known as

contractor-specific

error rates.

All contractor-specific 

rates are aggregated 

into one contractortype error rate.

1 QIO contractor-type

error rate

52 QIO error rates for the 50 states,

the District of Columbia, and Puerto Rico

44 45 46 47 48

8 9 10 11 12

17 18 19 20 21

26 27 28 29 30

35 36 37 38 39

49 50 51 52

13 14 15 16

22 23 24 25

31 32 33 34

40 41 42 43

1

1 2 3 4 67 5

HPMP

For fiscal year 2004, CMS contracted with two organizations known as 

Clinical Data Abstraction Centers (CDAC)—AdvanceMed and 

DynKePRO—that were responsible for requesting medical records from 

providers for each of the approximately 40,000 sampled claims. Each 

CDAC was responsible for reviewing the sampled claims, which were 

assigned on the basis of the geographic location where the discharge 

occurred. Upon receipt of the medical records, CDAC admission necessity

reviewers screened the related claims for the appropriateness of the 

hospitalization and, with the exception of claims from Maryland, coding

specialists independently recoded diagnosis-related groups (DRG) based 

Page 11 GAO-06-300 Medicare Error Rate
on the records submitted.23 Because Maryland does not use DRG coding, 

nonphysician reviewers screened claims from Maryland to determine 

whether the length of the acute care inpatient hospital stay was 

appropriate.24, 25 Claims that failed the screening process, including those 

where the admission was determined to be unnecessary or where an 

inappropriate DRG code was used, were forwarded to the QIO responsible 

for the state where the discharge occurred for further review. Records not 

received by the CDACs within 30 days of the request for information were 

“canceled” and referred to the QIO to be processed as overpayment errors 

caused by nonresponse. The QIO referred these claims to the FI 

responsible for paying the claim for the necessary payment adjustments. 

At the QIO, claims forwarded from the CDACs underwent further review, 

primarily medical necessity admission reviews and DRG validations. 

Determinations of error were made by QIO physician reviewers. Providers 

whose claims were reviewed were given the opportunity to provide 

comments or discuss the case and pursue additional review, which could

result in an appeal to an administrative law judge. After the matter was 

resolved, resulting in a determination that a provider was either underpaid 

or overpaid, the QIO forwarded the claim to the FI for payment 

adjustment. 

 23DRG coding is the classification system used by Medicare to group patients according to

diagnosis, type of treatment, age, and other criteria. Under PPS, hospitals are paid a

predetermined rate for treating patients based on the specific DRG category, regardless of

the actual cost of care for the individual. 

24Maryland is the only state that does not use the PPS system for acute care inpatient 

hospitals. Maryland instead has an alternative payment system, known as an all-payer 

system, in which the state decides each hospital’s level of reimbursement and requires that

all payers be charged the same rate for the same service. Medicare and Medicaid pay the 

state-approved rates. 

25Claims from Maryland with length of stay errors are considered medically unnecessary 

services. Length of stay reviews identified cases of potential delayed discharge. For

example, the patient was medically stable, and continued hospitalization was unnecessary. 

Page 12 GAO-06-300 Medicare Error Rate
CMS estimated the national Medicare FFS error rate by combining the 

three contractor-type error rates (carrier, DMERC, and FI) from the CERT 

Program and the one contractor-type error rate (QIO) from the HPMP. 

(See fig. 3.) 

Estimation of the National 

Medicare FFS Error Rate 

Figure 3: Medicare FFS Error Rates That Produce the National Error Rate 

Source: GAO analysis of CMS data.

All four contractor-type 

error rates are aggregated 

into the national error rate.

1 national FFS Medicare 

error rate

1

1 carrier contractor-type 

error rate

1 DMERC contractortype error rate1 FI contractor-type 

error rate

25 carrier-specific 

error rates

4 DMERC-specific

error rates

31 FI-specific 

error rates

21 22 23 24 25

1 2 3 4 5

6789 10

11 12 13 14 15

16 17 18 19 20

1 2 3 3 4 27 28 29 30 1

7 8 9 10 11

12 13 14 15 16

17 18 19 20 21

22 23 24 25 26

1

2 3 4 5 6

1 11

1 QIO contractor-type

error rate

52 QIO error rates for the 50 states,

the District of Columbia, and Puerto Rico

44 45 46 47 48

8 9 10 11 12

17 18 19 20 21

26 27 28 29 30

35 36 37 38 39

49 50 51 52

13 14 15 16

22 23 24 25

31 32 33 34

40 41 42 43

1

1 2 3 4 67 5

CERT CERT CERT HPMP

Page 13 GAO-06-300 Medicare Error Rate
Medicare FFS claims that were paid in error as identified by the CERT 

Program and the HPMP for the fiscal year 2004 error rates were sorted 

into one of five categories of error: 

• Insufficient documentation: Provider did not submit sufficient 

documentation to support that the services billed were actually provided. 

• Nonresponse: Provider did not submit any documentation to support that 

the services billed were actually provided. 

• Medically unnecessary services: Provider submitted sufficient 

documentation, but the services that were billed were deemed not 

medically necessary or the setting or level of care was deemed 

inappropriate. 

• Incorrect coding: Provider submitted documentation that supported a 

different billing code that was associated with a lower or higher payment 

than that submitted for the services billed. 

• Other: Provider submitted documentation, but the services billed did not 

comply with Medicare’s benefit or other billing requirements. 

See table 2 for the national Medicare FFS error rate by category of error 

for fiscal year 2004. 

Table 2: National Medicare FFS Error Rate by Category of Error, Fiscal Year 2004 

Net errors as a percentage of total dollar 

amount sampled (fiscal year 2004)

Category of error 

Insufficient documentation 4.1

Nonresponse 2.8

Medically unnecessary 1.6

Incorrect coding 0.7

Other 0.2

National Medicare FFS error rate 9.3

Source: CMS. 

Notes: This table reflects net Medicare FFS error rates generated by both the CERT Program and the

HPMP. Numbers do not sum to total because of rounding. 

As reported in CMS’s fiscal year 2004 Medicare error rate report, the 

agency planned to use the error rates to help determine the underlying 

reasons for claim errors, such as incorrect coding or nonresponse, and 

Page 14 GAO-06-300 Medicare Error Rate
implement corrective action plans for carriers, DMERCs, FIs, and QIOs.26

Draft statements of work, dated February and April 2005, for carriers, 

DMERCs, and FIs set goals for contractors to achieve a paid claims error 

rate of less than a certain percentage, to be determined by CMS. According 

to the standards for minimum performance on QIO statements of work 

that ended in 2005 for some QIOs and 2006 for other QIOs,27 QIOs are 

evaluated on 12 tasks, one of which is the HPMP. QIOs have to meet the 

performance criteria standards on 10 tasks set forth by CMS to be eligible

for a noncompetitive contract renewal. 

CMS’s use of the error rates is being done in the context of the agency’s 

current effort to significantly reform its contracting efforts for the 

payment of Medicare claims.28 By July 2009, CMS plans to reduce the total

number of contractors responsible for paying Medicare claims to 23 total 

contractors, which the agency refers to as Medicare administrative 

contractors (MAC). CMS also plans to institute performance incentives in 

the new contracts, which will be based on a number of different factors, 

including the Medicare error rates. According to CMS’s report to Congress 

on Medicare contracting reform, CMS believes that the consolidation of 

Medicare contractors and the integration of processing for Medicare 

claims29 will lead to a reduced Medicare error rate.30

 26See Department of Health and Human Services, Centers for Medicare & Medicaid 

Services, Improper Medicare Fee-for-Service Payments Report Fiscal Year 2004. 

27CMS entered into multiyear contracts with QIOs divided into three groups. Each of the

three groups had different contract end dates. 

28See GAO, Medicare Contracting Reform: CMS’s Plan Has Gaps and Its Anticipated

Savings Are Uncertain, GAO-05-873 (Washington, D.C.: Aug. 17, 2005). 

29Under the current contracting structure, Medicare Part A and Part B claims are paid by 

different types of contractors. Part A covers inpatient hospital care, skilled nursing facility 

care, some home health care services, and hospice care, which are paid by FIs. Part B 

services include physician and outpatient hospital services, diagnostic tests, mental health 

services, outpatient physical and occupational therapy, ambulance services, some home

health services, and medical equipment and supplies, which are paid by carriers and

DMERCs. Under the reformed structure, MACs will be responsible for both Part A and B

claims. 

30See Department of Health and Human Services, Medicare Contracting Reform: A 

Blueprint for a Better Medicare (Washington, D.C.: Feb. 7, 2005). 

Page 15 GAO-06-300 Medicare Error Rate
The methodology used by CMS in the CERT Program to estimate error 

rates by contractor type (carrier, DMERC, and FI) in fiscal year 2004 was 

adequate. We found that the sample size and the use of systematic 

sampling with a random start were adequate to reliably estimate the 

Medicare error rates by contractor type. The CERT Program also had 

adequate processes in place to collect medical records and to accurately 

identify and categorize payment errors. The statistical methods that CMS 

used to estimate the contractor-type error rates were valid.

The sample size that CMS used in the CERT Program, approximately 

120,000 claims, was sufficiently large to produce reliable estimates of the 

fiscal year 2004 Medicare error rates by contractor type (carrier, DMERC, 

and FI). CMS selected 167 claims each month on a daily basis from each of 

the 60 contractors, including 25 carriers, 4 DMERCs, and 31 FIs.31 This 

sample generated error rate estimates by contractor type within 

acceptable statistical standards, such as relative precision of no greater 

than 15 percent.32 Specifically, the error rate for the carrier contractor type 

was 10.7 percent with a relative precision of 3.7 percent, the error rate for 

the DMERC contractor type was 11.1 percent with a relative precision of 

13.5 percent, and the error rate for the FI contractor type was 15.7 percent 

with a relative precision of 4.5 percent. 

Further, we found that the sampling methods were adequate because CMS 

used a systematic sample with a random start.33 Sampling methods that 

employ a random start are designed to ensure that the sample selected is 

representative of the population from which it is drawn. We reviewed 

CERT Program documentation, which described the use of a systematic 

sample with a random start. The OIG contractor reviewed the computer 

CMS Methodology 

Adequate for 

Estimating the Error 

Rates in the CERT 

Program 

Sampling Methods 

 31While CMS selected an equal sample from each contractor, the final sample sizes among 

contractors varied. Some selected claims were excluded from the final sample because the

claims were missing information, such as dates of service and provider or patient 

information. 

32Relative precision of no greater than 15 percent is considered to be within acceptable 

statistical standards. See, for example, Hansen, Hurwitz, and Madow, 130.

33Systematic sampling is a selection procedure by which the sample is selected from the

population (Medicare claims) on the basis of a uniform interval, such as every fifth claim, 

between sampling units (claims), after a random starting point has been determined. The 

uniform interval is determined by dividing the given sample size into the population size 

and dropping decimals in the result. The random start is determined by using an acceptable

method of selecting random numbers and is a number between 1 and the uniform interval. 

Page 16 GAO-06-300 Medicare Error Rate
program used for the CERT Program sample selection and verified that the 

claims were selected according to the documentation. CMS officials told

us that the CERT Program conducts tests to compare the sampled claims 

to the population of claims. For example, CMS compared the percentage 

of claims sampled in each category of Medicare-covered service to the

percentage of claims in the population by category of Medicare-covered 

service. CMS provided us with an example of this test for one contractor’s 

claims from January 2003 through June 2003. 

While the relative precision of the fiscal year 2004 error rate estimates by 

contractor type for the CERT Program was within acceptable statistical 

standards of no greater than 15 percent, the relative precision of half of the

contractor-specific error rate estimates was not. (See app. II for 

contractor-specific error rate information, including the estimates and 

corresponding relative precision, for carriers, DMERCs, and FIs.) 

Thirty of the 60 contractor-specific error rates had relative precision that 

were not within acceptable statistical standards.34 Additionally, the relative 

precision of the contractor-specific error rates showed wide variation 

within each contractor type. Relative precision among carriers ranged 

from 8.9 percent to 17.0 percent; among DMERCs, relative precision 

ranged from 12.3 percent to 20.7 percent; and among FIs, relative precision 

ranged from 10.3 percent to 42.5 percent. As demonstrated by the range in 

relative precision among FIs, for example, the error rate estimate for one 

FI was nearly four times more reliable than the error rate estimate for 

another. 

The variation in relative precision among the contractor-specific error rate 

estimates was due, in part, to the sampling method CMS used for the 

CERT Program. CMS took an equal sample size from each contractor 

despite the fact that individual contractors accounted for varied amounts 

of Medicare claim volumes and total payments. For example, the claim 

volume for carriers in 2003 ranged from a minimum of 5.3 million claims to 

a maximum of 206 million claims; total payments for carriers in 2003 

ranged from a minimum of $168 million to about $6.7 billion.

CMS officials told us that they plan to reallocate the CERT Program 

sample at the contractor level by increasing the sample size for those 

 34Of the 30 contractor-specific error rates with relative precision above acceptable 

statistical standards, 25 were FIs. 

Page 17 GAO-06-300 Medicare Error Rate
contractors that are not reaching CMS’s targeted precision and by 

decreasing the sample size for those contractors that are reaching targeted

precision and achieving low error rates. In September 2005, CMS officials 

reported that this change to the methodology is expected to be

implemented for the fiscal year 2007 error rate estimation, which will be 

based on claims processed in parts of 2006 and 2007. We support CMS’s 

planned changes to its sampling methodology. We believe that reallocation 

of the sample as planned by CMS will improve the relative precision of 

these estimates. If future samples were based on the volume of claims or 

total payments of each contractor and the relative precision of the 

contractor-specific error rate rather than on the current basis of an equal 

allocation across contractors, relative precision would likely be improved 

for the contractor-specific error rates of those targeted contractors that 

were allocated a larger sample. This is because relative precision improves 

with increased sample size. There would also likely be decreased variation 

in relative precision across all contractor-specific error rates.35 These 

results could be achieved without increasing the overall sample size for 

the CERT Program. 

Based on our review of oversight work conducted by OIG, we found that 

the process CMS used to collect medical records from providers for the 

CERT Program was adequate. Staff of AdvanceMed, the CMS contractor

responsible for administering the CERT Program, were responsible for 

requesting medical records for each of the approximately 120,000 sampled 

claims used to estimate the fiscal year 2004 error rates. According to an 

OIG review of CMS’s corrective actions to improve nonresponse in the 

CERT Program for fiscal year 2004, AdvanceMed conducted a timely and 

systematic follow-up with providers that did not respond to initial requests

for medical records.36 For the medical records collection process for the 

fiscal year 2004 error rates, CMS implemented corrective actions in the 

CERT Program to address the factors associated with the high rate of 

nonresponse experienced during the medical records collection process 

Medical Record Collection 

Process 

 35See, for example, W.G. Cochran, Sampling Techniques, 3rd Ed. (New York, N.Y.: John

Wiley & Sons, 1977), 96-99.

36See Department of Health and Human Services, Office of Inspector General, Review of 

Corrective Actions to Improve the Comprehensive Error Rate Testing Process for 

Obtaining Medical Records, A-03-04-00005 (Washington, D.C.: June 2004). See also 

Department of Health and Human Services, Office of Inspector General, Review of 

Providers’ Responsiveness to Requests for Medical Records Under the Comprehensive

Error Rate Testing Program, A-01-04-00517 (Washington, D.C.: September 2004). 

Page 18 GAO-06-300 Medicare Error Rate
for the prior fiscal year. According to the CMS fiscal year 2003 error rate

report, for example, the agency found that some nonresponse in fiscal year 

2003 was due to providers’ lack of familiarity with AdvanceMed.37 In 

previous years when OIG had responsibility for estimating the Medicare 

error rate, OIG requested medical records directly from providers; 

providers were familiar with OIG and understood the importance of 

complying with the requests. However, when the responsibility for 

estimating the Medicare error rate was transferred to CMS, many 

providers were unfamiliar with AdvanceMed and may have been reluctant

to submit medical records to an unknown company. Another factor that 

caused provider nonresponse in fiscal year 2003, according to the CMS 

report, was providers’ confusion about the submission of medical records 

within the constraints of the privacy regulations issued by HHS under the 

Health Insurance Portability and Accountability Act of 1996,38 which limit 

the use and release of individually identifiable health information. 

According to the CMS report, CMS found that providers were sometimes 

unaware that sending medical records to the CERT Program contractor 

was permissible under the regulations. As reported in the OIG review cited

previously, CMS implemented corrective actions that increased provider 

compliance with medical record requests in fiscal year 2004. According to

the OIG report, CMS conducted educational efforts to clarify the role of 

AdvanceMed. Additionally, OIG further reported that CMS took action to 

address providers’ concerns about compliance with the privacy 

regulations by revising its request letters to providers to highlight 

AdvanceMed’s authorization, acting on CMS’s behalf, to obtain medical 

records as requested. OIG told us that CMS instructed carriers, DMERCs, 

and FIs to refer certain claims for nonresponding providers to OIG for 

follow-up.39

These improvements in the process used to collect medical records in the 

CERT Program helped reduce nonresponse. According to information 

provided to us by CMS, the percentage of error caused by nonresponse in 

the CERT Program decreased from 61 percent for fiscal year 2003 to 

 37See Department of Health and Human Services, Centers for Medicare & Medicaid 

Services, Improper Medicare Fee-for-Service Payments Fiscal Year 2003 (Baltimore, Md.: 

December 2003). 

3845 C.F.R. Parts 160 and 164 (2005). 

39Claims greater than $40 were referred to OIG for follow-up. 

Page 19 GAO-06-300 Medicare Error Rate
34 percent in fiscal year 2004.

40 According to CMS’s fiscal year 2005 error 

rate report, the agency continued several corrective actions to address 

nonresponse for sampled claims for the fiscal year 2005 error rates.41

Further, beginning with claims sampled to estimate the fiscal year 2006

Medicare error rates, CMS transferred the medical record collection duties 

to a second contractor, Lifecare Management Partners, which the agency 

refers to as the CERT Program documentation contractor. CMS officials 

told us that the CERT Program documentation contractor is automating 

the medical record collection process and eliminating paper copies of 

documentation. 

Based on our review of OIG’s fiscal year 2004 CERT Program evaluation, 

we concluded that the processes used in the CERT Program to identify 

and categorize payment errors for fiscal year 2004 were adequate because

the medical record reviews were performed appropriately and the CERT 

Program staff conducting the reviews were adequately trained and 

qualified.42 Staff of the CERT Program contractor, AdvanceMed, reviewed 

the medical records to verify that claims were processed according to 

Medicare payment rules; if not, a claim was found to be in error and 

assigned to one of five categories of error (insufficient documentation, 

nonresponse, medically unnecessary, incorrect coding, or other). We 

reviewed work conducted by OIG that found AdvanceMed, the CMS 

contractor responsible for administering the CERT Program, had 

appropriate controls in place to ensure that the medical record reviews 

were performed in accordance with established CERT Program 

Identification and 

Categorization of Payment 

Errors 

 40In fiscal year 2003, 54.7 percent of the national Medicare error rate was due to 

nonresponse. In fiscal year 2004, nonresponse decreased to 29.7 percent of the national 

Medicare error rate. 

41According to CMS’s fiscal year 2005 error rate report, the CERT Program reduced error

caused by nonresponse in fiscal year 2005 through several corrective actions, including 

educating providers about the CERT Program and encouraging providers to submit medical 

records by fax. Unlike fiscal year 2004, for which CMS reported net error rates, in fiscal 

year 2005, CMS reported only gross error rates and gross dollars paid in error; therefore we 

can only compare gross figures for nonresponse for fiscal years 2004 and 2005. As a

percentage of the total gross Medicare error rate, nonresponse decreased from 30.7 percent 

in fiscal year 2004 to 13.5 percent in fiscal year 2005. See Department of Health and Human 

Services, Centers for Medicare & Medicaid Services, Improper Medicare FFS Payments 

Long Report (Web Version) for November 2005. 

42See Department of Health and Human Services, Office of Inspector General, Oversight 

and Evaluation of the Fiscal Year 2004 Comprehensive Error Rate Testing Program, A03-04-00007 (Washington, D.C.: November 2004). 

Page 20 GAO-06-300 Medicare Error Rate
procedures. We also reviewed work by OIG, which examined the 

educational and training requirements for medical record reviewers as 

established in the CERT Program and assessed selected training files for 

selected medical record reviewers. OIG officials told us that they found 

these selected CERT Program medical record reviewers to be adequately

trained and qualified. 

OIG found that AdvanceMed did not complete all required quality 

assurance reviews within the designated time frame. CMS told OIG that it 

planned to reduce AdvanceMed’s workload. AdvanceMed conducts quality 

assurance reviews on a sample of medically reviewed claims to validate 

the initial reviewer’s decision on whether a claim was paid in error. OIG 

found that for the fiscal year 2004 CERT Program, AdvanceMed completed 

984 of the required 2,587 quality assurance reviews by the required date. 

To determine whether these quality assurance reviews ensured the 

reliability of the CERT Program claims review process, OIG randomly 

sampled 45 of the 2,587 claims selected for quality assurance reviews. Of 

these 45 claims, AdvanceMed had completed a quality assurance review on 

5 claims. OIG reported that the results of the 5 quality assurance reviews 

confirmed the results of the initial medical record reviews. Further, OIG 

reported that AdvanceMed stated that a backlog of medical reviews 

prevented the completion of the required quality assurance reviews within 

the prescribed time frame. In response to the OIG report on the fiscal year 

2004 CERT Program evaluation, CMS commented that with Lifecare 

Management Partners assuming responsibilities for medical record 

collection for the fiscal year 2006 Medicare error rate estimation, 

AdvanceMed’s workload would be reduced. As a result, CMS commented 

that this will free up the necessary resources for AdvanceMed to comply 

with the quality assurance requirements. Further, in its response to the 

OIG report, CMS commented that both AdvanceMed and Lifecare 

Management Partners are required to report to the agency on the results of 

the quality assurance activities conducted. According to OIG’s evaluation

of the fiscal year 2005 CERT Program, OIG found that AdvanceMed 

completed all of the required quality assurance reviews.43

 43See Department of Health and Human Services, Office of Inspector General, Oversight 

and Evaluation of the Fiscal Year 2005 Comprehensive Error Rate Testing Program, A03-05-00006 (Washington, D.C.: November 2005). 

Page 21 GAO-06-300 Medicare Error Rate
We found that the statistical methods used to estimate the error rates and 

standard errors by contractor type (carrier, DMERC, and FI) for the CERT 

Program were adequate. Based on our review of the computer

programming code that generated the error rate estimates and standard 

errors by the CERT Program subcontractor responsible for calculating the 

contractor-type error rates, The Lewin Group, we found that the statistical

methods were based on standard statistical principles and were used 

appropriately. For each contractor type, the stratified combined ratio 

estimation method was used to calculate the error rate by taking the 

difference between the overpaid dollars and the underpaid dollars divided 

by the total dollars paid by Medicare for FFS claims of each contractor 

type.44 The payment errors from the sample were then extrapolated to the

population for each contractor type to estimate total payment errors. 

Further, The Lewin Group used a standard statistical method to calculate 

the standard errors of each of the contractor-type error rates.45 This 

method is appropriate for obtaining the standard error of an estimate 

when the stratified combined ratio estimation method is used and is valid 

for large sample sizes, such as that used for the CERT Program. 

We found that the methodology used by CMS was adequate to produce a 

reliable estimate of the fiscal year 2004 Medicare error rate for the one 

contractor type (QIO) in the HPMP. We found the methodology adequate 

because the sample size was large enough to produce a reliable error rate 

estimate. Additionally, the sample was representative of the population. 

We found also that the methodology was adequate because the HPMP 

contractors responsible for collecting the medical records for the sampled 

claims, as well as for identifying and determining errors, had appropriate 

controls in place to ensure that established procedures were followed. 

Further, the statistical method that CMS used to calculate the contractortype error rate was valid. 

The sample size that CMS used for the HPMP, about 40,000 claims, was 

sufficiently large to produce a reliable estimate of the fiscal year 2004 

error rate for the QIO contractor type. Using a systematic sample, CMS 

selected 62 discharge claims per month for the District of Columbia, 

Statistical Methods 

CMS Methodology 

Adequate for 

Estimating the Error 

Rate in the HPMP 

Sampling Methods 

 44See, for example, Cochran, 164-166. 

45The Lewin Group used the Taylor series approximation method to calculate the standard 

errors. See, for example, Cochran, 319. 

Page 22 GAO-06-300 Medicare Error Rate
Puerto Rico, and each state except Alaska. CMS selected 42 claims per 

month for Alaska. The QIO contractor-type error rate was 3.6 percent with 

a relative precision of 5.6 percent. The relative precision for the QIO 

contractor-type error rate estimate is within acceptable statistical 

standards (a relative precision of no greater than 15 percent). 

For the QIO contractor-type error rate to be a reliable estimate, it was 

necessary that the sample of discharge claims from which the error rate 

was estimated be representative of the population from which it was 

drawn. CMS’s documentation stated that the HPMP used a systematic 

sample selection process with a random start, which is a generally 

accepted method of sampling that is designed to ensure that the sample 

drawn is representative of the population. Our review of the computer 

programming code that selected the sample, however, found that a 

random start was not used.46 To determine whether the HPMP sample was 

compromised by the lack of a random start and whether it represented the 

population from which it was drawn, we examined the OIG contractor’s 

comparison of the June 2003 sample to a re-created version of the 

June 2003 population file from which the sample was drawn.47 Based on 

our review, we found that the HPMP sample was representative of the 

population from which it was drawn in terms of average dollar amount per 

claim. 

While relative precision of the fiscal year 2004 QIO contractor-type error 

rate estimate was within acceptable statistical standards, relative 

precision of most of the state-specific QIO error rate estimates was not. 

(See app. II for state-specific QIO error rate information, including the 

error rate estimates and corresponding relative precision.) Only three 

states’ error rate estimates—Kentucky, Massachusetts, and New Mexico—

had relative precision of less than 15 percent. Additionally, there was wide 

 46A CMS official told us and provided documentation that beginning with the fiscal year 

2006 error rate estimation, the HPMP will move to a simple random sample in which all 

records are chosen at random within each state, thus eliminating the need for systematic 

sampling. Simple random sampling is also an accepted method of sampling to achieve a

sample that is representative of the population from which it was drawn. 

47It was not possible for the OIG contractor to obtain the exact June 2003 population file 

because the file is continuously updated and previous versions are not retained. We did not 

believe it was necessary to compare every month’s sample to the population from which it 

was drawn because the large size of the annual sample (approximately 40,000 claims) and 

population (approximately 11.5 million claims) would make the task too burdensome, and 

the fact that the sample was drawn in the same manner each month meant the results from 

one month should not differ significantly from the results from any other month. 

Page 23 GAO-06-300 Medicare Error Rate
variation in relative precision of the state-specific QIO error rate 

estimates. Relative precision of the state-specific QIO error rates ranged 

from 10.5 percent in Massachusetts to 83.3 percent in Mississippi. The 

differences in relative precision of these state-specific QIO error rate 

estimates indicate that the error rate estimate for the QIO that served 

Massachusetts was eight times more reliable than the error rate estimate 

for the QIO that served Mississippi. The variation in relative precision was 

due, in part, to the sampling methods used by CMS for the HPMP. CMS 

took an equal sample size for each state except Alaska, despite the fact 

that there was significant variation between states in the overall volume of

discharge claims and total payments. The number of discharges per state 

varied from a low of 15,166 in Wyoming to a high of 825,845 in Florida.48

Similarly, total dollars paid for acute-care inpatient hospital stays varied 

from less than $100 million in Wyoming to a high of $7.5 billion in 

California. 

Although in February 2006 a CMS official told us the agency has no plans 

to reallocate the HPMP sample, CMS could adopt a similar sampling 

strategy as it plans to do for the CERT Program. If future state samples 

were based on the volume of discharge claims or total payments per state 

and the relative precision of the state-specific QIO error rates, rather than

on the current basis of an equal allocation per state, relative precision 

would likely be improved for the state-specific QIO error rates in those 

states that were allocated a larger sample since relative precision 

improves as sample size increases. There would also likely be decreased 

variation in relative precision across all state-specific QIO error rates.49

These results could be achieved without increasing the overall sample size 

for the HPMP. 

In addition to issues with the wide variation of relative precision of the 

state-specific QIO error rate estimates, we also found large differences in

the average dollar amount per claim between the state-specific samples for 

some states and the respective state populations. These differences 

suggest that the samples drawn for more than half of the states were not 

representative of each state’s population. Based on our examination of the 

OIG contractor’s comparison of the state samples and the state 

 48The range reported here does not reflect the claims or total payment volume in Alaska 

since CMS takes a smaller sample from Alaska than from all other states, the District of 

Columbia, and Puerto Rico. 

49See, for example, Cochran, 96-99. 

Page 24 GAO-06-300 Medicare Error Rate
populations for June 2003, we found that the ratio of the average dollar 

amount per claim in a state’s sample to the average dollar amount per 

claim in a state’s population varied from 62 percent in Maryland to 

143 percent in Kentucky. Twelve states had a ratio above 110 percent, and

16 states had a ratio below 90 percent.50 It is still possible for the national

HPMP sample to be representative of the national HPMP population even 

if all of the state-specific samples are not representative of their state 

populations. The larger size of the HPMP sample overall mitigates the 

problems identified in the smaller state-specific samples. 

Based on our review of oversight work of the HPMP conducted by OIG,51

we found that the process CMS used for collecting medical records from 

providers was adequate. OIG selected 46 discharge claims that were 

sampled for the HPMP to determine if the CDACs, AdvanceMed and 

DynKePRO, followed established HPMP procedures for obtaining and 

reviewing medical records to identify payment errors. OIG found that the 

CDACs generally had appropriate controls in place to ensure that the 

medical records were obtained and reviewed according to established 

HPMP procedures. Of the 46 discharge claims reviewed, OIG found that in

two instances a required follow-up letter to the provider was not sent due 

to an error by a substitute CDAC employee. However, the medical records 

for these two discharge claims were obtained within 30 days of the 

original request, which resulted in no adverse effect on the error rate 

estimates. Overall, nonresponse for fiscal year 2004 represented 

approximately 5.1 percent of the total QIO contractor-type error rate of 

3.6 percent, or 0.2 percent of all discharge claims reviewed through the 

HPMP. 

The issue with providers not responding to requests for medical records 

was not as significant an issue for the HPMP as it was for the CERT 

Program. According to the CMS report on the fiscal year 2005 error rate, 

nonresponse was less problematic in the HPMP because of several factors, 

including the following: (1) providers were more likely to respond to 

requests from the HPMP since the average claim value was higher than the 

Medical Record Collection 

Process 

 50A ratio of 100 percent would mean that the average claim amount in the sample was equal 

to the average claim amount in the population. 

51See Department of Health and Human Services, Office of Inspector General, Oversight 

and Evaluation of the Fiscal Year 2004 Hospital Payment Monitoring Program, A-03-04-

00008 (Washington, D.C.: November 2004). 

Page 25 GAO-06-300 Medicare Error Rate
average claim value in the CERT Program;52 (2) providers were more 

familiar with the HPMP than with the CERT Program; and (3) providers 

were paid the cost of providing medical records by the HPMP, but not by 

the CERT Program.53

Based on our review of OIG’s fiscal year 2004 HPMP evaluation,54 we 

concluded that the CDACs (AdvanceMed and DynKePRO) generally had

processes in place to adequately identify and categorize claims paid in 

error in the HPMP for fiscal year 2004. OIG officials told us that they found 

the medical record reviewers, both admission necessity reviewers and 

DRG coding specialists, at the two CDACs met CMS’s qualifications for 

these positions.55 As part of its review of the fiscal year 2004 HPMP, OIG 

reviewed 46 discharge claims that were part of the sample for estimating 

the QIO contractor-type error rate. Based on that review, OIG reported 

that the CDACs generally had appropriate controls in place to ensure that

admission necessity and DRG validation reviews were performed in 

accordance with CMS established procedures and that the results of those 

reviews were adequately maintained, updated, and reported. 

As part of the internal HPMP quality control process, two activities were 

conducted regularly to ensure the reliability and accuracy of CDAC 

reviews both within each CDAC and across the two CDACs. Each CDAC 

randomly chose 30 claims per month to be reviewed by two of its medical 

record reviewers for intra-CDAC tests. Each CDAC compared the results 

of the two medical record reviews to determine the reliability of reviews 

within the CDACs and reported the results of the comparisons to CMS. 

The CDACs performed inter-CDAC tests to assess the reliability of the 

Identification and 

Categorization of Payment 

Errors 

 52For example, according to our analysis of data provided by the HPMP, the average claim 

value for claims reviewed for the fiscal year 2004 error rate for QIOs was approximately 

$7,500. According to our analysis of Medicare claims data from the Part B Extract 

Summary System Carrier Data File, the average claim value for carriers in 2003 was $32. 

53See Department of Health and Human Services, Centers for Medicare & Medicaid 

Services, Improper Medicare FFS Payments Long Report (Web Version) for 

November 2005. 

54See Department of Health and Human Services, Office of Inspector General, Oversight 

and Evaluation of the Fiscal Year 2004 Hospital Payment Monitoring Program. 

55According to OIG, CMS requires that CDACs employ admission necessity reviewers who 

are licensed practical nurses with utilization review experience. CMS requires that coding

specialists be registered health information administrators, registered health information 

technicians, or certified coding specialists. 

Page 26 GAO-06-300 Medicare Error Rate
reviews between the two CDACs. For these tests, an additional 30 claims 

were chosen at random per quarter by each of the CDACs for review by a 

medical records reviewer at the other CDAC. As part of its evaluation of

the fiscal year 2004 HPMP, OIG selected 45 claims that went through the 

intra-CDAC process and 42 claims that went through the inter-CDAC 

process to determine if these quality control activities ensured the 

reliability of the CDAC review process. OIG reported that the quality 

control reviews were generally operating effectively to ensure the 

reliability of the review process and the consistency of the error rate 

determination decisions.56

From the same evaluation of the fiscal year 2004 HPMP, OIG found that

the CMS contractor tasked with calculating the dollar amounts paid in 

error, Texas Medical Foundation, used a method that produced an amount 

of dollars in error that in some cases differed from what OIG found to be 

the amount of dollars in error. For claims identified by a QIO as having 

errors caused by changes in DRG codes, Texas Medical Foundation used a 

method that produced different dollar amounts in error than would have 

been produced if it had used the software that FIs used to pay the original 

discharge claims.57 The Texas Medical Foundation calculated a different 

amount in error for about 76 percent of 200 incorrectly coded claims that 

OIG reviewed. However, OIG reported that the differences did not have a 

significant effect on the QIO contractor-type error rate estimate. A CMS 

official told us that the agency has not invested in modifying the software 

for use by the Texas Medical Foundation for technical and financial 

reasons. For example, the software requires modifications using a specific

programming language for which CMS has limited personnel with the 

needed expertise. 

We verified the statistical methods CMS used to estimate the QIO 

contractor-type error rate and standard error in the HPMP by reviewing 

Statistical Methods 

 56See Department of Health and Human Services, Office of the Inspector General, 

Oversight and Evaluation of the Fiscal Year 2004 Hospital Payment Monitoring 

Program. 

57FIs, which are responsible for paying acute-care inpatient hospital claims, use a software 

program available on the CMS Web site, PRICER, to calculate the Medicare payment 

amount. The program calculates the Medicare payment amount using information supplied 

on the provider claim and current national and hospital-specific factors related to the

payment amount. CMS stated that the PRICER program does not consider all of the factors 

used by FIs when pricing acute-care inpatient hospital claims. 

Page 27 GAO-06-300 Medicare Error Rate
the computer programming code that produced this information. We 

found that the methods CMS used were adequate because they were based 

on standard statistical methods and were applied appropriately. To 

estimate the QIO contractor-type error rate, CMS weighted58 each statespecific QIO error rate according to that state’s share of the total Medicare 

FFS payments for acute-care inpatient hospital claims nationwide. This 

method is referred to as a stratified mean per unit estimation.59 Like the 

CERT Program, CMS used a standard statistical method to calculate the 

standard error of the estimate.60 In our review of the computer 

programming code that generated the QIO contractor-type error rate 

estimate, we found that CMS used annual instead of monthly weights in its 

estimate of the annual total dollars paid in error.61 It would have been 

more appropriate for CMS to have used monthly weights because the 

HPMP sample was drawn on a monthly, not an annual, basis. However, 

when we reviewed the OIG contractor’s comparison of the estimate of 

annual dollars paid in error using annual weights to what the estimate 

would have been had CMS used monthly weights, we concluded that the

use of annual weights did not significantly affect the QIO contractor-type 

error rate estimate. A CMS official told us and provided us with 

documentation that beginning with the HPMP’s fiscal year 2005 error rate

estimation process, monthly weights are being used. 

CMS appropriately combined the error rates under the CERT Program and 

the HPMP to estimate the fiscal year 2004 national Medicare error rate. 

CMS estimated the national Medicare error rate by averaging the error 

rates of the four contractor types (carrier, DMERC, FI, and QIO), weighted 

by each contractor type’s share of total Medicare FFS payments. Likewise, 

CMS calculated the standard error, or precision, of the national error rate 

based on the standard error of each of the four types of contractors’ error 

CMS Methodology 

Adequate for 

Estimating the 

National Error Rate 

 58To better ensure that data from a sample represent data from the population from which 

they are drawn, the sample data are often adjusted to reflect the probability of a specific 

data point, in this case an acute-care inpatient hospital discharge claim, being chosen. This 

process is called weighting. Sample weights reflect the different probabilities that each 

claim has of being chosen as part of the sample. The less likely a claim is to be selected, the

larger its sample weight. 

59See, for example, Hansen, Hurwitz, and Madow, 172-173. 

60CMS used a Taylor series linear approximation method. 

61To estimate the total annual dollars paid in error for QIOs, CMS projects the dollar

amounts found in error in the sample to the broad population. 

Page 28 GAO-06-300 Medicare Error Rate
rate estimates, weighted by each contractor type’s proportion of total 

Medicare FFS payments. The methods CMS used to calculate the national

error rate and the standard error were statistically valid, since the units of

measurement, which in this case were Medicare claims, of the four error 

rates that were combined were mutually exclusive (independent) among 

contractor types.62 Each contractor type consisted of multiple individual 

contractors. These contractors were independent in that one contractor’s 

estimated error rate or standard error did not affect the estimates of other

contractors, since the claims in the population and in the sample were not

overlapping among contractors. 

Since assuming responsibility for estimating the national Medicare error 

rate in fiscal year 2003, CMS has made changes to the methodology, which

have provided CMS with more detailed information about the error, 

thereby allowing the agency to better identify the underlying causes of 

error and implement corrective action plans to address them. For 

example, CMS significantly increased the size of the sample used to 

estimate the Medicare FFS claims paid in error. The increased sample size 

allowed the agency to estimate not only the error rate at the national level,

but also more detailed error rates at the contractor-type and contractorspecific levels. Further, CMS has made changes in the way it collects 

medical records from providers in an effort to reduce the rate of error 

caused by nonresponse and insufficient documentation. These changes 

may affect the error rate estimates and thus the comparability of the 

estimates over time. Consequently, users of the error rate information 

should exercise caution when making year-to-year comparisons. 

Our work focused on the methodology CMS used to estimate the national

Medicare error rate and contractor-type error rates for fiscal year 2004.

For these error rates, we found the methodology adequate for that year. 

Under CMS’s contracting reform initiative, there will be fewer individual 

contractors (carriers, DMERCs, and FIs). If CMS maintains the same 

overall sample size, the sample sizes of the remaining individual 

contractors would be increased. Reliability of the contractor-specific error 

rate estimates is likely to improve with the larger sample sizes. Until then, 

the wide variation in reliability of the contractor-specific error rate 

Concluding 

Observations 

 62Statistical theory demonstrates that combining the estimates based on independent

samples is a valid estimate of the aggregate of the samples. See, for example, Hansen,

Hurwitz, and Madow, 190. 

Page 29 GAO-06-300 Medicare Error Rate
estimates may preclude meaningful comparisons across individual 

contractors. 

We received written comments from HHS (see app. III.) In responding to 

our draft report, HHS noted that we found the CMS methodology adequate 

for estimating the fiscal year 2004 national Medicare FFS error rate. HHS 

also noted that CMS is continually committed to refining the processes to 

estimate, as well as lower, the level of improper payments in the Medicare

FFS program. 

In its comments, HHS noted improvement in the national Medicare error 

rate from fiscal years 2004 to 2005. The department attributed the decline 

in the error rate to marked improvement in the nonresponse (which CMS 

now calls “no documentation”) and the insufficient documentation error 

rates. Commenting on the adequacy of the fiscal year 2005 methodology 

was beyond the scope of our work; however, as we noted in the draft 

report, changes in the methodology may affect the estimation of the error 

rates and thus the comparability of these error rates over time. For 

example, we discussed in the draft report that CMS has made changes in

the way it collects medical records from providers in an effort to reduce 

the rate of error caused by nonresponse and insufficient documentation. 

These changes primarily affected HHS’s processes for calculating an 

annual error rate estimate for the Medicare FFS program. This may 

represent a refinement in the program’s estimation methodology rather 

than improved accountability over program dollars. 

The national Medicare error rates for fiscal years 2004 and 2005 provided

by HHS in its comments are not comparable to the error rates cited in this

report for fiscal years 2004 and 2005. HHS provided gross error rates, 

which were calculated using gross dollars paid in error. Gross dollars paid 

in error were calculated by adding dollars paid in error that were due to 

underpayments to those that were due to overpayments. As noted in the 

draft report, we reported net error rates. Net error rates were calculated 

using net dollars paid in error. Net dollars paid in error were calculated by

subtracting dollars paid in error that were due to underpayments from 

those that were due to overpayments. 

HHS also provided technical comments, which we have addressed as 

appropriate.

Agency Comments 

Page 30 GAO-06-300 Medicare Error Rate
We are sending copies of this report to the Secretary of Health and Human 

Services, the HHS Inspector General, the Administrator of CMS, and 

appropriate congressional committees. We will also provide copies to 

others upon request. In addition, the report is available at no charge on the 

GAO Web site at http://www.gao.gov. 

If you or your staff have any questions about this report, please contact me 

at (202) 512-7101 or steinwalda@gao.gov. Contact points for our Offices of 

Congressional Relations and Public Affairs may be found on the last page 

of this report. GAO staff who made major contributions to this report are 

listed in appendix IV. 

A. Bruce Steinwald 

Director, Health Care 

Page 31 GAO-06-300 Medicare Error Rate
Appendix I: Scope and Methodology Appendix I: Scope and Methodology 

We reviewed the following components of the Centers for Medicare & 

Medicaid Services’s (CMS) methodology for estimating the fiscal year 2004 

error rate: 

• Sampling methods, including sample size, sample selection, sample 

representation, and precision of the estimates. 

• The medical records collection process. 

• Identification and categorization of claims payment error, including the 

medical record review process and quality assurance reviews. 

• Statistical methods used to estimate the error rates and precision. 

To conduct our analysis of CMS’s sampling methods, we reviewed work 

performed by the Department of Health and Human Services (HHS) Office 

of Inspector General (OIG) contractor that assessed these methods and 

CMS documentation for the fiscal year 2004 Medicare error rate. For the 

Comprehensive Error Rate Testing (CERT) Program, we reviewed the 

program manual, which described the CERT Program sampling methods 

as well as CMS’s Medicare error rate reports for fiscal years 2003 and 

2004.

1

 For the Hospital Payment Monitoring Program (HPMP), we 

reviewed the program manual and the HPMP computer programming code 

that generated the sample to verify that the sample was taken in 

accordance with the procedures outlined in the manual. Additionally, we

reviewed the OIG contractor’s comparison of the June 2003 sample and a 

re-created version of the June 2003 sampling frame, or population, for the 

HPMP. It was not possible for the OIG contractor to obtain the exact 

June 2003 population file because the file is continuously updated and

previous versions are not retained. We did not believe it was necessary to

compare every month’s sample to the population from which it was drawn 

because of the large size of the sample (approximately 40,000 discharge 

claims) and population (approximately 11.5 million discharge claims), and 

the fact that the sample was drawn in the same manner each month. 

To conduct our analysis of CMS’s medical record collection and review 

processes and identification and categorization of payment errors, we 

relied primarily on reports published by OIG. Since 2003, OIG has 

conducted annual reviews of the CERT Program and the HPMP as part of 

 1

See Department of Health and Human Services, Centers for Medicare & Medicaid Services, 

Improper Medicare Fee-for-Service Payments Fiscal Year 2003 (Baltimore, Md.:

December 2003). See also Department of Health and Human Services, Centers for Medicare

& Medicaid Services, Improper Medicare Fee-for-Service Payments Report Fiscal Year

2004 (Baltimore, Md.: December 2004). 

Page 32 GAO-06-300 Medicare Error Rate
Appendix I: Scope and Methodology 

its review of work performed for HHS by contractors. These annual 

reviews examine whether the CERT Program and HPMP contractors have 

appropriate controls in place to ensure that the medical record reviews 

and quality assurance reviews were performed in accordance with 

established procedures. We reviewed OIG’s annual reviews of the CERT 

Program and the HPMP for fiscal year 2004.2

 Our analysis of provider 

nonresponse within the CERT Program relied on two OIG studies of CMS’s 

actions to reduce nonresponse implemented for the CERT Program for 

fiscal year 2004.3

 For the HPMP, we also reviewed four intra-Clinical Data 

Abstraction Center (CDAC) reports and two inter-CDAC reports, which 

were quality assurance reviews intended to assess the consistency of 

review decisions both within and across CDACs. 

To conduct our analysis of CMS’s statistical methods, we reviewed the 

OIG contractor’s computer programming code, which replicated CMS’s 

estimation of the error rates for carriers, durable medical equipment 

regional carriers (DMERC), and fiscal intermediaries (FI), as calculated by

the CERT Program subcontractor responsible for statistical analysis of the 

error rates for fiscal year 2004. We reviewed CMS’s computer 

programming code, which calculated the HPMP error rate for quality 

improvement organizations (QIO). In conducting these reviews of the 

computer programming codes for both the CERT Program and the HPMP, 

we verified that each code appropriately implemented a methodology that 

employed standard statistical principles and was used appropriately. 

To inform all aspects of our study, we interviewed OIG officials with 

oversight responsibility for the error rate estimation, OIG contractor staff 

who conducted the evaluation of the statistical methodology, CMS 

officials with programmatic responsibilities for the CERT Program and the 

 2

See Department of Health and Human Services, Office of Inspector General, Oversight 

and Evaluation of the Fiscal Year 2004 Comprehensive Error Rate Testing Program, A03-04-00007 (Washington, D.C.: November 2004). See also Department of Health and 

Human Services, Office of Inspector General, Oversight and Evaluation of the Fiscal Year

2004 Hospital Payment Monitoring Program, A-03-04-00008 (Washington, D.C.: 

November 2004). 

3

See Department of Health and Human Services, Office of Inspector General, Review of 

Corrective Actions to Improve the Comprehensive Error Rate Testing Process for 

Obtaining Medical Records, A-03-04-00005 (Washington, D.C.: June 2004). See also 

Department of Health and Human Services, Office of Inspector General, Review of 

Providers’ Responsiveness to Requests for Medical Records Under Comprehensive Error

Rate Testing Program, A-01-04-00517 (Washington, D.C.: September 2004). 

Page 33 GAO-06-300 Medicare Error Rate
Appendix I: Scope and Methodology 

HPMP, and staff of the CERT Program subcontractor for statistical 

analysis. 

We performed our work from April 2005 through March 2006 in

accordance with generally accepted government auditing standards. 

Page 34 GAO-06-300 Medicare Error Rate
Appendix II: Fiscal Year 2004 Error Rate 

Information by Contractor Type—Carriers, 

DMERCs, FIs, and QIOs 

Contractor

CMS 

targeted

sample 

sizea

Total Medicare feefor-service 

payments in fiscal 

year 2004b

(in dollars)

CMS estimated paid 

claims error rate 

(percentage)

CMS estimated

standard error

c 

(percentage)

Relative 

precisiond

(percentage)

Carrier

e

Triple S, Inc. PR/VI 2,004 $689,224,693 17.9 1.6 8.9

BCBS AR NM/OK/LA 2,004 2,293,083,008 12.7 1.2 9.4

NHIC CA 2,004 6,837,462,204 10.8 1.1 10.2

NHIC MA/ME/NH/VT 2,004 3,323,197,031 9.6 1.0 10.4

TrailBlazer TX 2,004 5,169,066,589 14.1 1.5 10.6

BCBS RI 2,004 232,458,933 13.5 1.5 11.1

GHI NY 2,004 372,383,958 14.3 1.6 11.2

Palmetto GBA OH/WV 2,004 4,226,979,481 10.6 1.2 11.3

First Coast Service Options FL 2,004 7,367,509,907 9.7 1.1 11.3

BCBS UT 2,004 287,713,078 10.2 1.2 11.8

First Coast Service Options CT 2,004 1,106,082,763 7.6 0.9 11.8

TrailBlazer MD/DC/DE/VA 2,004 4,158,091,772 9.2 1.1 12.0

BCBS AR AR/MO 2,004 2,292,786,396 10.6 1.4 13.2

HGSA PA 2,004 3,606,318,041 9.7 1.3 13.4

WPS WI/IL/MI/MN 2,004 8,126,245,486 11.1 1.6 14.4

Cahaba GBA AL/GA/MS 2,004 3,868,072,306 11.1 1.6 14.4

BCBS KS KS/NE/Kansas City 2,004 1,581,255,014 6.9 1.0 14.5

Palmetto SC 2,004 1,189,260,267 13.1 1.9 14.5

Noridian CO/ND/SD/WY/IA 2,004 1,865,892,800 9.5 1.4 14.7

Empire NY/NJ 2,004 7,268,107,083 10.8 1.6 14.8

Noridian AZ/HI/NV/AK/OR/WA 2,004 4,981,083,701 10.7 1.6 15.0

AdminaStar IN/KY 2,004 2,708,331,380 10.0 1.5 15.0

HealthNow NY 2,004 1,358,023,183 8.2 1.3 15.9

CIGNA ID/TN/NC 2,004 4,830,134,495 10.9 1.8 16.5

BCBS MT 2,004 193,432,019 5.3 0.9 17.0

All carriers 50,100 $79,932,195,591 10.7 0.4 3.7

DMERCf

TriCenturion Region Ag 2,004 $1,364,899,356 7.3 0.9 12.3

AdminaStar Federal-Region B 2,004 2,241,150,409 6.6 0.9 13.6

CIGNA-Region D 2,004 1,800,134,845 11.6 2.1 18.1

Palmetto GBA Region C 2,004 4,928,003,571 14.0 2.9 20.7

All DMERCs 8,016 $10,334,188,182 11.1 1.5 13.5

Appendix II: Fiscal Year 2004 Error Rate 

Information by Contractor Type—Carriers, 

DMERCs, FIs, and QIOs 

Page 35 GAO-06-300 Medicare Error Rate
Appendix II: Fiscal Year 2004 Error Rate 

Information by Contractor Type—Carriers, 

DMERCs, FIs, and QIOs 

Contractor

CMS 

targeted

sample 

sizea

Total Medicare feefor-service 

payments in fiscal 

year 2004b

(in dollars)

CMS estimated paid 

claims error rate 

(percentage)

CMS estimated

standard error

c 

(percentage)

Relative 

precisiond

(percentage)

FIh

UGS CA/HI/AS/GU/NMI 2,004 $6,003,110,480 20.4 2.1 10.3

Palmetto GBA SC 2,004 6,194,956,951 10.3 1.1 10.7

Mutual of Omaha 2,004 11,797,457,474 26.8 3.2 11.9

First Coast Service Options FL 2,004 2,472,517,626 23.0 2.9 12.6

TrailBlazer TX/CO/NM 2,004 4,556,783,468 14.1 2.0 14.2

Cahaba GBA AL 2,004 705,028,658 15.5 2.2 14.2

Trispan MS/LA/MO 2,004 1,675,273,646 15.8 2.6 16.5

BCBS RI 2,004 781,806,244 19.3 3.2 16.6

Empire NY/CT/DE 2,004 5,811,286,709 17.2 2.9 16.9

UGS VA/WV 2,004 1,449,840,434 16.6 2.8 16.9

COSVI PR/VI 2,004 158,822,429 11.9 2.1 17.6

Medicare Northwest OR/ID/UT 2,004 711,126,486 14.6 2.6 17.8

Palmetto GBA NC 2,004 3,190,067,317 16.7 3.0 18.0

Veritus PA 2,004 2,079,132,007 14.7 2.7 18.4

UGS MI/WI 2,004 4,952,538,415 13.5 2.5 18.5

Anthem NH/VT 2,004 641,811,111 9.0 1.7 18.9

BCBS WY 2,004 83,003,027 14.7 2.8 19.0

BCBS AZ 2,004 325,070,959 7.3 1.4 19.2

CareFirst MD/DC 2,004 2,159,553,514 25.3 4.9 19.4

Cahaba GBA IA 2,004 4,273,518,964 5.6 1.1 19.6

Noridian MN/ND 2,004 1,309,949,370 16.2 3.3 20.4

BCBS AR 2,004 481,442,284 26.1 5.5 21.1

BCBS NE 2,004 299,081,984 12.8 2.7 21.1

Anthem MA/ME 2,004 2,852,313,346 10.4 2.2 21.2

BCBS GA 2,004 2,105,558,870 6.9 1.5 21.7

BCBS KS 2,004 512,584,700 10.0 2.2 22.0

AdminaStar IN/IL/KY/OH 2,004 9,610,571,631 12.2 2.7 22.1

BCBS MT 2,004 229,695,544 6.8 1.7 25.0

BCBS OK 2,004 1,109,256,221 8.6 2.2 25.6

Riverbend TN/NJ 2,004 3,622,031,691 9.7 3.0 30.9

Premera WA/AK 2,004 1,004,968,329 7.3 3.1 42.5

All FIs 62,124 $83,160,159,889 15.7 0.7 4.5

Page 36 GAO-06-300 Medicare Error Rate
Appendix II: Fiscal Year 2004 Error Rate 

Information by Contractor Type—Carriers, 

DMERCs, FIs, and QIOs 

Contractor

CMS 

targeted

sample 

sizea

Total Medicare feefor-service 

payments in fiscal 

year 2004b

(in dollars)

CMS estimated paid 

claims error rate 

(percentage)

CMS estimated

standard error

c 

(percentage)

Relative 

precisiond

(percentage)

QIO by statei

Massachusetts 744 $2,135,744,081 8.6 0.90 10.5

Kentucky 744 1,482,350,516 9.3 1.10 11.8

New Mexico 744 324,592,033 6.1 0.90 14.8

Maine 744 417,801,848 4.6 0.70 15.2

Louisiana 744 1,388,303,707 5.8 0.90 15.5

Arkansas 744 851,144,822 4.5 0.70 15.6

Illinois 744 3,864,432,432 4.4 0.70 15.9

Delaware 744 271,799,810 4.2 0.70 16.7

Maryland 744 2,067,187,033 3.0 0.50 16.7

Iowa 744 812,196,278 3.6 0.60 16.7

Indiana 744 1,784,654,000 4.1 0.70 17.1

Nevada 744 402,837,978 4.6 0.80 17.4

New Hampshire 744 328,223,324 3.4 0.60 17.6

Florida 744 5,696,783,961 5.1 0.90 17.6

Michigan 744 3,467,564,282 3.9 0.70 17.9

West Virginia 744 729,042,409 4.4 0.80 18.2

Vermont 744 164,700,697 3.3 0.60 18.2

South Dakota 744 232,787,316 3.8 0.70 18.4

Ohio 744 3,469,584,344 3.2 0.60 18.8

Alabama 744 1,603,881,531 3.2 0.60 18.8

Rhode Island 744 269,904,786 4.2 0.80 19.0

Virginia 744 1,933,408,829 3.5 0.70 20.0

Oklahoma 744 964,748,057 3.5 0.70 20.0

Alaska 504 100,985,029 3.5 0.70 20.0

North Dakota 744 216,246,500 2.0 0.40 20.0

South Carolina 744 1,408,487,704 5.4 1.10 20.4

New Jersey 744 3,595,399,138 2.9 0.60 20.7

Puerto Rico 744 376,450,167 4.8 1.00 20.8

Utah 744 389,527,711 3.8 0.80 21.1

Connecticut 744 1,295,269,906 3.2 0.70 21.9

New York 744 6,522,717,692 2.6 0.60 23.1

Idaho 744 237,198,385 2.6 0.60 23.1

Texas 744 5,573,613,357 4.2 1.00 23.8

Page 37 GAO-06-300 Medicare Error Rate
Appendix II: Fiscal Year 2004 Error Rate 

Information by Contractor Type—Carriers, 

DMERCs, FIs, and QIOs 

Contractor

CMS 

targeted

sample 

sizea

Total Medicare feefor-service 

payments in fiscal 

year 2004b

(in dollars)

CMS estimated paid 

claims error rate 

(percentage)

CMS estimated

standard error

c 

(percentage)

Relative 

precisiond

(percentage)

North Carolina 744 2,720,223,476 2.1 0.50 23.8

Washington 744 1,253,681,476 2.1 0.50 23.8

Oregon 744 689,865,040 2.5 0.60 24.0

Pennsylvania 744 4,290,842,680 2.5 0.70 28.0

Nebraska 744 500,351,357 1.4 0.40 28.6

District of Columbia 744 393,305,231 1.3 0.40 30.8

Kansas 744 762,382,857 2.8 0.90 32.1

Georgia 744 2,215,263,714 2.1 0.70 33.3

Arizona 744 1,081,388,500 2.4 0.80 33.3

Tennessee 744 2,093,513,706 1.7 0.60 35.3

Missouri 744 1,935,671,182 1.1 0.40 36.4

Wyoming 744 99,863,364 1.1 0.40 36.4

California 744 7,517,783,935 4.6 1.70 37.0

Wisconsin 744 1,575,519,000 1.0 0.40 40.0

Minnesota 744 1,412,860,400 1.0 0.50 50.0

Colorado 744 703,166,846 1.3 0.70 53.8

Hawaii 744 203,010,800 0.5 0.30 60.0

Montana 744 226,885,429 0.7 0.50 71.4

Mississippi 744 884,792,083 1.2 1.00 83.3

All QIOs 38,448 $84,939,940,736 3.6 0.20 5.6

Source: GAO analysis of CMS data. 

Note: This table reflects net paid claims error rates.

a

For carriers, DMERCs, and FIs, sample size was the targeted number of claims drawn for the fiscal 

year 2004 error rate estimates for each contractor. While CMS selected an equal sample from each 

of these contractors, the final sample sizes varied among contractors. Some selected claims were 

excluded from the final sample because the claims were missing information, such as dates of 

service and provider or patient information. For QIOs, the targeted sample size was the actual sample

size. 

b

We calculated total Medicare fee-for-service payments by dividing the projected improper payment 

by the paid claims error rate. According to the CMS fiscal year 2004 error rate report, CMS did not 

adjust projected improper payments data to exclude beneficiary co-payments, deductibles, and 

reductions to recover previous overpayments. This means that the improper payment amounts 

appear larger than they would otherwise. However, error rates are unaffected. 

c

Standard error is a measure of variation around the estimate, in this case the error rate. 

d

Relative precision equals the contractor’s standard error divided by the contractor’s paid claims error 

rate. 

e

Carriers are health insurers and pay claims submitted by physicians, diagnostic laboratories and 

facilities, and ambulance service providers.

Page 38 GAO-06-300 Medicare Error Rate
Appendix II: Fiscal Year 2004 Error Rate 

Information by Contractor Type—Carriers, 

DMERCs, FIs, and QIOs 

f

DMERCs are health insurers and pay claims submitted by durable medical equipment suppliers. 

g

For the fiscal year 2004 error rate,TriCenturion, a program safeguard contractor, was responsible for 

medical review in one of the four DMERC regions. Program safeguard contractors are Medicare 

contractors that conduct activities to address or prevent improper payments. As such, it was 

TriCenturion, not the DMERC, which was responsible for lowering the error rates in its region. 

h

FIs are almost exclusively health insurers and pay claims submitted by home health agencies, nonprospective payment system (PPS) hospitals, hospital outpatient departments, skilled nursing 

facilities, and hospices. PPS is a reimbursement method used by Medicare where the payment is

made based on a predetermined rate and is unaffected by the provider’s actual costs. 

i

QIOs (formally known as peer review organizations) are responsible for ascertaining the accuracy of 

coding and payment of paid Medicare FFS claims for acute care inpatient hospital stays—generally 

those that are covered by PPS—for Medicare beneficiaries in all 50 states, the District of Columbia, 

and Puerto Rico. Unlike carriers, DMERCs, and FIs, however, QIOs do not process and pay claims. 

These activities are conducted by FIs. 

Page 39 GAO-06-300 Medicare Error Rate
Appendix III: Comments from the Department of Health Appendix III: Comments from the and Human Services 

Department of Health and Human Services 

Page 40 GAO-06-300 Medicare Error Rate
Appendix III: Comments from the Department of Health 

and Human Services 

Page 41 GAO-06-300 Medicare Error Rate
Appendix IV: GAO Contact and Staff Acknowledgments 

Page 42 GAO-06-300

Appendix IV: GAO Contact and Staff 

Acknowledgments 

GAO Contact A. Bruce Steinwald, (202) 512-7101 or steinwalda@gao.gov 

In addition to the contact named above, Debra Draper, Assistant Director; 

Lori Achman; Jennie Apter; Dae Park; and Ann Tynan made key 

contributions to this report. 

Medicare Error Rate

Acknowledgments 

(290355) 
GAO’s Mission The Government Accountability Office, the audit, evaluation and 

investigative arm of Congress, exists to support Congress in meeting its 

constitutional responsibilities and to help improve the performance and 

accountability of the federal government for the American people. GAO 

examines the use of public funds; evaluates federal programs and policies; 

and provides analyses, recommendations, and other assistance to help

Congress make informed oversight, policy, and funding decisions. GAO’s 

commitment to good government is reflected in its core values of 

accountability, integrity, and reliability. 

The fastest and easiest way to obtain copies of GAO documents at no cost

is through GAO’s Web site (www.gao.gov). Each weekday, GAO posts 

newly released reports, testimony, and correspondence on its Web site. To

have GAO e-mail you a list of newly posted products every afternoon, go 

to www.gao.gov and select “Subscribe to Updates.” 

The first copy of each printed report is free. Additional copies are $2 each. 

A check or money order should be made out to the Superintendent of 

Documents. GAO also accepts VISA and Mastercard. Orders for 100 or 

more copies mailed to a single address are discounted 25 percent. Orders 

should be sent to: 

U.S. Government Accountability Office 

441 G Street NW, Room LM 

Washington, D.C. 20548 

To order by Phone: Voice: (202) 512-6000 

TDD: (202) 512-2537 

Fax: (202) 512-6061 

Contact: 

Web site: www.gao.gov/fraudnet/fraudnet.htm

E-mail: fraudnet@gao.gov

Automated answering system: (800) 424-5454 or (202) 512-7470 

Gloria Jarmon, Managing Director, JarmonG@gao.gov (202) 512-4400 

U.S. Government Accountability Office, 441 G Street NW, Room 7125 

Washington, D.C. 20548 

Paul Anderson, Managing Director, AndersonP1@gao.gov (202) 512-4800

U.S. Government Accountability Office, 441 G Street NW, Room 7149 

Washington, D.C. 20548 

Obtaining Copies of 

GAO Reports and 

Testimony

Order by Mail or Phone 

To Report Fraud,

Waste, and Abuse in 

Federal Programs 

Congressional 

Relations 

Public Affairs 

PRINTED ON RECYCLED PAPER