Department of Health and Human Services


OFFICE OF 

INSPECTOR GENERAL 

COST AND PERFORMANCE OF 


MEDICARE’S 2005 CHEMOTHERAPY


DEMONSTRATION PROJECT


Daniel R. Levinson


Inspector General 


August 2006


OEI-09-05-00171

Office of Inspector General 
 http://oig.hhs.gov 

The mission of the Office of Inspector General (OIG), as mandated by Public Law 95-452, 

as amended, is to protect the integrity of the Department of Health and Human Services 

(HHS) programs, as well as the health and welfare of beneficiaries served by those 

programs. This statutory mission is carried out through a nationwide network of audits, 

investigations, and inspections conducted by the following operating components: 

Office of Audit Services 

The Office of Audit Services (OAS) provides all auditing services for HHS, either by 

conducting audits with its own audit resources or by overseeing audit work done by others. 

Audits examine the performance of HHS programs and/or its grantees and contractors 

in carrying out their respective responsibilities and are intended to provide independent 

assessments of HHS programs and operations. These assessments help reduce waste, 

abuse, and mismanagement and promote economy and efficiency throughout HHS. 

Office of Evaluation and Inspections 

The Office of Evaluation and Inspections (OEI) conducts national evaluations to provide 

HHS, Congress, and the public with timely, useful, and reliable information on 

significant issues. Specifically, these evaluations focus on preventing fraud, waste, or 

abuse and promoting economy, efficiency, and effectiveness in departmental programs. 

To promote impact, the reports also present practical recommendations for improving 

program operations. 

Office of Investigations 

The Office of Investigations (OI) conducts criminal, civil, and administrative 

investigations of allegations of wrongdoing in HHS programs or to HHS beneficiaries 

and of unjust enrichment by providers. The investigative efforts of OI lead to criminal 

convictions, administrative sanctions, or civil monetary penalties. 

Office of Counsel to the Inspector General 

The Office of Counsel to the Inspector General (OCIG) provides general legal services to 

OIG, rendering advice and opinions on HHS programs and operations and providing all 

legal support in OIG's internal operations. OCIG imposes program exclusions and civil 

monetary penalties on health care providers and litigates those actions within HHS. 

OCIG also represents OIG in the global settlement of cases arising under the Civil False 

Claims Act, develops and monitors corporate integrity agreements, develops compliance 

program guidances, renders advisory opinions on OIG sanctions to the health care 

community, and issues fraud alerts and other industry guidance. 
Δ EXECUTIVE SUMMAR Y 

OBJECTIVE 

To assess the cost and the overall performance of Medicare’s 2005 

chemotherapy demonstration project. 

BACKGROUND 

On January 1, 2005, the Centers for Medicare & Medicaid Services 

(CMS) initiated a 1-year Medicare “Demonstration of Improved Quality 

of Care for Cancer Patients Undergoing Chemotherapy.” According to 

CMS, the purpose of the demonstration was to “ . . . assess and provide 

new support for the quality of care for cancer patients undergoing 

chemotherapy. . . . ” CMS provided a $130 allowance each time a 

chemotherapy practitioner reported to Medicare, via the claims system, 

an assessment of a patient’s levels of nausea and vomiting, pain, and 

fatigue—three conditions commonly experienced as symptoms of cancer 

or side effects of cancer treatment. Beneficiaries were responsible for 

the usual 20-percent coinsurance ($26 in this case) each time the codes 

were billed. CMS estimated that Medicare and beneficiary 

expenditures for the demonstration would total $300 million. The 

demonstration has continued in 2006 in a significantly modified form. 

On September 8, 2005, the Office of Inspector General provided Senator 

Charles E. Grassley, per his request, a preliminary assessment of the 

demonstration’s costs and the adequacy of its data collection methods. 

We estimated that Medicare and its beneficiaries would pay 

$270 million for the demonstration in 2005 and noted that 

approximately 3 percent of demonstration payments were for services 

that did not meet program rules. We also reported that participants 

administered the demonstration assessments inconsistently and that 

CMS was not collecting information on the interventions used to 

manage patients’ symptoms. Finally, we noted that the demonstration 

allowance was disproportionate to the amount of effort involved on the 

part of the practitioners and that assessing symptoms was already part 

of routine cancer care. 

To evaluate the cost of the demonstration, we analyzed demonstration 

claims received by Medicare through the end of 2005. To produce our 

overall assessment of the demonstration project, we conducted 

interviews with CMS staff and reviewed e-mails, meeting notes, and 

other workpapers CMS sent in response to our request for the entire 

project file and any supplemental documents related to the 

 OE I-09-05-00 171 MEDI C ARE ’ S 2 0 05 CHEMOTHERAP Y DE M O N S T R AT I O N P R O J E C T i 
EXE CUTIVE S U MMAR Y 

demonstration. We also interviewed staff at four oncology practices to 

learn how they implemented the demonstration. 

FINDINGS 

Medicare and its beneficiaries will spend approximately $275 million 

on the 2005 chemotherapy demonstration. Medicare is on track to 

allow approximately $275 million for the demonstration; beneficiary 

liability will total approximately $55 million. Approximately 90 percent 

of eligible practitioners took part in the demonstration, and 85 percent 

billed the demonstration codes at least half the time. The median 

amount allowed per physician was approximately $23,000, but the top 

10 billers were allowed more than $270,000 each. 

Seven percent of demonstration claims did not comply with 

program rules or were paid incorrectly, resulting in $17 million in net 

overpayments. Claims that Medicare carriers paid in error included 

duplicate billings, paying for the demonstration without concurrent 

chemotherapy, and paying when the beneficiary did not have a cancer 

diagnosis. 

CMS did not sufficiently define the parameters of the demonstration, 

leading to inconsistent data collection and incomplete and 

unreliable data. Because CMS did not mandate a specific approach to 

collecting the demonstration data, oncology practices implemented 

inconsistent data collection procedures. We identified numerous 

anomalies and gaps in the data and collection methods that 

demonstrate the unreliability and undermine the usefulness of the data. 

CONCLUSION 

We previously reported that physicians who participated in the 

demonstration used inconsistent methods and timeframes to assess 

beneficiaries’ symptoms. Our analysis of a full year of demonstration 

data revealed numerous inconsistencies and anomalies that raise 

further issues concerning the integrity of the data. Based on these 

concerns, we conclude that the demonstration data are unreliable. 

 OE I-09-05-00 171 MEDI C ARE ’ S 2 0 05 CHEMOTHERAP Y DE M O N S T R AT I O N P R O J E C T ii 
Δ T ABLE OF CONTENTS 

EXECUTIVE SUMMARY ..................................... i

INTR O D U C T I ON ........................................... 1

FINDI NGS ................................................. 7

Demonstration to cost $275 million ......................... 7

Inappropriate payments total $17 million .................... 9

Demonstration data inconsistent and unreliable ............. 10

CON C L U SION ............................................ 13

ACKN O W LEDGMENTS .................................... 14
Δ INTRODU C TION 

OBJECTIVE 

To assess the cost and the overall performance of Medicare’s 2005 

chemotherapy demonstration project. 

BACKGROUND 

2005 Chemotherapy Demonstration Project 

On January 1, 2005, the Centers for Medicare & Medicaid Services 

(CMS) initiated a 1-year Medicare “Demonstration of Improved Quality 

of Care for Cancer Patients Undergoing Chemotherapy.” CMS first 

publicly announced the demonstration in a November 15, 2004, final 

rule that implemented a revised payment methodology for drugs, which 

was mandated by the Medicare Prescription Drug, Improvement, and 

Modernization Act of 2003 (commonly known as the Medicare 

Modernization Act).1 

CMS cites section 402(a)(1)(B) of the Social Security Act Amendments of 

1967 (the Amendments) as the authority under which the 

demonstration was operated. This section of the Amendments permits 

the Secretary of the Department of Health and Human Services to 

conduct demonstrations “ . . . to determine whether payments for 

services other than those for which payment may be made under 

[Medicare] (and which are incidental to services for which payment may 

be made under [Medicare]) would, in the judgment of the Secretary, 

result in more economical provision and more effective utilization of 

services for which payment may be made. . . . ” According to CMS, the 

purpose of the demonstration project was to “ . . . assess and provide 

new support for the quality of care for cancer patients undergoing 

chemotherapy. . . . ”2 

Under the demonstration, CMS provided a $130 allowance each time a 

chemotherapy practitioner reported to Medicare an assessment of a 

patient’s levels of nausea and vomiting, pain, and fatigue—three 

conditions commonly experienced as symptoms of cancer or side effects 

of cancer treatment. To participate in the demonstration, physicians 

1 69 FR 66236. 

2 CMS. “Fact Sheet: Demonstration of Improved Quality of Care for Cancer Patients 

Undergoing Chemotherapy.” November 1, 2004. Available online at 

http://www.cms.hhs.gov/media/press/release.asp?Counter=1245. Accessed on 

August 26, 2005. 

 OE I-09-05-00 171 MEDI C ARE ’ S 2 0 05 CHEMOTHERAP Y DE M O N S T R AT I O N P R O J E C T 1 
INTROD UCT ION 

submitted claims to Medicare that included special billing codes that 

CMS created for the demonstration.

3 These codes described, on a 

four-point scale (“not at all,” “a little,” “quite a bit,” or “very much”), the 

degree to which the beneficiary had been affected by the three 

conditions.4 The demonstration required no formal enrollment; 

however, it was limited to physicians practicing in an office setting and 

to beneficiary visits at which the physician administered chemotherapy 

via infusion or push for a cancer diagnosis.5 

CMS estimated that Medicare and its beneficiaries would spend 

$300 million for the demonstration in 2005. CMS did not waive the 

Medicare Part B coinsurance requirement for this demonstration. 

Therefore, beneficiaries were liable for the usual 20-percent coinsurance 

($26 in this case) each time their physician billed the demonstration 

codes. 

CMS regularly carries out demonstration and other research projects, 

but the chemotherapy demonstration was much larger in scope than is 

typical. The $300 million in estimated expenditures greatly exceeds the 

budgets of other CMS research projects; the next most expensive 

research project listed in CMS’s Active Projects Report is a $60 million 

project with a performance period of 8 years.6 Also, while 

demonstration projects usually are limited geographically to a small 

number of sites, the 2005 chemotherapy demonstration was open to all 

physicians and beneficiaries nationwide. 

3 Although nonphysician practitioners were eligible to participate in the demonstration, 

physicians submitted more than 99 percent of paid demonstration claims. Throughout this 

report we use the term “physician” to mean any provider of Medicare Part B services. 

4 CMS based the demonstration’s four-point scale on the Rotterdam Symptom Checklist, 

an instrument developed as a patient self-assessment tool for measuring the quality of life 

of cancer patients. A discussion of the Rotterdam scale and how it should be used can be 

found in Northern Centre for Healthcare Research, University of Groningen, The 

Netherlands, “Measuring the Quality of Life of Cancer Patients with the Rotterdam 

Symptom Checklist (RCSL): A Manual.” 1996. 

5 In infusion chemotherapy, the physician dilutes the chemotherapy drug in a bag of fluid 

and then administers this solution into a vein over a specified period of time. In the push 

technique, the physician uses a syringe to administer the chemotherapy drug directly into a 

vein. 

6 The Active Project Report inventories research, demonstration, and evaluation projects 

undertaken by CMS staff or by external entities with CMS support. 

 OE I-09-05-00 171 MEDI C ARE ’ S 2 0 05 CHEMOTHERAP Y DE M O N S T R AT I O N P R O J E C T 2 
INTROD UCT ION 

Pursuant to CMS Change Request7 (CR) 3670, dated December 30, 

2004, demonstration claims were payable only if the following 

conditions were met: 

• The physician submitted at least one code each for 

nausea/vomiting, pain, and fatigue. (If the physician submitted 

more than one code for a symptom category, carriers were 

instructed to allow only the highest-level code.) 

• The physician billed the demonstration codes for the same date 

of service as an allowed code for chemotherapy via infusion or 

push. 

• The date of service of the demonstration codes (and, therefore, 

the chemotherapy administration code) was in 2005. 

• The place of service referenced for both the demonstration codes 

and the chemotherapy administration was “office.” 

• The diagnosis code reported and referenced was for cancer. 

According to CMS, no further official instructions were provided to 

carriers about the demonstration. 

CMS has contracted with Mathematica Policy Research, a professional 

evaluation firm, to analyze the demonstration data. According to CMS, 

beneficiaries at 80 percent of demonstration assessments reported 

having no nausea/vomiting and those at 70 percent reported having no 

pain. Fatigue was more variable, with beneficiaries at approximately 

30 percent of assessments reporting none, at 40 percent a little, at 20 

percent quite a bit, and at the rest very much. Because the levels of 

nausea/vomiting and pain reported were consistently low, the contract 

project officer stated that CMS will focus on beneficiaries’ reported 

levels of fatigue in further analyses. Among other things, CMS plans to 

correlate the demonstration data with other claims data, such as 

hospital admissions and emergency department visits. 

2006 Demonstration Project 

In a final rule dated November 21, 2005, CMS announced that the 

demonstration would continue in 2006 in a revised and less costly 

version.

8 While both versions collect data via the submission of special 

billing codes, the codes for the 2006 demonstration must be submitted 

7 CMS uses Change Requests to instruct carriers to implement Medicare policy changes. 8 70 FR 70272. 

 OE I-09-05-00 171 MEDI C ARE ’ S 2 0 05 CHEMOTHERAP Y DE M O N S T R AT I O N P R O J E C T 3 
INTROD UCT ION 

with particular evaluation and management (E & M) visits, not with 

chemotherapy administration. The 2006 demonstration requires that 

participants (1) submit special codes that correspond to the reason for 

the E & M visit, (2) indicate whether the participant is following clinical 

guidelines, and (3) describe the current disease state. Medicare will 

allow $23 per encounter, and the usual Part B coinsurance requirement 

applies, meaning beneficiaries are liable for $4.60 per encounter. 

Participation in the 2006 demonstration is limited to hematologists, 

oncologists, hematologist/oncologists, and gynecological oncologists, and 

participants may only bill the demonstration for beneficiaries who have 

specific cancer diagnoses that appear on a list enumerated by CMS. 

The participant (or staff) provides all information used to bill the 2006 

demonstration; other than the physician’s assessment of the current 

disease state, no beneficiary data are collected. 

Previous Work 

On August 12, 2005, Senator Charles E. Grassley, Chairman of the 

Senate Committee on Finance, sent a letter to Inspector General 

Daniel R. Levinson expressing concerns about the cost of the 

chemotherapy demonstration project and whether the data CMS was 

collecting would contribute to the goal of improving care for cancer 

patients. Senator Grassley requested that the Office of Inspector 

General (OIG) provide an assessment of the cost of the demonstration 

and the adequacy of its data collection methods. 

OIG provided an interim assessment to Senator Grassley on 

September 8, 2005.9 In that assessment, we estimated that Medicare 

and its beneficiaries would pay approximately $270 million to 

physicians and that two-thirds of eligible physicians would participate. 

Some participants had received large payments under the 

demonstration—as much as $320,000 in the first 6 months of the 

demonstration in one case. Approximately 3 percent of the 

demonstration claims did not meet the requirements of CR 3670, 

resulting in $3.6 million in inappropriate expenditures. We also 

expressed concerns about the reliability and usefulness of the 

demonstration data because of inconsistency in the ways participants 

administered the demonstration assessments and because CMS was not 

collecting information on the interventions physicians used to manage 

9 The letter is available on the Web site of the Senate Committee on Finance at 

http://finance.senate.gov/press/Gpress/2005/prg101305lett.pdf. 

 OE I-09-05-00 171 MEDI C ARE ’ S 2 0 05 CHEMOTHERAP Y DE M O N S T R AT I O N P R O J E C T 4 
INTROD UCT ION 

patients’ symptoms. Finally, we noted that the demonstration 

allowance was disproportionate to the amount of effort involved on the 

part of the participant and that assessing symptoms was already part of 

routine cancer care. 

In its January 2006 report to Congress, the Medicare Payment Advisory 

Commission (MedPAC) questioned the demonstration’s validity and 

methodology. MedPAC expressed concern that the demonstration: 

• did not have appropriate controls, 

• was announced and implemented without any period for 

comments by clinicians and researchers, 

• did not impose a uniform data collection process, 

• did not gather data over a consistent timeframe, and 

• did not collect data on interventions used to alleviate symptoms. 

MedPAC also reported that while all oncology practices it visited (in five 

States) were participating in the demonstration, most oncologists did 

not believe it would lead to quality improvements for patients or 

produce any useful research findings. MedPAC concluded that the 

demonstration was designed to increase payments to oncologists and 

stated in a recommendation to Congress that “ . . . [t]he Secretary 

should use his demonstration authority to test innovations in the 

delivery and quality of health care. Demonstrations should not be used 

as a mechanism to increase payments. . . .”10 

METHODOLOGY 

To evaluate the cost of the demonstration and its financial impact on 

Medicare and its beneficiaries, we analyzed demonstration claims 

received through the end of 2005. These data, which we received in 

February 2006, represent approximately 87 percent of the services 

rendered in 2005. Because Medicare processes claims submitted up to 

1 year after the year in which services are rendered, complete data on 

the demonstration will not be available before December 31, 2006. 

10 Medicare Payment Advisory Commission. "Report to the Congress: Effects of 

Medicare Payment Changes on Oncology Services.” January 2006. 

 OE I-09-05-00 171 MEDI C ARE ’ S 2 0 05 CHEMOTHERAP Y DE M O N S T R AT I O N P R O J E C T 5 
INTROD UCT ION 

We combined the qualitative data we collected for our September 

interim report with data from additional sources to produce our overall 

assessment of the demonstration project. Previously, we had visited 

four oncology practices that participated in the demonstration and 

interviewed staff at each about the methods they used to collect the 

data. We also interviewed cancer researchers and spoke with CMS staff 

about the project. For this report, we conducted additional interviews 

with CMS staff and analyzed the 2005 claims data to identify patterns 

and issues that warrant further evaluation. We also reviewed work 

papers that CMS sent in response to our request for the entire project 

file and any supplemental documents related to the demonstration. The 

work papers contained internal e-mails, meeting notes, and other work 

papers from the development and implementation of the demonstration, 

as well as publicly released information about the demonstration. 

 OE I-09-05-00 171 MEDI C ARE ’ S 2 0 05 CHEMOTHERAP Y DE M O N S T R AT I O N P R O J E C T 6 
Δ FINDINGS 

Medicare and its beneficiaries will spend 

approximately $275 million on the 2005 

chemotherapy demonstration 

Assuming that the rate of claims 

submission in the fourth quarter of 

2005 is consistent with that of the 

fourth quarter of prior years, 

Medicare will allow approximately $275 million for the 2005 

chemotherapy demonstration.11 Because beneficiaries are responsible 

for 20-percent coinsurance on demonstration claims, beneficiary liability 

for the demonstration project will total approximately $55 million. 

Participation in the demonstration was high: approximately 90 percent 

of eligible physicians took part. Based on a chi-square test, physician 

practices that had 11 or more beneficiaries receiving chemotherapy for a 

FIGURE 1: 

Percentage of 

eligible physicians 

who billed the 

demonstration 

codes at a given 

frequency 

(e.g, 11 percent of 

eligible physicians 

always billed the 

demonstration 

codes) 

At least 

half 

74% 

Always 

11% 

Never 

10% Less 

than half 

5% 

cancer diagnosis were more 

likely to participate than 

those with fewer eligible 

patients. Approximately 

11 percent of eligible 

physicians billed and were 

paid for the demonstration 

codes at every eligible 

chemotherapy visit, and 

85 percent billed the 

demonstration codes at 

least half the time. (See 

Figure 1.) Overall, 

Medicare paid for a 

demonstration assessment 

at 88 percent of eligible 

chemotherapy visits. 

Source: OIG analysis of Medicare claims data, 2006. 

The demonstration provided substantial reimbursement to some physicians 

Approximately 7,500 physicians participated in the chemotherapy 

demonstration. The median amount allowed per physician was 

approximately $23,000; however, some physicians were allowed 

11 Medicare processes claims submitted up to 1 year after the end of the year in which 

the service was rendered. As of December 31, 2005, Medicare carriers had allowed 

1.8 million assessments performed under the demonstration project, totaling $239 million in 

Medicare and beneficiary payments. CMS projects that spending on the demonstration will 

total $255 million based on claims received through July 31, 2006. 

 OE I-09-05-00 171 MEDI C ARE ’ S 2 0 05 CHEMOTHERAP Y DE M O N S T R AT I O N P R O J E C T 7 
F I NDING S 

significantly more than the median. Figure 2 shows the distribution of 

demonstration payments to individual physicians. Medicare allowed 

more than $100,000 each for 308 individual physicians, and the top 10 

billers were allowed more than $270,000 each. The top biller, an 

oncologist in Florida, was allowed $625,603 for the demonstration codes. 

The physician with the second-highest allowed amount for the 

demonstration, an oncologist in Kansas, was allowed $507,563. These 

high billers represent practices that performed substantial amounts of 

push and infusion chemotherapy and, therefore, had many 

opportunities to bill for the demonstration. 

FIGURE 2: 

Distribution of 

demonstration 

allowances 

Source: OIG analysis of Medicare claims data, 2006. 

33 

275 

1,310 

1,958 

1,455 

2,535 

0 

500 

1,000 

1,500 

2,000 

2,500 

3,000 

Less than 

$10,000 

$10,000 to 

$24,999 

$25,000 to 

$49,999 

$50,000 to 

$99,999 

$100,000 to 

$199,999 

$200,000 or 

more 

Total allowed for demonstration codes 

Number of providers 

On average, participants in the demonstration were allowed $529 per 

patient; for some, the amount was much higher. An oncologist in 

California had the highest mean allowed amount per patient, $5,214, of 

which the patient was responsible for $1,043. This oncologist typically 

saw patients daily for 2 weeks, billing the demonstration codes each 

time. His patients then had 2 weeks off chemotherapy before beginning 

the cycle again. This oncologist was also among the top 

10 billers overall and was allowed a total of $443,170 for the 

demonstration codes. 

 OE I-09-05-00 171 MEDI C ARE ’ S 2 0 05 CHEMOTHERAP Y DE M O N S T R AT I O N P R O J E C T 8 
F I NDING S 

Seven percent of demonstration claims 

did not comply with demonstration rules 

or were paid incorrectly, resulting in 

almost $17 million in net overpayments 

As shown in Table 1, 5.3 percent of the 

assessments allowed under the 

demonstration did not meet one or more 

of the requirements of CR 3670, 

resulting in $12 million in 

inappropriate allowances. According to claims data, in 62 percent of 

noncompliant assessments, beneficiaries received chemotherapy for 

conditions other than cancer.12 In 46 percent, no push or infusion 

chemotherapy administration code was allowed on the same day as the 

demonstration codes. Medicare also paid for a relatively small number 

of assessments for which carriers allowed codes related to only one or 

two symptoms, instead of the required three, and assessments that took 

place in a setting other than the physician’s office. 

Table 1: Demonstration Payment Errors 

Type of Error Number Percentage Allowed Amount 

Noncompliance* 94,801 5.3% $12,061,947 

- No cancer diagnosis 58,891 3.3% $7,700,901 

- No chemotherapy 43,996 2.4% $5,703,925 

- Less than 3 codes 8,681 0.5% $667,499 

- Not in office 116 0.01% $14,950 

Incorrect Amount 41,758 2.3% $4,936,708 

- Duplicates 37,397 2.1% $4,797,354 

- Multiple units 556 0.03% $161,847 

- Uncertain reason 3,805 0.2% ($22,493) 

Overlapping (3,518) (0.2%) ($208,789) 

Total 133,041 7.4% $16,789,865** 

*Numbers in this subcategory sum to more than the subtotal because some demonstration 

assessments did not comply with more than one requirement. 

**Total does not equal sum of subtotals due to rounding. 

Source: OIG analysis of Medicare claims data, 2006. 

Medicare allowed too great or too small an amount for 2.3 percent of 

demonstration assessments, yielding an additional $4.9 million in net 

overpayments. These mostly consisted of duplicate claims for one or 

12 CMS did not provide a list of diagnosis codes that defined a “cancer patient” for the 

2005 demonstration. For this analysis, we considered a beneficiary a “cancer patient” if he 

or she had any diagnosis in the “Cancer (Neoplasm, by site, malignant)” or “Carcinoma” 

sections of the “International Classification of Diseases, 9th Revision, Clinical Modification, 

Volume 2” listed on any claim for chemotherapy administration with a service date in 2005. 

 OE I-09-05-00 171 MEDI C ARE ’ S 2 0 05 CHEMOTHERAP Y DE M O N S T R AT I O N P R O J E C T 9 
F I NDING S 

more of the demonstration codes. In some cases, participants obtained 

excess reimbursement—$5,936.40 for a single demonstration 

assessment in one case—by entering multiple units for one or more of 

the demonstration codes associated with a single date of service. We 

were unable to determine the reason for the miscalculations in the 

remaining cases, but most involved apparent Medicare underpayments. 

One carrier accounted for approximately 87 percent of these 

unexplained payment reductions, and the remainder was almost 

entirely confined to three additional carriers. 

CMS did not sufficiently define the parameters 

of the demonstration, leading to inconsistent 

data collection and incomplete and 

unreliable data 

CMS did not mandate a 

specific approach to collecting 

the demonstration data. 

Consequently, as we reported 

in our September 2005 letter 

to Senator Grassley, oncology practices implemented different 

procedures for collecting the data. Reviewing the full year’s data has 

not allayed our previous concerns and has provided further evidence 

that the data are unreliable. 

Prior concerns revisited 

In our September letter, we expressed concern that participants in the 

demonstration were not using consistent methods to administer the 

assessments. Some of the practices we visited were asking patients 

about their symptoms during the past week, but others reported the 

data based on how the patients were feeling on the day of 

chemotherapy. This could introduce considerable measurement error 

into the data CMS is collecting. For example, one nurse told us if a 

patient vomited several times in the last week, but was feeling fine on 

the day of the assessment, she would report the lowest level of 

nausea/vomiting. Based on the same information, another practitioner 

could legitimately report the highest level of nausea/vomiting, based on 

the frequency of the patient’s symptoms during the past week. 

Furthermore, while nurses in the practices we visited asked patients 

directly about their experiences and then interpreted them on the 

four-point scale, some national oncology associations had recommended 

that their members instead have patients fill out self-assessment forms 

and use those to bill the demonstration codes. (CMS had instructed 

that participants or qualified staff members perform the assessments.) 

Because practitioners used different methods to gather demonstration 

 OE I-09-05-00 171 MEDI C ARE ’ S 2 0 05 CHEMOTHERAP Y DE M O N S T R AT I O N P R O J E C T 10 
F I NDING S 

data, the reliability of the data was further compromised. Studies have 

shown that research subjects consistently offer more positive health 

assessments to interviewers than to paper questionnaires.13 

Our analysis of the full year’s data reinforces our concerns about the 

timeframes for which participants assessed their patients under the 

demonstration. As mentioned previously, CMS indicated in CR 3670 

that the symptoms were to be assessed over the past week. However, 

approximately 32 percent of the symptom assessments occurred less 

than 1 week after the previous chemotherapy administration, meaning 

that certain days would be “in the last week” for more than one 

assessment. CMS did not instruct participants on how to report 

symptoms in these situations and indicated that carriers should pay the 

claims. Based on our visits to clinics, beneficiaries at these assessments 

probably were asked about their symptoms since their last visit or on 

the day of the current visit, not over the last week, raising further 

questions about the consistency of data collection in the demonstration. 

CMS did not require participants to document demonstration 

assessments and their results in the patient’s medical record. 

Therefore, CMS cannot verify the accuracy of the data. Given that 

participants did not generally submit demonstration claims 

immediately after performing an assessment (the median delay was 

approximately 9 days), not documenting the services could present a 

significant data integrity concern. This concern is heightened for the 

16 percent of demonstration claims submitted separately from the 

related chemotherapy administration—approximately 31 days later, on 

average. Although the practices we visited generally documented their 

patients’ symptoms, some assessments were not documented. In the 

absence of documentation, it is unclear how the participant would have 

ensured the submitted codes accurately reflected their patients’ 

symptoms. 

Lastly, as discussed in the September 8 letter to Senator Grassley, CMS 

failed to collect information about the interventions physicians use to 

address nausea/vomiting, pain, and fatigue. The oncologists and 

13 Hochstim, J.R., “A Critical Comparison of Three Stages of Collecting Data From 

Households.” Journal of the American Statistical Association, Volume 62, 976-989, 1967;” 

and Biemer, P.P., “Health Insurance Finance Agency Evaluation—Unpublished Data.” 

Research Triangle Institute, 1997, as cited in Dillman, Don, Mail and Internet Surveys: 

The Tailored Design Method, 2000. 

 OE I-09-05-00 171 MEDI C ARE ’ S 2 0 05 CHEMOTHERAP Y DE M O N S T R AT I O N P R O J E C T 11 
F I NDING S 

researchers to whom we spoke suggested that omitting this information 

would limit the usefulness of the demonstration data. The American 

Society of Clinical Oncology has expressed similar concerns. 

New concerns surface 

The demonstration focused specifically on the symptoms of cancer 

patients, but the demonstration data may include the experiences of 

patients who received chemotherapy for other conditions. CMS 

specified in CR 3670 that, to be eligible for payment, the demonstration 

codes had to point to a cancer diagnosis. However, CMS did not define a 

list of valid diagnosis codes for the demonstration, leaving the carriers 

to create their own lists. According to our analysis, these lists were not 

consistent. For example, several of the larger carriers allowed no 

demonstration claims with a diagnosis code corresponding to a 

chemotherapy encounter (which does not specify the condition for which 

chemotherapy is being administered), while others allowed up to 

6 percent with this diagnosis and 5 smaller carriers allowed more than 

15 percent. 

According to CMS, the demonstration focused on measuring outcomes in 

patients undergoing chemotherapy, but 8 percent of the allowed 

demonstration assessments were conducted on the first day of a course 

of treatment.

14 Although data from these assessments could serve as a 

baseline measurement for each patient, the data could not be considered 

an assessment of the symptoms associated with chemotherapy because, 

on the first day of treatment, the beneficiary has not yet been given any 

chemotherapy. CMS has not indicated that “first-day” demonstration 

data will be treated any differently than data gathered during a course 

of treatment. 

14 We considered an assessment to be on the first day of a course of treatment if it 

occurred after at least a 60-day break in chemotherapy billings. Since we only had 2005 

data available, we only looked at assessments that occurred on March 1 or later, to account 

for chemotherapy that may have taken place at the end of 2004. 

 OE I-09-05-00 171 MEDI C ARE ’ S 2 0 05 CHEMOTHERAP Y DE M O N S T R AT I O N P R O J E C T 12 
ΔF I NDING S 

 CONCLUSION 

We previously reported that physicians who participated in the 

demonstration used inconsistent methods and timeframes to assess 

beneficiaries’ symptoms. Our analysis of a full year of demonstration 

data revealed numerous inconsistencies and anomalies that raise 

further issues concerning the integrity of the data. Based on these 

concerns, we conclude that the demonstration data are unreliable. 

 OE I-09-05-00 171 MEDI C ARE ’ S 2 0 05 CHEMOTHERAP Y DE M O N S T R AT I O N P R O J E C T 13 
Δ ACKNOWLEDGMENTS 

This report was prepared under the direction of Paul A. Gottlober, 

Regional Inspector General for Evaluation and Inspections in the 

San Francisco regional office, and Deborah Harvey, Assistant Regional 

Inspector General. Other principal Office of Evaluation and Inspections 

staff who contributed include: 

Scott Hutchison, Project Leader 

Steven Zerebecki, Program Analyst 

Rob Gibbons, Technical Support 

Scott Horning, Technical Support 

Doris Jackson, Program Specialist 

Tricia Davis, Director, Medicare Branch 

 OE I-09-05-00 171 MEDI C ARE ’ S 2 0 05 CHEMOTHERAP Y DE M O N S T R AT I O N P R O J E C T 14