Department of Health and Human Services

OFFICE OF 

INSPECTOR GENERAL 

C P
OST AND  ERFORMANCE OF 

M  2005 C
EDICARE S  HEMOTHERAPY

D P
EMONSTRATION  ROJECT

Daniel R. Levinson

Inspector General 

August 2006

OEI-09-05-00171
 Office of Inspector General 

http://oig.hhs.gov 
The mission of the Office of Inspector General (OIG), as mandated by Public Law 95-452, 
as amended, is to protect the integrity of the Department of Health and Human Services 
(HHS) programs, as well as the health and welfare of beneficiaries served by those 
programs.  This statutory mission is carried out through a nationwide network of audits, 
investigations, and inspections conducted by the following operating components: 
Office of Audit Services 
The Office of Audit Services (OAS) provides all auditing services for HHS, either by 
conducting audits with its own audit resources or by overseeing audit work done by others. 
Audits examine the performance of HHS programs and/or its grantees and contractors 
in carrying out their respective responsibilities and are intended to provide independent 
assessments of HHS programs and operations.  These assessments help reduce waste, 
abuse, and mismanagement and promote economy and efficiency throughout HHS. 
Office of Evaluation and Inspections 
The Office of Evaluation and Inspections (OEI) conducts national evaluations to provide 
HHS, Congress, and the public with timely, useful, and reliable information on 
significant issues.  Specifically, these evaluations focus on preventing fraud, waste, or 
abuse and promoting economy, efficiency, and effectiveness in departmental programs.  
To promote impact, the reports also present practical recommendations for improving 
program operations. 
Office of Investigations 
The Office of Investigations (OI) conducts criminal, civil, and administrative 
investigations of allegations of wrongdoing in HHS programs or to HHS beneficiaries 
and of unjust enrichment by providers.  The investigative efforts of OI lead to criminal 
convictions, administrative sanctions, or civil monetary penalties. 
Office of Counsel to the Inspector General 
The Office of Counsel to the Inspector General (OCIG) provides general legal services to 
OIG, rendering advice and opinions on HHS programs and operations and providing all 
legal support in OIG's internal operations. OCIG imposes program exclusions and civil 
monetary penalties on health care providers and litigates those actions within HHS. 
OCIG also represents OIG in the global settlement of cases arising under the Civil False 
Claims Act, develops and monitors corporate integrity agreements, develops compliance 
program guidances, renders advisory opinions on OIG sanctions to the health care 
community, and issues fraud alerts and other industry guidance.    E X E C U T I V E   S U M M A R Y  

OBJECTIVE 
To assess the cost and the overall performance of Medicares 2005 
chemotherapy demonstration project. 
BACKGROUND 
On January 1, 2005, the Centers for Medicare & Medicaid Services 
(CMS) initiated a 1-year Medicare Demonstration of Improved Quality 
of Care for Cancer Patients Undergoing Chemotherapy.  According to 
CMS, the purpose of the demonstration was to  . . . assess and provide 
new support for the quality of care for cancer patients undergoing 
chemotherapy. . . .  CMS provided a $130 allowance each time a 
chemotherapy practitioner reported to Medicare, via the claims system, 
an assessment of a patients levels of nausea and vomiting, pain, and 
fatiguethree conditions commonly experienced as symptoms of cancer 
or side effects of cancer treatment. Beneficiaries were responsible for 
the usual 20-percent coinsurance ($26 in this case) each time the codes 
were billed.  CMS estimated that Medicare and beneficiary 
expenditures for the demonstration would total $300 million.  The 
demonstration has continued in 2006 in a significantly modified form. 
On September 8, 2005, the Office of Inspector General provided Senator 
Charles E. Grassley, per his request, a preliminary assessment of the 
demonstrations costs and the adequacy of its data collection methods.  
We estimated that Medicare and its beneficiaries would pay    
$270 million for the demonstration in 2005 and noted that 
approximately 3 percent of demonstration payments were for services 
that did not meet program rules.  We also reported that participants 
administered the demonstration assessments inconsistently and that 
CMS was not collecting information on the interventions used to 
manage patients symptoms.  Finally, we noted that the demonstration 
allowance was disproportionate to the amount of effort involved on the 
part of the practitioners and that assessing symptoms was already part 
of routine cancer care. 
To evaluate the cost of the demonstration, we analyzed demonstration 
claims received by Medicare through the end of 2005.  To produce our 
overall assessment of the demonstration project, we conducted 
interviews with CMS staff and reviewed e-mails, meeting notes, and 
other workpapers CMS sent in response to our request for the entire 
project file and any supplemental documents related to the 
  OEI-09-05-00171  MEDICARES 2005 CHEMOTHERAPY DEMONSTRATION PROJECT  i  E X E C U T I V E   S U M M A R Y  
demonstration.  We also interviewed staff at four oncology practices to 
learn how they implemented the demonstration.  
FINDINGS 
Medicare and its beneficiaries will spend approximately $275 million 
on the 2005 chemotherapy demonstration.  Medicare is on track to 
allow approximately $275 million for the demonstration; beneficiary 
liability will total approximately $55 million. Approximately 90 percent 
of eligible practitioners took part in the demonstration, and 85 percent 
billed the demonstration codes at least half the time.  The median 
amount allowed per physician was approximately $23,000, but the top 
10 billers were allowed more than $270,000 each. 
Seven percent of demonstration claims did not comply with 
program rules or were paid incorrectly, resulting in $17 million in net 
overpayments.  Claims that Medicare carriers paid in error included 
duplicate billings, paying for the demonstration without concurrent 
chemotherapy, and paying when the beneficiary did not have a cancer 
diagnosis. 
CMS did not sufficiently define the parameters of the demonstration, 
leading to inconsistent data collection and incomplete and 
unreliable data.  Because CMS did not mandate a specific approach to 
collecting the demonstration data, oncology practices implemented 
inconsistent data collection procedures.  We identified numerous 
anomalies and gaps in the data and collection methods that 
demonstrate the unreliability and undermine the usefulness of the data. 
CONCLUSION 
We previously reported that physicians who participated in the 
demonstration used inconsistent methods and timeframes to assess 
beneficiaries symptoms. Our analysis of a full year of demonstration 
data revealed numerous inconsistencies and anomalies that raise 
further issues concerning the integrity of the data.  Based on these 
concerns, we conclude that the demonstration data are unreliable. 
  OEI-09-05-00171  MEDICARES 2005 CHEMOTHERAPY DEMONSTRATION PROJECT  ii    T A B L E   O F   C O N T E N T S  

EXECUTIVE SUMMARY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . i

I N T RO DUC T IO N  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  1

F I N D IN G S  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  7

Demonstration to cost $275 million . . . . . . . . . . . . . . . . . . . . . . . . .  7

Inappropriate payments total $17 million. . . . . . . . . . . . . . . . . . . .  9

Demonstration data inconsistent and unreliable  . . . . . . . . . . . . .  10

CONC LUSION  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  13

A C K NO WL E D G M E N T S  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  14
   I N T R O D U C T I O N  

OBJECTIVE 
To assess the cost and the overall performance of Medicares 2005 
chemotherapy demonstration project. 
BACKGROUND 
2005 Chemotherapy Demonstration Project 
On January 1, 2005, the Centers for Medicare & Medicaid Services 
(CMS) initiated a 1-year Medicare Demonstration of Improved Quality 
of Care for Cancer Patients Undergoing Chemotherapy.  CMS first 
publicly announced the demonstration in a November 15, 2004, final 
rule that implemented a revised payment methodology for drugs, which 
was mandated by the Medicare Prescription Drug, Improvement, and 
Modernization Act of 2003 (commonly known as the Medicare 
Modernization Act).1 
CMS cites section 402(a)(1)(B) of the Social Security Act Amendments of 
1967 (the Amendments) as the authority under which the 
demonstration was operated.  This section of the Amendments permits 
the Secretary of the Department of Health and Human Services to 
conduct demonstrations  . . . to determine whether payments for 
services other than those for which payment may be made under 
[Medicare] (and which are incidental to services for which payment may 
be made under [Medicare]) would, in the judgment of the Secretary, 
result in more economical provision and more effective utilization of 
services for which payment may be made. . . .  According to CMS, the 
purpose of the demonstration project was to  . . . assess and provide 
new support for the quality of care for cancer patients undergoing 
chemotherapy. . . . 2 
Under the demonstration, CMS provided a $130 allowance each time a 
chemotherapy practitioner reported to Medicare an assessment of a 
patients levels of nausea and vomiting, pain, and fatiguethree 
conditions commonly experienced as symptoms of cancer or side effects 
of cancer treatment.  To participate in the demonstration, physicians 
1 69 FR 66236. 
2 CMS. Fact Sheet:  Demonstration of Improved Quality of Care for Cancer Patients 
Undergoing Chemotherapy.  November 1, 2004.  Available online at 
http://www.cms.hhs.gov/media/press/release.asp?Counter=1245.  Accessed on 
August 26, 2005.   
  OEI-09-05-00171  MEDICARES 2005 CHEMOTHERAPY DEMONSTRATION PROJECT  1  I N T R O D U C T I O N  
submitted claims to Medicare that included special billing codes that 
CMS created for the demonstration.3  These codes described, on a  
four-point scale (not at all, a little, quite a bit, or very much), the 
degree to which the beneficiary had been affected by the three 
conditions.4  The demonstration required no formal enrollment; 
however, it was limited to physicians practicing in an office setting and 
to beneficiary visits at which the physician administered chemotherapy 
via infusion or push for a cancer diagnosis.5 
CMS estimated that Medicare and its beneficiaries would spend    
$300 million for the demonstration in 2005.  CMS did not waive the 
Medicare Part B coinsurance requirement for this demonstration. 
Therefore, beneficiaries were liable for the usual 20-percent coinsurance 
($26 in this case) each time their physician billed the demonstration 
codes. 
CMS regularly carries out demonstration and other research projects, 
but the chemotherapy demonstration was much larger in scope than is 
typical. The $300 million in estimated expenditures greatly exceeds the 
budgets of other CMS research projects; the next most expensive 
research project listed in CMSs Active Projects Report is a $60 million 
project with a performance period of 8 years.6  Also, while 
demonstration projects usually are limited geographically to a small 
number of sites, the 2005 chemotherapy demonstration was open to all 
physicians and beneficiaries nationwide. 
3 Although nonphysician practitioners were eligible to participate in the demonstration, 
physicians submitted more than 99 percent of paid demonstration claims.  Throughout this 
report we use the term physician to mean any provider of Medicare Part B services. 
4 CMS based the demonstrations four-point scale on the Rotterdam Symptom Checklist, 
an instrument developed as a patient self-assessment tool for measuring the quality of life 
of cancer patients.  A discussion of the Rotterdam scale and how it should be used can be 
found in Northern Centre for Healthcare Research, University of Groningen, The 
Netherlands, Measuring the Quality of Life of Cancer Patients with the Rotterdam 
Symptom Checklist (RCSL):  A Manual. 1996. 
5 In infusion chemotherapy, the physician dilutes the chemotherapy drug in a bag of fluid 
and then administers this solution into a vein over a specified period of time.  In the push 
technique, the physician uses a syringe to administer the chemotherapy drug directly into a 
vein. 
6 The Active Project Report inventories research, demonstration, and evaluation projects 
undertaken by CMS staff or by external entities with CMS support. 
  OEI-09-05-00171  MEDICARES 2005 CHEMOTHERAPY DEMONSTRATION PROJECT  2  I N T R O D U C T I O N  
Pursuant to CMS Change Request7 (CR) 3670, dated December 30, 
2004, demonstration claims were payable only if the following 
conditions were met: 
 	 The physician submitted at least one code each for 
nausea/vomiting, pain, and fatigue.  (If the physician submitted 
more than one code for a symptom category, carriers were 
instructed to allow only the highest-level code.) 
 	 The physician billed the demonstration codes for the same date 
of service as an allowed code for chemotherapy via infusion or 
push. 
 	 The date of service of the demonstration codes (and, therefore, 
the chemotherapy administration code) was in 2005. 
 	 The place of service referenced for both the demonstration codes 
and the chemotherapy administration was office. 
 	 The diagnosis code reported and referenced was for cancer. 
According to CMS, no further official instructions were provided to 
carriers about the demonstration. 
CMS has contracted with Mathematica Policy Research, a professional 
evaluation firm, to analyze the demonstration data.  According to CMS, 
beneficiaries at 80 percent of demonstration assessments reported 
having no nausea/vomiting and those at 70 percent reported having no 
pain. Fatigue was more variable, with beneficiaries at approximately 
30 percent of assessments reporting none, at 40 percent a little, at 20 
percent quite a bit, and at the rest very much.  Because the levels of 
nausea/vomiting and pain reported were consistently low, the contract 
project officer stated that CMS will focus on beneficiaries reported 
levels of fatigue in further analyses.  Among other things, CMS plans to 
correlate the demonstration data with other claims data, such as 
hospital admissions and emergency department visits. 
2006 Demonstration Project 
In a final rule dated November 21, 2005, CMS announced that the 
demonstration would continue in 2006 in a revised and less costly 
version.8  While both versions collect data via the submission of special 
billing codes, the codes for the 2006 demonstration must be submitted 
7 CMS uses Change Requests to instruct carriers to implement Medicare policy changes. 
8 70 FR 70272. 
  OEI-09-05-00171  MEDICARES 2005 CHEMOTHERAPY DEMONSTRATION PROJECT  3  I N T R O D U C T I O N  
with particular evaluation and management (E & M) visits, not with 
chemotherapy administration.  The 2006 demonstration requires that 
participants (1) submit special codes that correspond to the reason for 
the E & M visit, (2) indicate whether the participant is following clinical 
guidelines, and (3) describe the current disease state.  Medicare will 
allow $23 per encounter, and the usual Part B coinsurance requirement 
applies, meaning beneficiaries are liable for $4.60 per encounter.  
Participation in the 2006 demonstration is limited to hematologists, 
oncologists, hematologist/oncologists, and gynecological oncologists, and 
participants may only bill the demonstration for beneficiaries who have 
specific cancer diagnoses that appear on a list enumerated by CMS.  
The participant (or staff) provides all information used to bill the 2006 
demonstration; other than the physicians assessment of the current 
disease state, no beneficiary data are collected. 
Previous Work 
On August 12, 2005, Senator Charles E. Grassley, Chairman of the 
Senate Committee on Finance, sent a letter to Inspector General  
Daniel R. Levinson expressing concerns about the cost of the 
chemotherapy demonstration project and whether the data CMS was 
collecting would contribute to the goal of improving care for cancer 
patients. Senator Grassley requested that the Office of Inspector 
General (OIG) provide an assessment of the cost of the demonstration 
and the adequacy of its data collection methods. 
OIG provided an interim assessment to Senator Grassley on 
September 8, 2005.9  In that assessment, we estimated that Medicare 
and its beneficiaries would pay approximately $270 million to 
physicians and that two-thirds of eligible physicians would participate. 
Some participants had received large payments under the 
demonstrationas much as $320,000 in the first 6 months of the 
demonstration in one case.  Approximately 3 percent of the 
demonstration claims did not meet the requirements of CR 3670, 
resulting in $3.6 million in inappropriate expenditures.  We also 
expressed concerns about the reliability and usefulness of the 
demonstration data because of inconsistency in the ways participants 
administered the demonstration assessments and because CMS was not 
collecting information on the interventions physicians used to manage 
9 The letter is available on the Web site of the Senate Committee on Finance at 
http://finance.senate.gov/press/Gpress/2005/prg101305lett.pdf. 
  OEI-09-05-00171  MEDICARES 2005 CHEMOTHERAPY DEMONSTRATION PROJECT  4  I N T R O D U C T I O N  
patients symptoms.  Finally, we noted that the demonstration 
allowance was disproportionate to the amount of effort involved on the 
part of the participant and that assessing symptoms was already part of 
routine cancer care. 
In its January 2006 report to Congress, the Medicare Payment Advisory 
Commission (MedPAC) questioned the demonstrations validity and 
methodology.  MedPAC expressed concern that the demonstration: 
 	 did not have appropriate controls, 
 	 was announced and implemented without any period for 
comments by clinicians and researchers, 
 	 did not impose a uniform data collection process, 
 	 did not gather data over a consistent timeframe, and 
 	 did not collect data on interventions used to alleviate symptoms. 
MedPAC also reported that while all oncology practices it visited (in five 
States) were participating in the demonstration, most oncologists did 
not believe it would lead to quality improvements for patients or 
produce any useful research findings.  MedPAC concluded that the 
demonstration was designed to increase payments to oncologists and 
stated in a recommendation to Congress that  . . . [t]he Secretary 
should use his demonstration authority to test innovations in the 
delivery and quality of health care. Demonstrations should not be used 
as a mechanism to increase payments. . . .10 
METHODOLOGY 
To evaluate the cost of the demonstration and its financial impact on 
Medicare and its beneficiaries, we analyzed demonstration claims 
received through the end of 2005.  These data, which we received in 
February 2006, represent approximately 87 percent of the services 
rendered in 2005.  Because Medicare processes claims submitted up to  
1 year after the year in which services are rendered, complete data on 
the demonstration will not be available before December 31, 2006. 
10 Medicare Payment Advisory Commission.  "Report to the Congress:  Effects of 
Medicare Payment Changes on Oncology Services.  January 2006. 
  OEI-09-05-00171  MEDICARES 2005 CHEMOTHERAPY DEMONSTRATION PROJECT  5  I N T R O D U C T I O N  
We combined the qualitative data we collected for our September 
interim report with data from additional sources to produce our overall 
assessment of the demonstration project. Previously, we had visited 
four oncology practices that participated in the demonstration and 
interviewed staff at each about the methods they used to collect the 
data. We also interviewed cancer researchers and spoke with CMS staff 
about the project.  For this report, we conducted additional interviews 
with CMS staff and analyzed the 2005 claims data to identify patterns 
and issues that warrant further evaluation.  We also reviewed work 
papers that CMS sent in response to our request for the entire project 
file and any supplemental documents related to the demonstration. The 
work papers contained internal e-mails, meeting notes, and other work 
papers from the development and implementation of the demonstration, 
as well as publicly released information about the demonstration. 
  OEI-09-05-00171  MEDICARES 2005 CHEMOTHERAPY DEMONSTRATION PROJECT  6    F I N D I N G S  
Medicare and its beneficiaries will spend  Assuming that the rate of claims 
approximately $275 million on the 2005  submission in the fourth quarter of 
2005 is consistent with that of the 
chemotherapy demonstration 
fourth quarter of prior years, 
Medicare will allow approximately $275 million for the 2005 
chemotherapy demonstration.11 Because beneficiaries are responsible 
for 20-percent coinsurance on demonstration claims, beneficiary liability 
for the demonstration project will total approximately $55 million. 
Participation in the demonstration was high:  approximately 90 percent 
of eligible physicians took part.  Based on a chi-square test, physician 
practices that had 11 or more beneficiaries receiving chemotherapy for a 
cancer diagnosis were more 
FIGURE 1:  likely to participate than 
Percentage of  those with fewer eligible 
Never 
eligible physicians  Always 
10%  patients.  Approximately   
who billed the  11%  Less 
than half  11 percent of eligible 
demonstration 
5% 
codes at a given  physicians billed and were 
frequency  paid for the demonstration 
(e.g, 11 percent of 
codes at every eligible 
eligible physicians 
chemotherapy visit, and 
always billed the 
demonstration  85 percent billed the 
codes)  demonstration codes at 
least half the time. (See 
Figure 1.)  Overall, 
At least 
half  Medicare paid for a 
74% 
demonstration assessment 
at 88 percent of eligible 
chemotherapy visits. 
Source: OIG analysis of Medicare claims data, 2006. 
The demonstration provided substantial reimbursement to some physicians 
Approximately 7,500 physicians participated in the chemotherapy 
demonstration.  The median amount allowed per physician was 
approximately $23,000; however, some physicians were allowed 
11 Medicare processes claims submitted up to 1 year after the end of the year in which 
the service was rendered.  As of December 31, 2005, Medicare carriers had allowed 
1.8 million assessments performed under the demonstration project, totaling $239 million in 
Medicare and beneficiary payments.  CMS projects that spending on the demonstration will 
total $255 million based on claims received through July 31, 2006. 
  OEI-09-05-00171  MEDICARES 2005 CHEMOTHERAPY DEMONSTRATION PROJECT  7  F I N D I N G S  
significantly more than the median.  Figure 2 shows the distribution of 
demonstration payments to individual physicians.  Medicare allowed 
more than $100,000 each for 308 individual physicians, and the top 10 
billers were allowed more than $270,000 each.  The top biller, an 
oncologist in Florida, was allowed $625,603 for the demonstration codes. 
The physician with the second-highest allowed amount for the 
demonstration, an oncologist in Kansas, was allowed $507,563.  These 
high billers represent practices that performed substantial amounts of 
push and infusion chemotherapy and, therefore, had many 
opportunities to bill for the demonstration. 
FIGURE 2: 
Distribution of  3,000 
demonstration 
2,535 
allowances  2,500 
s 
r
e 1,958 
d 2,000 
vi
o
r
p 1,455 
of  1,500  1,310 
r 
e
b
m 1,000 
u
N
500 
275 
33 
0 
Less than  $10,000 to  $25,000 to  $50,000 to  $100,000 to  $200,000 or 
$10,000  $24,999  $49,999  $99,999  $199,999  more 
Total allowed for demonstration codes 
Source:  OIG analysis of Medicare claims data, 2006. 
On average, participants in the demonstration were allowed $529 per 
patient; for some, the amount was much higher.  An oncologist in 
California had the highest mean allowed amount per patient, $5,214, of 
which the patient was responsible for $1,043.  This oncologist typically 
saw patients daily for 2 weeks, billing the demonstration codes each 
time.  His patients then had 2 weeks off chemotherapy before beginning 
the cycle again.  This oncologist was also among the top 
10 billers overall and was allowed a total of $443,170 for the 
demonstration codes. 
  OEI-09-05-00171  MEDICARES 2005 CHEMOTHERAPY DEMONSTRATION PROJECT  8  F I N D I N G S  

Seven percent of demonstration claims  As shown in Table 1, 5.3 percent of the 
did not comply with demonstration rules  assessments allowed under the 
demonstration did not meet one or more 
or were paid incorrectly, resulting in 
of the requirements of CR 3670, 
almost $17 million in net overpayments 
resulting in $12 million in 
inappropriate allowances. According to claims data, in 62 percent of 
noncompliant assessments, beneficiaries received chemotherapy for 
conditions other than cancer.12  In 46 percent, no push or infusion 
chemotherapy administration code was allowed on the same day as the 
demonstration codes.  Medicare also paid for a relatively small number 
of assessments for which carriers allowed codes related to only one or 
two symptoms, instead of the required three, and assessments that took 
place in a setting other than the physicians office. 
Table 1: Demonstration Payment Errors 
Type of Error  Number  Percentage  Allowed Amount 
Noncompliance*  94,801  5.3%  $12,061,947 
- No cancer diagnosis  58,891  3.3%  $7,700,901 
- No chemotherapy  43,996  2.4%  $5,703,925 
-Less than 3 codes  8,681  0.5%  $667,499 
-Not in office  116  0.01%  $14,950 
Incorrect Amount  41,758  2.3%  $4,936,708 
-Duplicates  37,397  2.1%  $4,797,354 
- Multiple units  556  0.03%  $161,847 
- Uncertain reason  3,805  0.2%  ($22,493) 
Overlapping  (3,518)  (0.2%)  ($208,789) 
Total  133,041  7.4%  $16,789,865** 
*Numbers in this subcategory sum to more than the subtotal because some demonstration 

assessments did not comply with more than one requirement. 

**Total does not equal sum of subtotals due to rounding. 

Source:  OIG analysis of Medicare claims data, 2006. 

Medicare allowed too great or too small an amount for 2.3 percent of 
demonstration assessments, yielding an additional $4.9 million in net 
overpayments.  These mostly consisted of duplicate claims for one or 
12 CMS did not provide a list of diagnosis codes that defined a cancer patient for the 
2005 demonstration.  For this analysis, we considered a beneficiary a cancer patient if he 
or she had any diagnosis in the Cancer (Neoplasm, by site, malignant) or Carcinoma 
sections of the International Classification of Diseases, 9th Revision, Clinical Modification, 
Volume 2 listed on any claim for chemotherapy administration with a service date in 2005. 
  OEI-09-05-00171  MEDICARES 2005 CHEMOTHERAPY DEMONSTRATION PROJECT  9  F I N D I N G S  
more of the demonstration codes.  In some cases, participants obtained 
excess reimbursement$5,936.40 for a single demonstration 
assessment in one caseby entering multiple units for one or more of 
the demonstration codes associated with a single date of service. We 
were unable to determine the reason for the miscalculations in the 
remaining cases, but most involved apparent Medicare underpayments. 
One carrier accounted for approximately 87 percent of these 
unexplained payment reductions, and the remainder was almost 
entirely confined to three additional carriers. 
CMS did not sufficiently define the parameters  CMS did not mandate a 
of the demonstration, leading to inconsistent  specific approach to collecting 
the demonstration data.  
data collection and incomplete and   
Consequently, as we reported 
unreliable data 
in our September 2005 letter 
to Senator Grassley, oncology practices implemented different 
procedures for collecting the data.  Reviewing the full years data has 
not allayed our previous concerns and has provided further evidence 
that the data are unreliable. 
Prior concerns revisited 
In our September letter, we expressed concern that participants in the 
demonstration were not using consistent methods to administer the 
assessments.  Some of the practices we visited were asking patients 
about their symptoms during the past week, but others reported the 
data based on how the patients were feeling on the day of 
chemotherapy.  This could introduce considerable measurement error 
into the data CMS is collecting.  For example, one nurse told us if a 
patient vomited several times in the last week, but was feeling fine on 
the day of the assessment, she would report the lowest level of 
nausea/vomiting.  Based on the same information, another practitioner 
could legitimately report the highest level of nausea/vomiting, based on 
the frequency of the patients symptoms during the past week. 
Furthermore, while nurses in the practices we visited asked patients 
directly about their experiences and then interpreted them on the  
four-point scale, some national oncology associations had recommended 
that their members instead have patients fill out self-assessment forms 
and use those to bill the demonstration codes.  (CMS had instructed 
that participants or qualified staff members perform the assessments.)  
Because practitioners used different methods to gather demonstration 
  OEI-09-05-00171  MEDICARES 2005 CHEMOTHERAPY DEMONSTRATION PROJECT  10  F I N D I N G S  
data, the reliability of the data was further compromised.  Studies have 
shown that research subjects consistently offer more positive health 
assessments to interviewers than to paper questionnaires.13 
Our analysis of the full years data reinforces our concerns about the 
timeframes for which participants assessed their patients under the 
demonstration.  As mentioned previously, CMS indicated in CR 3670 
that the symptoms were to be assessed over the past week. However, 
approximately 32 percent of the symptom assessments occurred less 
than 1 week after the previous chemotherapy administration, meaning 
that certain days would be in the last week for more than one 
assessment. CMS did not instruct participants on how to report 
symptoms in these situations and indicated that carriers should pay the 
claims. Based on our visits to clinics, beneficiaries at these assessments 
probably were asked about their symptoms since their last visit or on 
the day of the current visit, not over the last week, raising further 
questions about the consistency of data collection in the demonstration. 
CMS did not require participants to document demonstration 
assessments and their results in the patients medical record.  
Therefore, CMS cannot verify the accuracy of the data.  Given that 
participants did not generally submit demonstration claims 
immediately after performing an assessment (the median delay was 
approximately 9 days), not documenting the services could present a 
significant data integrity concern. This concern is heightened for the  
16 percent of demonstration claims submitted separately from the 
related chemotherapy administrationapproximately 31 days later, on 
average.  Although the practices we visited generally documented their 
patients symptoms, some assessments were not documented.  In the 
absence of documentation, it is unclear how the participant would have 
ensured the submitted codes accurately reflected their patients 
symptoms. 
Lastly, as discussed in the September 8 letter to Senator Grassley, CMS 
failed to collect information about the interventions physicians use to 
address nausea/vomiting, pain, and fatigue.  The oncologists and 
13 Hochstim, J.R., A Critical Comparison of Three Stages of Collecting Data From 
Households. Journal of the American Statistical Association, Volume 62, 976-989, 1967; 
and Biemer, P.P., Health Insurance Finance Agency EvaluationUnpublished Data.  
Research Triangle Institute, 1997, as cited in Dillman, Don, Mail and Internet Surveys:  
The Tailored Design Method, 2000. 
  OEI-09-05-00171  MEDICARES 2005 CHEMOTHERAPY DEMONSTRATION PROJECT  11  F I N D I N G S  
researchers to whom we spoke suggested that omitting this information 
would limit the usefulness of the demonstration data.  The American 
Society of Clinical Oncology has expressed similar concerns. 
New concerns surface 
The demonstration focused specifically on the symptoms of cancer 
patients, but the demonstration data may include the experiences of 
patients who received chemotherapy for other conditions.  CMS 
specified in CR 3670 that, to be eligible for payment, the demonstration 
codes had to point to a cancer diagnosis.  However, CMS did not define a 
list of valid diagnosis codes for the demonstration, leaving the carriers 
to create their own lists.  According to our analysis, these lists were not 
consistent.  For example, several of the larger carriers allowed no 
demonstration claims with a diagnosis code corresponding to a 
chemotherapy encounter (which does not specify the condition for which 
chemotherapy is being administered), while others allowed up to   
6 percent with this diagnosis and 5 smaller carriers allowed more than 
15 percent. 
According to CMS, the demonstration focused on measuring outcomes in 
patients undergoing chemotherapy, but 8 percent of the allowed 
demonstration assessments were conducted on the first day of a course 
of treatment.14  Although data from these assessments could serve as a 
baseline measurement for each patient, the data could not be considered 
an assessment of the symptoms associated with chemotherapy because, 
on the first day of treatment, the beneficiary has not yet been given any 
chemotherapy. CMS has not indicated that first-day demonstration 
data will be treated any differently than data gathered during a course 
of treatment. 
14 We considered an assessment to be on the first day of a course of treatment if it 
occurred after at least a 60-day break in chemotherapy billings.  Since we only had 2005 
data available, we only looked at assessments that occurred on March 1 or later, to account 
for chemotherapy that may have taken place at the end of 2004. 
  OEI-09-05-00171  MEDICARES 2005 CHEMOTHERAPY DEMONSTRATION PROJECT  12  F I N D I N G S  
  C O N C L U S I O N  

We previously reported that physicians who participated in the 
demonstration used inconsistent methods and timeframes to assess 
beneficiaries symptoms. Our analysis of a full year of demonstration 
data revealed numerous inconsistencies and anomalies that raise 
further issues concerning the integrity of the data.  Based on these 
concerns, we conclude that the demonstration data are unreliable. 
  OEI-09-05-00171  MEDICARES 2005 CHEMOTHERAPY DEMONSTRATION PROJECT  13    A C K N O W L E D G M E N T S  

This report was prepared under the direction of Paul A. Gottlober, 
Regional Inspector General for Evaluation and Inspections in the 
San Francisco regional office, and Deborah Harvey, Assistant Regional 
Inspector General.  Other principal Office of Evaluation and Inspections 
staff who contributed include: 
Scott Hutchison, Project Leader 
Steven Zerebecki, Program Analyst 
Rob Gibbons, Technical Support 
Scott Horning, Technical Support 
Doris Jackson, Program Specialist 
Tricia Davis, Director, Medicare Branch 
  OEI-09-05-00171  MEDICARES 2005 CHEMOTHERAPY DEMONSTRATION PROJECT  14