United States Government Accountability Office

GAO Testimony 

Before the Subcommittee on Health, 

Committee on Ways and Means, House of 

Representatives

MEDICARE

Quality of CMS 

Communications to 

Beneficiaries on the 

Prescription Drug Benefit 

Could Be Improved 

Statement of Leslie G. Aronovitz 

Director, Health Care 

For Release on Delivery

Expected at 2:00 p.m. EDT

Thursday, May 4, 2006 

 

GAO-06-715T 
What GAO Found

United States Government Accountability Office

Why GAO Did This Study

Highlights

Accountability Integrity Reliability

May 4, 2004

MEDICARE

Quality of CMS Communications to 

Beneficiaries on the Prescription Drug 

Benefit Could Be Improved 

Highlights of GAO-06-715T, a testimony 

before the Subcommittee on Health, 

Committee on Ways and Means, U.S. 

House of Representatives

Today’s hearing focuses on

Medicare Part D, the program’s 

new outpatient prescription drug 

benefit. On January 1, 2006,

Medicare began providing this 

benefit, and beneficiaries have until 

May 15, 2006, to enroll without the 

risk of penalties. The Centers for 

Medicare & Medicaid Services 

(CMS), which administers the Part 

D benefit, has undertaken outreach

and education efforts to inform

beneficiaries and their advisers.

GAO was asked to discuss how 

CMS can better ensure that 

Medicare beneficiaries are

informed about the Part D benefit. 

This testimony is based on 

Medicare: Communications 

to Beneficiaries on the 

Prescription Drug Benefit Could 

Be Improved, GAO-06-654 

(May 3, 2006). 

What GAO Recommends

In its May 2006 report, GAO 

recommended that the CMS 

Administrator enhance the quality

of its communications on the Part

D benefit, including clarifying 

written materials, monitoring the 

accuracy and completeness of help

line responses, and improving the

usability of the Medicare Web site. 

CMS said that GAO’s findings did 

not present a complete and

accurate picture of its activities. 

However, CMS said that it supports 

the goals of GAO’s 

recommendations and is already 

taking steps to implement them. 

Information given in the six sample documents that GAO reviewed 

describing the Part D benefit was largely complete and accurate, although 

this information lacked clarity. First, about 40 percent of seniors read at or 

below the fifth-grade level, but the reading levels of these documents ranged 

from seventh grade to postcollege. Second, on average, the six documents 

we reviewed did not comply with about half of 60 common guidelines for 

good communication. For example, the documents used too much technical 

jargon and often did not define difficult terms. Moreover, 16 beneficiaries 

and advisers that GAO tested reported frustration with the documents’ lack 

of clarity and had difficulty completing the tasks assigned to them. 

Customer service representatives (CSRs) answered about two-thirds of the 

500 calls GAO placed to CMS’s 1-800-MEDICARE help line accurately and 

completely. Of the remainder, 18 percent of the calls received inaccurate 

responses, 8 percent of the responses were inappropriate given the question 

asked, and about 3 percent received incomplete responses. In addition, 

about 5 percent of GAO’s calls were not answered, primarily because of 

disconnections. The accuracy and completeness of CSRs’ responses varied 

significantly across the five questions. For example, while CSRs provided 

accurate and complete responses to calls about beneficiaries’ eligibility for 

financial assistance 90 percent of the time, the accuracy rate for calls 

concerning the drug plan that would cost the least for a beneficiary with 

specified prescription drug needs was 41 percent. For this question, the 

CSRs responded inappropriately for 35 percent of the calls by explaining that 

they could not identify the least costly plan without the beneficiary’s 

personal information—even though CSRs had the information needed to 

answer the question. The time GAO callers waited to speak with CSRs also 

varied, ranging from no wait time to over 55 minutes. For 75 percent of the 

calls—374 of the 500—the wait was less than 5 minutes. 

The Part D benefit portion of the Medicare Web site can be difficult to use. 

GAO’s test of the site’s overall usability—the ease of finding needed 

information and performing various tasks—resulted in scores of 47 percent 

for seniors and 53 percent for younger adults, out of a possible 100 percent. 

While there is no widely accepted benchmark for usability, these scores

indicate that using the site can be difficult. For example, the prescription 

drug plan finder was complicated to use and some of its key functions, such 

as “continue” and “choose a drug plan,” were often not visible on the page 

without scrolling down. 

www.gao.gov/cgi-bin/getrpt?GAO-06-715T.

To view the full product, including the scope 

and methodology, click on the link above. 

For more information, contact Leslie G. 

Aronovitz at aronovitzl@gao.gov or (312) 

220-7600. 
Mr. Chairman and Members of the Subcommittee: 

I am pleased to be here today as you discuss the Medicare outpatient 

prescription drug benefit, known as the Part D benefit, which was 

established by the Medicare Prescription Drug, Improvement, and 

Modernization Act of 20031 with coverage beginning on January 1, 2006.

Until this time, Medicare, the program that finances health care benefits 

for about 42 million elderly and disabled beneficiaries, had not generally

provided coverage for outpatient prescription drugs. Beneficiaries who opt 

to enroll in Part D may choose a drug plan from those offered by private 

plan sponsors under contract to the Centers for Medicare & Medicaid 

Services (CMS), which administers the Part D benefit. These plans differ in 

the drugs covered, pharmacies used, and enrollee costs. As of 

April 20, 2006, more than 30 million of Medicare’s 42 million beneficiaries 

were enrolled in a Part D plan or had other outpatient prescription drug 

coverage. Beneficiaries have until the end of the current enrollment 

period, May 15, 2006, to enroll in the Part D benefit and select a plan 

without the risk of penalties in the form of higher premiums. 

Given the newness and complexity of the Part D benefit, it is critical that 

beneficiaries and their advisers, including members of their families, 

understand the options available to them. Understanding these options 

enables beneficiaries to make informed decisions on whether to enroll in

the Part D benefit, and if they decide to enroll, which drug plan to choose. 

As part of its responsibilities, CMS has undertaken efforts to provide 

beneficiaries and their advisers with the information they need about the 

Part D benefit through various media, including written documents, the 

1-800-MEDICARE help line,2

 and the Medicare Web site.3 CMS’s education

efforts are important because widespread confusion has been reported 

among beneficiaries about the costs and coverage under the new benefit.

 1

Pub. L. No. 108-173, § 101, 117 Stat. 2066, 2071-2152 (to be codified at 42 U.S.C. §§ 1395w101—1395w-152). The MMA redesignated the previous part D of title XVIII of the Social 

Security Act as part E and inserted a new part D after part C.

2

In December 2004, we reported on the information being provided to beneficiaries through

the Medicare help line on eligibility, enrollment, and benefits. See GAO, Medicare:

Accuracy of Responses from the 1-800-MEDICARE Help Line Should Be Improved, 

GAO-05-130 (Washington, D.C.: Dec. 8, 2004).

3

The Medicare Web site is www.medicare.gov.

Page 1 GAO-06-715T
You and others have expressed interest in ensuring that Medicare 

beneficiaries receive the information they need to make informed 

decisions. My remarks today will focus on (1) the extent to which CMS’s 

written documents describe the Part D benefit in a clear, complete, and 

accurate manner; (2) the effectiveness of CMS’s 1-800-MEDICARE help 

line in providing accurate, complete, and prompt responses to callers 

inquiring about the Part D benefit; and (3) whether CMS’s Medicare Web

site presents information on the Part D benefit in a usable manner. My 

testimony will summarize findings of a report we released yesterday that 

examines CMS’s Medicare Part D benefit communications to beneficiaries 

in more detail and includes recommendations to the CMS Administrator 

for improving the quality of the agency’s Part D benefit education and 

outreach materials.4

To address these issues, we interviewed CMS officials responsible for 

written documents about the Part D benefit, the 1-800-MEDICARE help 

line, and the Medicare Web site. To assess the clarity, completeness, and 

accuracy of written documents, we performed an in-depth review of a 

sample of six CMS documents describing the Part D benefit, including 

contracting with the American Institutes for Research (AIR), a firm with 

experience in evaluating written documents, to assess their clarity. (See 

app. I for a list of written documents reviewed.) We also placed 500 calls 

to 1-800-MEDICARE, posing one of five questions related to the Part D 

benefit in each call so that each question was asked 100 times. We 

evaluated the accuracy and completeness of customer service 

representatives’ (CSR) responses to these questions. (See app. II for the 

questions and criteria we used to evaluate the accuracy and completeness 

of CSR responses to calls we made to 1-800 MEDICARE.) To assess the 

usability of the Part D benefit information available on the Medicare Web 

site, we contracted with the Nielsen Norman Group (NN/g), an expert on 

Web design. We conducted our work from November 2005 through 

May 2006, in accordance with generally accepted government auditing 

standards. 

In summary, CMS successfully developed a large volume of information 

about the new Part D benefit and made it available to beneficiaries 

through a variety of sources, despite the challenge of developing this 

information within a short time frame. However, the quality of CMS’s 

 4

GAO, Medicare: Communications to Beneficiaries on the Prescription Drug Benefit 

Could Be Improved, GAO-06-654 (Washington, D.C.: May 3, 2006). 

Page 2 GAO-06-715T
communications to beneficiaries and their advisers about the Part D 

benefit could be improved. For example, although the six CMS written 

documents we reviewed were largely accurate and complete, they often 

lacked clarity. Specifically, while about 40 percent of seniors read at or 

below the fifth-grade level, the reading levels of the documents ranged 

from seventh grade to postcollege. Moreover, the six documents used too 

much technical jargon and often did not define difficult terms. Similarly, 

although 67 percent of the responses to the 500 calls we placed to CMS’s 

1-800-MEDICARE help line regarding the Part D benefit were accurate and 

complete, we nonetheless received a substantial number of responses that 

were not. Eighteen percent of the calls received inaccurate responses, 

8 percent of the responses were inappropriate given the question asked, 

about 3 percent received incomplete responses, and about 5 percent of our 

calls were not answered, primarily due to disconnections.5

 In addition, our 

review of the Part D benefit portion of the Medicare Web site showed that 

this site can be difficult to use. In usability tests that examined the ease of

finding needed information and performing various tasks, we found that, 

for overall usability, the Web site scored 47 percent for seniors and 

53 percent for younger adults, out of a possible 100 percent. While there is

no widely accepted benchmark for usability, these scores indicate that

using the site can be difficult. Therefore, in the report we issued yesterday,

we made specific recommendations to the CMS Administrator to enhance 

the quality of the agency’s communications on the Part D benefit, 

including clarifying written materials, monitoring the accuracy and 

completeness of CSR’s responses to callers’ inquiries, and improving the 

usability of the Part D benefit portion of the Medicare Web site. In its 

comments on a draft of our report (see app. III), CMS said that it supports 

the goals of our recommendations and is already taking steps to 

implement them. However, CMS said that our findings did not present a 

complete and accurate picture of its Part D benefit communications 

activities. We believe that our report provides an accurate examination of 

the CMS communications mechanisms that have the greatest impact on 

beneficiaries (see app. IV). 

 5

The percentages related to the responses we received to our 500 calls exceed 100 percent 

due to rounding. 

Page 3 GAO-06-715T
CMS has undertaken steps to educate beneficiaries about the Part D 

benefit using written documents, a toll-free help line, and the Medicare 

Web site. To explain the Part D benefit to beneficiaries, CMS had 

produced more than 70 written documents as of December 2005. Medicare 

& You—the beneficiary handbook—is the most widely available and was 

sent directly to beneficiaries in October 2005. Other written documents 

were targeted to specific groups of beneficiaries, such as dual-eligible 

beneficiaries6

 and beneficiaries with Medicare Advantage or Medigap 

policies.7

Background 

Beneficiaries can obtain answers to questions about the Part D benefit by

calling the 1-800-MEDICARE help line. This help line, which is 

administered by CMS, was established in March 1999, to answer 

beneficiaries’ questions about the Medicare program. As of December 

2005, about 7,500 CSRs were handling calls on the help line, which 

operates 24 hours a day, 7 days a week, and is run by two CMS

contractors. CMS provides CSRs with detailed scripts to use in answering 

the questions. Call center contractors write the scripts, and CMS checks 

them for accuracy and completeness. 

In addition, CMS’s Medicare Web site provides information about various

aspects of the Medicare program. The Web site contains basic information 

about the Part D benefit, suggests factors for beneficiaries to consider 

when choosing plans and provides guidance on enrollment and plan 

selection. It also lists frequently asked questions, and allows users to view, 

print, or order publications. In addition, the site contains information on 

cost and coverage of individual plans. There is also a tool that allows 

beneficiaries to enroll directly in the plan they have chosen. 

 6

Dual-eligible beneficiaries are Medicare beneficiaries who are also eligible for Medicaid—

the federal-state health program for low-income individuals—and receive full Medicaid 

benefits for services not covered by Medicare. 

7

Medicare Advantage replaced the Medicare+Choice managed care program and expanded 

the availability of private health plan options to Medicare beneficiaries. Medigap policies

provide supplemental health coverage sold by private insurers to help pay for Medicare 

cost-sharing requirements, as well as for some services not provided by Medicare. 

Page 4 GAO-06-715T
Although the six sample documents we reviewed informed readers of 

enrollment steps and factors affecting coverage, they lacked clarity in two 

ways. First, about 40 percent of seniors read at or below the fifth-grade 

level, but the reading levels of the documents ranged from seventh grade 

to postcollege. As a result, these documents are challenging for many 

seniors. Even after adjusting the text for 26 multisyllabic words, such as 

Medicare, Medicare Advantage, and Social Security Administration, the 

estimated reading level ranged from seventh to twelfth grade, a reading 

level that would remain challenging for at least 40 percent of seniors. 

Second, on average, the six documents we reviewed did not comply with 

about half of the 60 commonly recognized guidelines for good

communications. For example, although the documents included concise 

and descriptive headings, they used too much technical jargon and often 

did not define difficult terms such as formulary.8

 The 11 beneficiaries and 

5 advisers we tested reported frustration with the documents’ lack of 

clarity as they encountered difficulties in understanding and attempting to 

complete 18 specified tasks. For example, none of these beneficiaries and 

only 2 of the advisers were able to complete the task of computing their 

projected total out-of-pocket costs for a plan that provided Part D 

standard coverage. Only one of 18 specified tasks was completed by all 

beneficiaries and advisers. Even those who were able to complete a given

task expressed confusion as they worked to comprehend the relevant text. 

Of the 500 calls we placed to CMS’s 1-800-MEDICARE help line regarding

the Part D benefit, CSRs answered about 67 percent of the calls accurately

and completely. Of the remainder, 18 percent of the calls received 

inaccurate responses, 8 percent of the responses were inappropriate given 

the question asked, and about 3 percent received incomplete responses. In 

addition, about 5 percent of our calls were not answered, primarily 

because of disconnections.9

Clarity of CMS 

Written Documents 

Could Be Improved 

Help Line Responses 

Frequently Complete 

and Accurate, but 

Varied By Question 

The accuracy and completeness of CSR responses varied significantly 

across our five questions. (See fig. 1.) For example, while CSRs provided 

accurate and complete responses to calls about beneficiaries’ eligibility for 

financial assistance 90 percent of the time, the accuracy rate for calls 

 8

A formulary is a list of prescription drugs covered by a health plan. 

9

The percentages related to the responses we received to our 500 calls exceed 100 percent 

because of rounding. 

Page 5 GAO-06-715T
concerning the drug plan that would cost the least for a beneficiary with 

specified prescription drug needs was 41 percent. CSRs inappropriately 

responded 35 percent of the time that this question could not be answered 

without personal identifying information—such as the beneficiary’s 

Medicare number or date of birth—even though the CSRs could have 

answered our question using CMS’s Web-based prescription drug plan 

finder tool. CSRs’ failure to read the correct script also contributed to 

inaccurate responses. The time GAO callers waited to speak with CSRs 

also varied, ranging from no wait time to over 55 minutes. For 75 percent 

of the calls—374 of the 500—the wait was less than 5 minutes. 

Figure 1: Variation in CSRs’ Responses for Individual Questions

Page 6 GAO-06-715T
We found that the Part D benefit portion of the Medicare Web site can be

difficult to use. In our evaluation of overall usability—the ease of finding 

needed information and performing various tasks—we found usability 

scores of 47 percent for seniors and 53 percent for younger adults, out of a 

possible 100 percent. While there is no widely accepted benchmark for 

usability, these scores indicate difficulties in using the site. For example, 

tools such as the drug plan finder were complicated to use, and forms that 

collect information on-line from users were difficult to correct if the user 

made an error. 

We also evaluated the usability of 137 detailed aspects of the Part D 

benefit portion of the site, including features of Web design and on-line 

tools, and found that 70 percent of these aspects could be expected to 

cause users confusion. For example, key functions of the prescription 

drug plan finder tool, such as the “continue” and “choose a drug plan” 

buttons, were often not visible on the page without scrolling down. In 

addition, the drug plan finder tool defaults—or is automatically reset—to 

generic drugs, which may complicate users’ search for drug plans covering 

brand name drugs. The material in this portion of the Web site is written at 

the 11th grade level, which can also present challenges to some users. 

Finally, in our evaluation of the ability of seven participants to collectively 

complete 34 user tests, we found that on average, participants were only 

able to proceed slightly more than half way though each test. When asked

about their experiences with using the Web site, the seven participants, on

average, indicated high levels of frustration and low levels of satisfaction. 

Within the past 6 months, millions of Medicare beneficiaries have been 

making important decisions about their prescription drug coverage and 

have needed access to information about the new Part D benefit to make 

appropriate choices. CMS faced a tremendous challenge in responding to 

this need and, within short time frames, developed a range of outreach and 

educational materials to inform beneficiaries and their advisers about the 

Part D benefit. To disseminate these materials, CMS largely added 

information to existing resources, including written documents, such as 

Medicare & You; the 1-800-MEDICARE help line; and the Medicare Web

site. However, CMS has not ensured that its communications to 

beneficiaries and their advisers are provided in a manner that is 

consistently clear, complete, accurate, and usable. Although the initial 

enrollment period for the Part D benefit will end on May 15, 2006, CMS 

will continue to play a pivotal role in providing beneficiaries with 

information about the drug benefit in the future. The recommendations we 

have made would help CMS to ensure that beneficiaries and their advisers

Part D Benefit Portion 

of Medicare Web Site 

Can Be Challenging to 

Use 

Concluding 

Observations 

Page 7 GAO-06-715T
are prepared when deciding whether to enroll in the benefit, and if 

enrolling, which drug plan to choose. 

Mr. Chairman, this concludes my prepared remarks. I would be happy to 

respond to any questions that you or other Members of the subcommittee 

may have at this time. 

For further information regarding this statement, please contact Leslie G. 

Aronovitz at (312) 220-7600. Contact points for our Offices of 

Congressional Relations and Public Affairs may be found in the last page 

of this statement. Susan T. Anthony and Geraldine Redican-Bigott, 

Assistant Directors; Shaunessye D. Curry; Helen T. Desaulniers; Margaret 

J. Weber; and Craig H. Winslow made key contributions to this statement. 

Contact and 

Acknowledgments 

Page 8 GAO-06-715T
Page 9 GAO-06-715T

Appendix I: Sample of CMS Written 

Documents Reviewed 

To assess the clarity, completeness, and accuracy of written documents, 

we compiled a list of all available CMS-issued Part D benefit publications

intended to inform beneficiaries and their advisers and selected a sample 

of 6 from the 70 CMS documents available, as of December 7, 2005, for indepth review, as shown in Table 1. The sample documents were chosen to 

represent a variety of publication types, such as frequently asked 

questions and fact sheets available to beneficiaries about the Part D 

benefit. We selected documents that targeted all beneficiaries or those 

with unique drug coverage concerns, such as dual-eligibles and 

beneficiaries with Medigap plans. 

Table 1: Sample of Six Selected Documents 

Document Target audience 

Medicare & You, Section 6: Medicare 

Prescription Drug Coverage

All beneficiaries

Things to Think about When You Compare 

Plans 

All beneficiaries

Frequently Asked Questions about: Retiree 

Prescription Drug Coverage & the New 

Medicare Prescription Drug Coverage 

Beneficiaries with employer or union 

coverage 

Introduction to the Auto-Enrollment Notice Dual-eligible beneficiariesa

Quick Facts about Medicare’s New 

Coverage for Prescription Drugs for People 

with a Medicare Health Plan with 

Prescription Drug Coverage

Beneficiaries with Medicare Advantageb

Do You Have a Medigap Policy with 

Prescription Drug Coverage? 

Beneficiaries with Medigapc

Source: GAO.

a

Dual-eligible beneficiaries are Medicare beneficiaries who receive full Medicaid benefits for services

not covered by Medicare. 

b

Medicare Advantage replaced the Medicare + Choice managed care program and expanded the 

availability of private health plan options to Medicare beneficiaries. 

c

Medigap policies provide supplemental health coverage sold by private insurers to help pay for 

Medicare cost-sharing requirements, as well as for some services not provided by Medicare. 
Appendix III: Comments from the 

Centers for Medicare & Medicaid’s Services 

Page 10 GAO-06-715T

Appendix II: Questions and Criteria Used to 

Evaluate Accuracy and Completeness of 

CSR’s Help Line Responses 

To determine the accuracy and completeness of information provided 

regarding the Part D benefit, we placed a total of 500 calls to the 1-800-

MEDICARE help line. We posed one of five questions about the Part D 

benefit in each call, so that each question was asked 100 times. Table 2 

summarizes the questions we asked and the criteria we used to evaluate 

the accuracy of responses. 

Table 2: Questions and Criteria Used to Evaluate Accuracy and Completeness 

Question GAO Asked MEDICARE help 

line CSRs 

Criteria GAO used to evaluate accuracy 

and completeness of CSR Responses 

1. What drug plan can a beneficiary get 

that will cover all of his/her [specified] 

drugs at a [specified] pharmacy, have a 

mail-order option; and cost the least 

amount annually with [or without] a 

deductible? 

An accurate and complete response would 

identify the prescription drug plan that has 

the lowest estimated annual cost for the 

drugs the beneficiary uses. 

2. Can a beneficiary who is in a nursing

home and not on Medicaid sign up for a 

prescription drug plan? 

An accurate and complete response would 

indicate that a beneficiary can choose 

whether to enroll in a Medicare prescription 

drug plan. 

3. Can a beneficiary enroll in the Medicare 

prescription drug program and keep 

his/her current Medigap policy?

An accurate and complete response would 

inform the caller that enrolling for the 

prescription drug benefit would depend on 

whether the beneficiary’s Medigap plan was 

creditable—that is, whether the coverage it 

provided was at least as good as 

Medicare’s standard prescription drug 

coverage—or noncreditable. The CSR 

response would also mention that the 

beneficiary’s Medigap plan should have 

sent him/her information that outlined 

options. 

4. What options does a beneficiary, who 

has retiree health insurance with 

prescription drug coverage that is not as 

good as the Medicare prescription drug 

coverage, have as it relates to the 

Medicare benefit? 

An accurate and complete response would 

indicate that a beneficiary has two options: 

(1) keep current health plan and join the 

prescription drug plan later with a penalty, 

or (2) drop current coverage and join a 

Medicare drug plan. 

5. How do I know if a beneficiary qualifies 

for financial assistance?

An accurate and complete response would 

refer the beneficiary to the Social Security 

Administration.

Source: GAO.
Appendix III: Comments from the Centers for 

Medicare & Medicaid’s Services 

Page 11 GAO-06-715T

Appendix III: Comments from the Centers for 

Medicare & Medicaid’s Services 
Appendix III: Comments from the Centers for 

Medicare & Medicaid’s Services 

Page 12 GAO-06-715T
Appendix III: Comments from the Centers for 

Medicare & Medicaid’s Services 

Page 13 GAO-06-715T
Appendix III: Comments from the Centers for 

Medicare & Medicaid’s Services 

Page 14 GAO-06-715T
Appendix III: Comments from the Centers for 

Medicare & Medicaid’s Services 

Page 15 GAO-06-715T
Appendix III: Comments from the Centers for 

Medicare & Medicaid’s Services 

Page 16 GAO-06-715T
Appendix III: Comments from the Centers for 

Medicare & Medicaid’s Services 

Page 17 GAO-06-715T
Appendix III: Comments from the Centers for 

Medicare & Medicaid’s Services 

Page 18 GAO-06-715T
Appendix III: Comments from the Centers for 

Medicare & Medicaid’s Services 

Page 19 GAO-06-715T
Appendix III: Comments from the Centers for 

Medicare & Medicaid’s Services 

Page 20 GAO-06-715T
Appendix III: Comments from the Centers for 

Medicare & Medicaid’s Services 

Page 21 GAO-06-715T
Appendix III: Comments from the Centers for 

Medicare & Medicaid’s Services 

Page 22 GAO-06-715T
Appendix III: Comments from the Centers for 

Medicare & Medicaid’s Services 

Page 23 GAO-06-715T
Appendix III: Comments from the Centers for 

Medicare & Medicaid’s Services 

Page 24 GAO-06-715T
Appendix III: Comments from the Centers for 

Medicare & Medicaid’s Services 

Page 25 GAO-06-715T
Appendix III: Comments from the Centers for 

Medicare & Medicaid’s Services 

Page 26 GAO-06-715T
Appendix IV: Agency Comments and Our 

Evaluation

Page 27 GAO-06-715T

Appendix IV: Agency Comments and Our 

Evaluation 

We received written comments on a draft of our report from CMS (see 

app. III). CMS said that it did not believe our findings presented a 

complete and accurate picture of its Part D communications activities. 

CMS discussed several concerns regarding our findings on its written 

documents and the 1-800-MEDICARE help line. However, CMS did not 

disagree with our findings regarding the Medicare Web site or the role of 

SHIPs. CMS also said that it supports the goals of our recommendations 

and is already taking steps to implement them, such as continually 

enhancing and refining its Web-based tools. 

CMS discussed concerns regarding the completeness and accuracy of our 

findings in terms of activities we did not examine, as well as those we did. 

CMS stated that our findings were not complete because our report did 

not examine all of the agency’s efforts to educate Medicare beneficiaries 

and specifically mentioned that we did not examine the broad array of 

communication tools it has made available, including the development of 

its network of grassroots partners throughout the country. We recognize 

that CMS has taken advantage of many vehicles to communicate with 

beneficiaries and their advisers. However, we focused our work on the 

four specific mechanisms that we believed would have the greatest impact 

on beneficiaries—written materials, the 1-800-MEDICARE help line, the 

Medicare Web site, and the SHIPs. In addition, CMS stated that our report 

is based on information from January and February 2006, and that it has 

undertaken a number of activities since then to address the problems we 

identified. Although we appreciate CMS’s efforts to improve its Part D 

communications to beneficiaries on an ongoing basis, we believe it is 

unlikely that the problems we identified in our report could have been 

corrected yet given their nature and scope. 

CMS raised two concerns with our examination of a sample of written 

materials. First, it criticized our use of readability tests to assess the clarity 

of the six sample documents we reviewed. For example, CMS said that 

common multisyllabic words would inappropriately inflate the reading 

level. However, we found that reading levels remained high after adjusting 

for 26 multisyllabic words a Medicare beneficiary would encounter, such 

as Social Security Administration. CMS also pointed out that some experts 

find such assessments to be misleading. Because we recognize that there

is some controversy surrounding the use of reading levels, we included 

two additional assessments to supplement this readability analysis—the 

assessment of design and organization of the sample documents based on

60 commonly recognized communications guidelines and an examination 

of the usability of six sample documents, involving 11 beneficiaries and 5

advisers. 
Appendix IV: Agency Comments and Our 

Evaluation

Second, CMS expressed concern about our examination of the usability of 

the six sample documents. The participating beneficiaries and advisers 

were called on to perform 18 specified tasks, after reading the selected

materials, including a section of the Medicare & You handbook. CMS 

suggested that the task asking beneficiaries and advisers to calculate their 

out-of-pocket drug costs was inappropriate because there are many other 

tools that can be used to more effectively compare costs. We do not 

disagree with CMS that there are a number of ways beneficiaries may 

complete this calculation; however, we nonetheless believe that it is 

important that beneficiaries be able to complete this task on the basis of 

reading Medicare & You, which, as CMS points out, is widely disseminated 

to beneficiaries, reaching all beneficiary households each year. In addition,

CMS noted that it was not able to examine our detailed methodology 

regarding the clarity of written materials—including assessments 

performed by one of our contractors concerning readability and document 

design and organization. We plan to share this information with CMS. 

Finally, CMS took issue with one aspect of our evaluation of the 1-800-

MEDICARE help line. Specifically, CMS said the 41 percent accuracy rate 

associated with one of the five questions we asked was misleading,

because, according to CMS, we failed to analyze 35 of the 100 responses. 

However, we disagree. This question addressed which drug plan would 

cost the least for a beneficiary with certain specified prescription drug 

needs. We analyzed these 35 responses to this question and found the 

responses to be inappropriate. The CSRs would not provide us with the 

information we were seeking because we did not supply personal 

identifying information, such as the beneficiary’s Medicare number or date 

of birth. We considered such responses inappropriate because the CSRs 

could have answered this question without personal identifying 

information by using CMS’s Web-based prescription drug plan finder tool. 

Although CMS said that it has emphasized to CSRs, through training and 

broadcast messages, that it is permissible to provide the information we 

requested without requiring information that would personally identify a 

beneficiary, in these 35 instances, the CSR simply told us that our question

could not be answered. CMS also said that the bulk of these inappropriate 

responses were related to our request that the CSR use only brand-name 

drugs. This is incorrect—none of these 35 responses were considered 

incorrect or inappropriate because of a request that the CSR use only 

brand-name drugs—as that was not part of our question. 

(290546) Page 28 GAO-06-715T
This is a work of the U.S. government and is not subject to copyright protection in the 

United States. It may be reproduced and distributed in its entirety without further 

permission from GAO. However, because this work may contain copyrighted images or 

other material, permission from the copyright holder may be necessary if you wish to 

reproduce this material separately. 
GAO’s Mission The Government Accountability Office, the audit, evaluation and 

investigative arm of Congress, exists to support Congress in meeting its 

constitutional responsibilities and to help improve the performance and 

accountability of the federal government for the American people. GAO 

examines the use of public funds; evaluates federal programs and policies; 

and provides analyses, recommendations, and other assistance to help

Congress make informed oversight, policy, and funding decisions. GAO’s 

commitment to good government is reflected in its core values of 

accountability, integrity, and reliability. 

The fastest and easiest way to obtain copies of GAO documents at no cost

is through GAO’s Web site (www.gao.gov). Each weekday, GAO posts 

newly released reports, testimony, and correspondence on its Web site. To

have GAO e-mail you a list of newly posted products every afternoon, go 

to www.gao.gov and select “Subscribe to Updates.” 

The first copy of each printed report is free. Additional copies are $2 each. 

A check or money order should be made out to the Superintendent of 

Documents. GAO also accepts VISA and Mastercard. Orders for 100 or 

more copies mailed to a single address are discounted 25 percent. Orders 

should be sent to: 

U.S. Government Accountability Office 

441 G Street NW, Room LM 

Washington, D.C. 20548 

To order by Phone: Voice: (202) 512-6000 

TDD: (202) 512-2537 

Fax: (202) 512-6061 

Contact: 

Web site: www.gao.gov/fraudnet/fraudnet.htm

E-mail: fraudnet@gao.gov

Automated answering system: (800) 424-5454 or (202) 512-7470 

Gloria Jarmon, Managing Director, JarmonG@gao.gov (202) 512-4400 

U.S. Government Accountability Office, 441 G Street NW, Room 7125 

Washington, D.C. 20548 

Paul Anderson, Managing Director, AndersonP1@gao.gov (202) 512-4800

U.S. Government Accountability Office, 441 G Street NW, Room 7149 

Washington, D.C. 20548 

Obtaining Copies of 

GAO Reports and 

Testimony

Order by Mail or Phone 

To Report Fraud,

Waste, and Abuse in 

Federal Programs 

Congressional 

Relations 

Public Affairs 

PRINTED ON RECYCLED PAPER