Medicare: Quality of CMS Communications to Beneficiaries on the Prescription Drug Benefit Could Be Improved
Published: 2006-05-04T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: Today's hearing focuses on Medicare Part D, the program's new outpatient prescription drug benefit. On January 1, 2006, Medicare began providing this benefit, and beneficiaries have until May 15, 2006, to enroll without the risk of penalties. The Centers for Medicare & Medicaid Services (CMS), which administers the Part D benefit, has undertaken outreach and education efforts to inform beneficiaries and their advisers. GAO was asked to discuss how CMS can better ensure that Medicare beneficiaries are informed about the Part D benefit. This testimony is based on Medicare: CMS Communications to Beneficiaries on the Prescription Drug Benefit Could Be Improved, GAO-06-654 (May 3, 2006).
section_title: What GAO Found
paragraphs: Information given in the six sample documents that GAO reviewed describing the Part D benefit was largely complete and accurate, although this information lacked clarity. First, about 40 percent of seniors read at or below the fifth-grade level, but the reading levels of these documents ranged from seventh grade to postcollege. Second, on average, the six documents we reviewed did not comply with about half of 60 common guidelines for good communication. For example, the documents used too much technical jargon and often did not define difficult terms. Moreover, 16 beneficiaries and advisers that GAO tested reported frustration with the documents' lack of clarity and had difficulty completing the tasks assigned to them. Customer service representatives (CSRs) answered about two-thirds of the 500 calls GAO placed to CMS's 1-800-MEDICARE help line accurately and completely. Of the remainder, 18 percent of the calls received inaccurate responses, 8 percent of the responses were inappropriate given the question asked, and about 3 percent received incomplete responses. In addition, about 5 percent of GAO's calls were not answered, primarily because of disconnections. The accuracy and completeness of CSRs' responses varied significantly across the five questions. For example, while CSRs provided accurate and complete responses to calls about beneficiaries' eligibility for financial assistance 90 percent of the time, the accuracy rate for calls concerning the drug plan that would cost the least for a beneficiary with specified prescription drug needs was 41 percent. For this question, the CSRs responded inappropriately for 35 percent of the calls by explaining that they could not identify the least costly plan without the beneficiary's personal information--even though CSRs had the information needed to answer the question. The time GAO callers waited to speak with CSRs also varied, ranging from no wait time to over 55 minutes. For 75 percent of the calls--374 of the 500--the wait was less than 5 minutes. The Part D benefit portion of the Medicare Web site can be difficult to use. GAO's test of the site's overall usability--the ease of finding needed information and performing various tasks--resulted in scores of 47 percent for seniors and 53 percent for younger adults, out of a possible 100 percent. While there is no widely accepted benchmark for usability, these scores indicate that using the site can be difficult. For example, the prescription drug plan finder was complicated to use and some of its key functions, such as "continue" and "choose a drug plan," were often not visible on the page without scrolling down.

section_title: Letter
paragraphs: I am pleased to be here today as you discuss the Medicare outpatient  prescription drug benefit, known as the Part D benefit, which was  established by the Medicare Prescription Drug, Improvement, and  Modernization Act of 2003 with coverage beginning on January 1, 2006.  Until this time, Medicare, the program that finances health care benefits  for about 42 million elderly and disabled beneficiaries, had not generally  provided coverage for outpatient prescription drugs. Beneficiaries who opt  to enroll in Part D may choose a drug plan from those offered by private  plan sponsors under contract to the Centers for Medicare & Medicaid  Services (CMS), which administers the Part D benefit. These plans differ in  the drugs covered, pharmacies used, and enrollee costs. As of   April 20, 2006, more than 30 million of Medicare’s 42 million beneficiaries  were enrolled in a Part D plan or had other outpatient prescription drug  coverage. Beneficiaries have until the end of the current enrollment  period, May 15, 2006, to enroll in the Part D benefit and select a plan  without the risk of penalties in the form of higher premiums.
Given the newness and complexity of the Part D benefit, it is critical that  beneficiaries and their advisers, including members of their families,  understand the options available to them. Understanding these options  enables beneficiaries to make informed decisions on whether to enroll in  the Part D benefit, and if they decide to enroll, which drug plan to choose.  As part of its responsibilities, CMS has undertaken efforts to provide  beneficiaries and their advisers with the information they need about the  Part D benefit through various media, including written documents, the   1-800-MEDICARE help line, and the Medicare Web site. CMS’s education  efforts are important because widespread confusion has been reported  among beneficiaries about the costs and coverage under the new benefit.
You and others have expressed interest in ensuring that Medicare  beneficiaries receive the information they need to make informed  decisions. My remarks today will focus on (1) the extent to which CMS’s  written documents describe the Part D benefit in a clear, complete, and  accurate manner; (2) the effectiveness of CMS’s 1-800-MEDICARE help  line in providing accurate, complete, and prompt responses to callers  inquiring about the Part D benefit; and (3) whether CMS’s Medicare Web  site presents information on the Part D benefit in a usable manner. My  testimony will summarize findings of a report we released yesterday that  examines CMS’s Medicare Part D benefit communications to beneficiaries  in more detail and includes recommendations to the CMS Administrator  for improving the quality of the agency’s Part D benefit education and  outreach materials.
To address these issues, we interviewed CMS officials responsible for  written documents about the Part D benefit, the 1-800-MEDICARE help  line, and the Medicare Web site. To assess the clarity, completeness, and  accuracy of written documents, we performed an in-depth review of a  sample of six CMS documents describing the Part D benefit, including  contracting with the American Institutes for Research (AIR), a firm with  experience in evaluating written documents, to assess their clarity. (See  app. I for a list of written documents reviewed.) We also placed 500 calls  to 1-800-MEDICARE, posing one of five questions related to the Part D  benefit in each call so that each question was asked 100 times. We  evaluated the accuracy and completeness of customer service  representatives’ (CSR) responses to these questions. (See app. II for the  questions and criteria we used to evaluate the accuracy and completeness  of CSR responses to calls we made to 1-800 MEDICARE.) To assess the  usability of the Part D benefit information available on the Medicare Web  site, we contracted with the Nielsen Norman Group (NN/g), an expert on  Web design. We conducted our work from November 2005 through   May 2006, in accordance with generally accepted government auditing  standards.
In summary, CMS successfully developed a large volume of information  about the new Part D benefit and made it available to beneficiaries  through a variety of sources, despite the challenge of developing this  information within a short time frame. However, the quality of CMS’s  communications to beneficiaries and their advisers about the Part D  benefit could be improved. For example, although the six CMS written  documents we reviewed were largely accurate and complete, they often  lacked clarity. Specifically, while about 40 percent of seniors read at or  below the fifth-grade level, the reading levels of the documents ranged  from seventh grade to postcollege. Moreover, the six documents used too  much technical jargon and often did not define difficult terms. Similarly,  although 67 percent of the responses to the 500 calls we placed to CMS’s   1-800-MEDICARE help line regarding the Part D benefit were accurate and  complete, we nonetheless received a substantial number of responses that  were not. Eighteen percent of the calls received inaccurate responses,   8 percent of the responses were inappropriate given the question asked,  about 3 percent received incomplete responses, and about 5 percent of our  calls were not answered, primarily due to disconnections. In addition, our  review of the Part D benefit portion of the Medicare Web site showed that  this site can be difficult to use. In usability tests that examined the ease of  finding needed information and performing various tasks, we found that,  for overall usability, the Web site scored 47 percent for seniors and   53 percent for younger adults, out of a possible 100 percent. While there is  no widely accepted benchmark for usability, these scores indicate that  using the site can be difficult. Therefore, in the report we issued yesterday,  we made specific recommendations to the CMS Administrator to enhance  the quality of the agency’s communications on the Part D benefit,  including clarifying written materials, monitoring the accuracy and  completeness of CSR’s responses to callers’ inquiries, and improving the  usability of the Part D benefit portion of the Medicare Web site. In its  comments on a draft of our report (see app. III), CMS said that it supports  the goals of our recommendations and is already taking steps to  implement them. However, CMS said that our findings did not present a  complete and accurate picture of its Part D benefit communications  activities. We believe that our report provides an accurate examination of  the CMS communications mechanisms that have the greatest impact on  beneficiaries (see app. IV).
subsections: section_title: Background
paragraphs: CMS has undertaken steps to educate beneficiaries about the Part D  benefit using written documents, a toll-free help line, and the Medicare  Web site. To explain the Part D benefit to beneficiaries, CMS had  produced more than 70 written documents as of December 2005. Medicare  & You—the beneficiary handbook—is the most widely available and was  sent directly to beneficiaries in October 2005. Other written documents  were targeted to specific groups of beneficiaries, such as dual-eligible  beneficiaries and beneficiaries with Medicare Advantage or Medigap  policies.
Beneficiaries can obtain answers to questions about the Part D benefit by  calling the 1-800-MEDICARE help line. This help line, which is  administered by CMS, was established in March 1999, to answer  beneficiaries’ questions about the Medicare program. As of December  2005, about 7,500 CSRs were handling calls on the help line, which  operates 24 hours a day, 7 days a week, and is run by two CMS  contractors. CMS provides CSRs with detailed scripts to use in answering  the questions. Call center contractors write the scripts, and CMS checks  them for accuracy and completeness.
In addition, CMS’s Medicare Web site provides information about various  aspects of the Medicare program. The Web site contains basic information  about the Part D benefit, suggests factors for beneficiaries to consider  when choosing plans and provides guidance on enrollment and plan  selection. It also lists frequently asked questions, and allows users to view,  print, or order publications. In addition, the site contains information on  cost and coverage of individual plans. There is also a tool that allows  beneficiaries to enroll directly in the plan they have chosen.
section_title: Clarity of CMS Written Documents Could Be Improved
paragraphs: Although the six sample documents we reviewed informed readers of  enrollment steps and factors affecting coverage, they lacked clarity in two  ways. First, about 40 percent of seniors read at or below the fifth-grade  level, but the reading levels of the documents ranged from seventh grade  to postcollege. As a result, these documents are challenging for many  seniors. Even after adjusting the text for 26 multisyllabic words, such as  Medicare, Medicare Advantage, and Social Security Administration, the  estimated reading level ranged from seventh to twelfth grade, a reading  level that would remain challenging for at least 40 percent of seniors.
Second, on average, the six documents we reviewed did not comply with  about half of the 60 commonly recognized guidelines for good  communications. For example, although the documents included concise  and descriptive headings, they used too much technical jargon and often  did not define difficult terms such as formulary. The 11 beneficiaries and   5 advisers we tested reported frustration with the documents’ lack of  clarity as they encountered difficulties in understanding and attempting to  complete 18 specified tasks. For example, none of these beneficiaries and  only 2 of the advisers were able to complete the task of computing their  projected total out-of-pocket costs for a plan that provided Part D  standard coverage. Only one of 18 specified tasks was completed by all  beneficiaries and advisers. Even those who were able to complete a given  task expressed confusion as they worked to comprehend the relevant text.
section_title: Help Line Responses Frequently Complete and Accurate, but Varied By Question
paragraphs: Of the 500 calls we placed to CMS’s 1-800-MEDICARE help line regarding  the Part D benefit, CSRs answered about 67 percent of the calls accurately  and completely. Of the remainder, 18 percent of the calls received  inaccurate responses, 8 percent of the responses were inappropriate given  the question asked, and about 3 percent received incomplete responses. In  addition, about 5 percent of our calls were not answered, primarily  because of disconnections.
The accuracy and completeness of CSR responses varied significantly  across our five questions. (See fig. 1.) For example, while CSRs provided  accurate and complete responses to calls about beneficiaries’ eligibility for  financial assistance 90 percent of the time, the accuracy rate for calls  concerning the drug plan that would cost the least for a beneficiary with  specified prescription drug needs was 41 percent. CSRs inappropriately  responded 35 percent of the time that this question could not be answered  without personal identifying information—such as the beneficiary’s  Medicare number or date of birth—even though the CSRs could have  answered our question using CMS’s Web-based prescription drug plan  finder tool. CSRs’ failure to read the correct script also contributed to  inaccurate responses. The time GAO callers waited to speak with CSRs  also varied, ranging from no wait time to over 55 minutes. For 75 percent  of the calls—374 of the 500—the wait was less than 5 minutes.
section_title: Part D Benefit Portion of Medicare Web Site Can Be Challenging to Use
paragraphs: We found that the Part D benefit portion of the Medicare Web site can be  difficult to use. In our evaluation of overall usability—the ease of finding  needed information and performing various tasks—we found usability  scores of 47 percent for seniors and 53 percent for younger adults, out of a  possible 100 percent. While there is no widely accepted benchmark for  usability, these scores indicate difficulties in using the site. For example,  tools such as the drug plan finder were complicated to use, and forms that  collect information on-line from users were difficult to correct if the user  made an error.
We also evaluated the usability of 137 detailed aspects of the Part D  benefit portion of the site, including features of Web design and on-line  tools, and found that 70 percent of these aspects could be expected to  cause users confusion. For example, key functions of the prescription  drug plan finder tool, such as the “continue” and “choose a drug plan”  buttons, were often not visible on the page without scrolling down. In  addition, the drug plan finder tool defaults—or is automatically reset—to  generic drugs, which may complicate users’ search for drug plans covering  brand name drugs. The material in this portion of the Web site is written at  the 11th grade level, which can also present challenges to some users.  Finally, in our evaluation of the ability of seven participants to collectively  complete 34 user tests, we found that on average, participants were only  able to proceed slightly more than half way though each test. When asked  about their experiences with using the Web site, the seven participants, on  average, indicated high levels of frustration and low levels of satisfaction.
section_title: Concluding Observations
paragraphs: Within the past 6 months, millions of Medicare beneficiaries have been  making important decisions about their prescription drug coverage and  have needed access to information about the new Part D benefit to make  appropriate choices. CMS faced a tremendous challenge in responding to  this need and, within short time frames, developed a range of outreach and  educational materials to inform beneficiaries and their advisers about the  Part D benefit. To disseminate these materials, CMS largely added  information to existing resources, including written documents, such as  Medicare & You; the 1-800-MEDICARE help line; and the Medicare Web  site. However, CMS has not ensured that its communications to  beneficiaries and their advisers are provided in a manner that is  consistently clear, complete, accurate, and usable. Although the initial  enrollment period for the Part D benefit will end on May 15, 2006, CMS  will continue to play a pivotal role in providing beneficiaries with  information about the drug benefit in the future. The recommendations we  have made would help CMS to ensure that beneficiaries and their advisers  are prepared when deciding whether to enroll in the benefit, and if  enrolling, which drug plan to choose.
Mr. Chairman, this concludes my prepared remarks. I would be happy to  respond to any questions that you or other Members of the subcommittee  may have at this time.
section_title: Contact and Acknowledgments
paragraphs: For further information regarding this statement, please contact Leslie G.  Aronovitz at (312) 220-7600. Contact points for our Offices of  Congressional Relations and Public Affairs may be found in the last page  of this statement. Susan T. Anthony and Geraldine Redican-Bigott,  Assistant Directors; Shaunessye D. Curry; Helen T. Desaulniers; Margaret  J. Weber; and Craig H. Winslow made key contributions to this statement.
section_title: Appendix I: Sample of CMS Written Documents Reviewed
paragraphs: To assess the clarity, completeness, and accuracy of written documents,  we compiled a list of all available CMS-issued Part D benefit publications  intended to inform beneficiaries and their advisers and selected a sample  of 6 from the 70 CMS documents available, as of December 7, 2005, for in- depth review, as shown in Table 1. The sample documents were chosen to  represent a variety of publication types, such as frequently asked  questions and fact sheets available to beneficiaries about the Part D  benefit. We selected documents that targeted all beneficiaries or those  with unique drug coverage concerns, such as dual-eligibles and  beneficiaries with Medigap plans.
section_title: Appendix II: Questions and Criteria Used to Evaluate Accuracy and Completeness of CSR’s Help Line Responses
paragraphs: To determine the accuracy and completeness of information provided  regarding the Part D benefit, we placed a total of 500 calls to the 1-800- MEDICARE help line. We posed one of five questions about the Part D  benefit in each call, so that each question was asked 100 times. Table 2  summarizes the questions we asked and the criteria we used to evaluate  the accuracy of responses.
section_title: Appendix III: Comments from the Centers for Medicare & Medicaid’s Services
section_title: Appendix IV: Agency Comments and Our Evaluation
paragraphs: We received written comments on a draft of our report from CMS (see  app. III). CMS said that it did not believe our findings presented a  complete and accurate picture of its Part D communications activities.  CMS discussed several concerns regarding our findings on its written  documents and the 1-800-MEDICARE help line. However, CMS did not  disagree with our findings regarding the Medicare Web site or the role of  SHIPs. CMS also said that it supports the goals of our recommendations  and is already taking steps to implement them, such as continually  enhancing and refining its Web-based tools.
CMS discussed concerns regarding the completeness and accuracy of our  findings in terms of activities we did not examine, as well as those we did.  CMS stated that our findings were not complete because our report did  not examine all of the agency’s efforts to educate Medicare beneficiaries  and specifically mentioned that we did not examine the broad array of  communication tools it has made available, including the development of  its network of grassroots partners throughout the country. We recognize  that CMS has taken advantage of many vehicles to communicate with  beneficiaries and their advisers. However, we focused our work on the  four specific mechanisms that we believed would have the greatest impact  on beneficiaries—written materials, the 1-800-MEDICARE help line, the  Medicare Web site, and the SHIPs. In addition, CMS stated that our report  is based on information from January and February 2006, and that it has  undertaken a number of activities since then to address the problems we  identified. Although we appreciate CMS’s efforts to improve its Part D  communications to beneficiaries on an ongoing basis, we believe it is  unlikely that the problems we identified in our report could have been  corrected yet given their nature and scope.
CMS raised two concerns with our examination of a sample of written  materials. First, it criticized our use of readability tests to assess the clarity  of the six sample documents we reviewed. For example, CMS said that  common multisyllabic words would inappropriately inflate the reading  level. However, we found that reading levels remained high after adjusting  for 26 multisyllabic words a Medicare beneficiary would encounter, such  as Social Security Administration. CMS also pointed out that some experts  find such assessments to be misleading. Because we recognize that there  is some controversy surrounding the use of reading levels, we included  two additional assessments to supplement this readability analysis—the  assessment of design and organization of the sample documents based on  60 commonly recognized communications guidelines and an examination  of the usability of six sample documents, involving 11 beneficiaries and 5  advisers.
Second, CMS expressed concern about our examination of the usability of  the six sample documents. The participating beneficiaries and advisers  were called on to perform 18 specified tasks, after reading the selected  materials, including a section of the Medicare & You handbook. CMS  suggested that the task asking beneficiaries and advisers to calculate their  out-of-pocket drug costs was inappropriate because there are many other  tools that can be used to more effectively compare costs. We do not  disagree with CMS that there are a number of ways beneficiaries may  complete this calculation; however, we nonetheless believe that it is  important that beneficiaries be able to complete this task on the basis of  reading Medicare & You, which, as CMS points out, is widely disseminated  to beneficiaries, reaching all beneficiary households each year. In addition,  CMS noted that it was not able to examine our detailed methodology  regarding the clarity of written materials—including assessments  performed by one of our contractors concerning readability and document  design and organization. We plan to share this information with CMS.
Finally, CMS took issue with one aspect of our evaluation of the 1-800- MEDICARE help line. Specifically, CMS said the 41 percent accuracy rate  associated with one of the five questions we asked was misleading,  because, according to CMS, we failed to analyze 35 of the 100 responses.  However, we disagree. This question addressed which drug plan would  cost the least for a beneficiary with certain specified prescription drug  needs. We analyzed these 35 responses to this question and found the  responses to be inappropriate. The CSRs would not provide us with the  information we were seeking because we did not supply personal  identifying information, such as the beneficiary’s Medicare number or date  of birth. We considered such responses inappropriate because the CSRs  could have answered this question without personal identifying  information by using CMS’s Web-based prescription drug plan finder tool.  Although CMS said that it has emphasized to CSRs, through training and  broadcast messages, that it is permissible to provide the information we  requested without requiring information that would personally identify a  beneficiary, in these 35 instances, the CSR simply told us that our question  could not be answered. CMS also said that the bulk of these inappropriate  responses were related to our request that the CSR use only brand-name  drugs. This is incorrect—none of these 35 responses were considered  incorrect or inappropriate because of a request that the CSR use only  brand-name drugs—as that was not part of our question.
This is a work of the U.S. government and is not subject to copyright protection in the  United States. It may be reproduced and distributed in its entirety without further  permission from GAO. However, because this work may contain copyrighted images or  other material, permission from the copyright holder may be necessary if you wish to  reproduce this material separately.