Medicare: Quality of CMS Communications to Beneficiaries on the Prescription Drug Benefit Could Be Improved
Released: 2006-05-04T00:00:00

Letter

Background

CMS has undertaken steps to educate beneficiaries about the Part D benefit using written documents, a toll-free help line, and the Medicare Web site. To explain the Part D benefit to beneficiaries, CMS had produced more than 70 written documents as of December 2005. Medicare & You—the beneficiary handbook—is the most widely available and was sent directly to beneficiaries in October 2005. Other written documents were targeted to specific groups of beneficiaries, such as dual-eligible beneficiaries and beneficiaries with Medicare Advantage or Medigap policies.

Beneficiaries can obtain answers to questions about the Part D benefit by calling the 1-800-MEDICARE help line. This help line, which is administered by CMS, was established in March 1999, to answer beneficiaries’ questions about the Medicare program. As of December 2005, about 7,500 CSRs were handling calls on the help line, which operates 24 hours a day, 7 days a week, and is run by two CMS contractors. CMS provides CSRs with detailed scripts to use in answering the questions. Call center contractors write the scripts, and CMS checks them for accuracy and completeness.

In addition, CMS’s Medicare Web site provides information about various aspects of the Medicare program. The Web site contains basic information about the Part D benefit, suggests factors for beneficiaries to consider when choosing plans and provides guidance on enrollment and plan selection. It also lists frequently asked questions, and allows users to view, print, or order publications. In addition, the site contains information on cost and coverage of individual plans. There is also a tool that allows beneficiaries to enroll directly in the plan they have chosen.

Clarity of CMS Written Documents Could Be Improved

Although the six sample documents we reviewed informed readers of enrollment steps and factors affecting coverage, they lacked clarity in two ways. First, about 40 percent of seniors read at or below the fifth-grade level, but the reading levels of the documents ranged from seventh grade to postcollege. As a result, these documents are challenging for many seniors. Even after adjusting the text for 26 multisyllabic words, such as Medicare, Medicare Advantage, and Social Security Administration, the estimated reading level ranged from seventh to twelfth grade, a reading level that would remain challenging for at least 40 percent of seniors.

Second, on average, the six documents we reviewed did not comply with about half of the 60 commonly recognized guidelines for good communications. For example, although the documents included concise and descriptive headings, they used too much technical jargon and often did not define difficult terms such as formulary. The 11 beneficiaries and 5 advisers we tested reported frustration with the documents’ lack of clarity as they encountered difficulties in understanding and attempting to complete 18 specified tasks. For example, none of these beneficiaries and only 2 of the advisers were able to complete the task of computing their projected total out-of-pocket costs for a plan that provided Part D standard coverage. Only one of 18 specified tasks was completed by all beneficiaries and advisers. Even those who were able to complete a given task expressed confusion as they worked to comprehend the relevant text.

Help Line Responses Frequently Complete and Accurate, but Varied By Question

Of the 500 calls we placed to CMS’s 1-800-MEDICARE help line regarding the Part D benefit, CSRs answered about 67 percent of the calls accurately and completely. Of the remainder, 18 percent of the calls received inaccurate responses, 8 percent of the responses were inappropriate given the question asked, and about 3 percent received incomplete responses. In addition, about 5 percent of our calls were not answered, primarily because of disconnections.

The accuracy and completeness of CSR responses varied significantly across our five questions. (See fig. 1.) For example, while CSRs provided accurate and complete responses to calls about beneficiaries’ eligibility for financial assistance 90 percent of the time, the accuracy rate for calls concerning the drug plan that would cost the least for a beneficiary with specified prescription drug needs was 41 percent. CSRs inappropriately responded 35 percent of the time that this question could not be answered without personal identifying information—such as the beneficiary’s Medicare number or date of birth—even though the CSRs could have answered our question using CMS’s Web-based prescription drug plan finder tool. CSRs’ failure to read the correct script also contributed to inaccurate responses. The time GAO callers waited to speak with CSRs also varied, ranging from no wait time to over 55 minutes. For 75 percent of the calls—374 of the 500—the wait was less than 5 minutes.

Part D Benefit Portion of Medicare Web Site Can Be Challenging to Use

We found that the Part D benefit portion of the Medicare Web site can be difficult to use. In our evaluation of overall usability—the ease of finding needed information and performing various tasks—we found usability scores of 47 percent for seniors and 53 percent for younger adults, out of a possible 100 percent. While there is no widely accepted benchmark for usability, these scores indicate difficulties in using the site. For example, tools such as the drug plan finder were complicated to use, and forms that collect information on-line from users were difficult to correct if the user made an error.

We also evaluated the usability of 137 detailed aspects of the Part D benefit portion of the site, including features of Web design and on-line tools, and found that 70 percent of these aspects could be expected to cause users confusion. For example, key functions of the prescription drug plan finder tool, such as the “continue” and “choose a drug plan” buttons, were often not visible on the page without scrolling down. In addition, the drug plan finder tool defaults—or is automatically reset—to generic drugs, which may complicate users’ search for drug plans covering brand name drugs. The material in this portion of the Web site is written at the 11th grade level, which can also present challenges to some users. Finally, in our evaluation of the ability of seven participants to collectively complete 34 user tests, we found that on average, participants were only able to proceed slightly more than half way though each test. When asked about their experiences with using the Web site, the seven participants, on average, indicated high levels of frustration and low levels of satisfaction.

Concluding Observations

Within the past 6 months, millions of Medicare beneficiaries have been making important decisions about their prescription drug coverage and have needed access to information about the new Part D benefit to make appropriate choices. CMS faced a tremendous challenge in responding to this need and, within short time frames, developed a range of outreach and educational materials to inform beneficiaries and their advisers about the Part D benefit. To disseminate these materials, CMS largely added information to existing resources, including written documents, such as Medicare & You; the 1-800-MEDICARE help line; and the Medicare Web site. However, CMS has not ensured that its communications to beneficiaries and their advisers are provided in a manner that is consistently clear, complete, accurate, and usable. Although the initial enrollment period for the Part D benefit will end on May 15, 2006, CMS will continue to play a pivotal role in providing beneficiaries with information about the drug benefit in the future. The recommendations we have made would help CMS to ensure that beneficiaries and their advisers are prepared when deciding whether to enroll in the benefit, and if enrolling, which drug plan to choose.

Mr. Chairman, this concludes my prepared remarks. I would be happy to respond to any questions that you or other Members of the subcommittee may have at this time.

Contact and Acknowledgments

For further information regarding this statement, please contact Leslie G. Aronovitz at (312) 220-7600. Contact points for our Offices of Congressional Relations and Public Affairs may be found in the last page of this statement. Susan T. Anthony and Geraldine Redican-Bigott, Assistant Directors; Shaunessye D. Curry; Helen T. Desaulniers; Margaret J. Weber; and Craig H. Winslow made key contributions to this statement.

Appendix I: Sample of CMS Written Documents Reviewed

To assess the clarity, completeness, and accuracy of written documents, we compiled a list of all available CMS-issued Part D benefit publications intended to inform beneficiaries and their advisers and selected a sample of 6 from the 70 CMS documents available, as of December 7, 2005, for in- depth review, as shown in Table 1. The sample documents were chosen to represent a variety of publication types, such as frequently asked questions and fact sheets available to beneficiaries about the Part D benefit. We selected documents that targeted all beneficiaries or those with unique drug coverage concerns, such as dual-eligibles and beneficiaries with Medigap plans.

Appendix II: Questions and Criteria Used to Evaluate Accuracy and Completeness of CSR’s Help Line Responses

To determine the accuracy and completeness of information provided regarding the Part D benefit, we placed a total of 500 calls to the 1-800- MEDICARE help line. We posed one of five questions about the Part D benefit in each call, so that each question was asked 100 times. Table 2 summarizes the questions we asked and the criteria we used to evaluate the accuracy of responses.

Appendix III: Comments from the Centers for Medicare & Medicaid’s Services

Appendix IV: Agency Comments and Our Evaluation

We received written comments on a draft of our report from CMS (see app. III). CMS said that it did not believe our findings presented a complete and accurate picture of its Part D communications activities. CMS discussed several concerns regarding our findings on its written documents and the 1-800-MEDICARE help line. However, CMS did not disagree with our findings regarding the Medicare Web site or the role of SHIPs. CMS also said that it supports the goals of our recommendations and is already taking steps to implement them, such as continually enhancing and refining its Web-based tools.

CMS discussed concerns regarding the completeness and accuracy of our findings in terms of activities we did not examine, as well as those we did. CMS stated that our findings were not complete because our report did not examine all of the agency’s efforts to educate Medicare beneficiaries and specifically mentioned that we did not examine the broad array of communication tools it has made available, including the development of its network of grassroots partners throughout the country. We recognize that CMS has taken advantage of many vehicles to communicate with beneficiaries and their advisers. However, we focused our work on the four specific mechanisms that we believed would have the greatest impact on beneficiaries—written materials, the 1-800-MEDICARE help line, the Medicare Web site, and the SHIPs. In addition, CMS stated that our report is based on information from January and February 2006, and that it has undertaken a number of activities since then to address the problems we identified. Although we appreciate CMS’s efforts to improve its Part D communications to beneficiaries on an ongoing basis, we believe it is unlikely that the problems we identified in our report could have been corrected yet given their nature and scope.

CMS raised two concerns with our examination of a sample of written materials. First, it criticized our use of readability tests to assess the clarity of the six sample documents we reviewed. For example, CMS said that common multisyllabic words would inappropriately inflate the reading level. However, we found that reading levels remained high after adjusting for 26 multisyllabic words a Medicare beneficiary would encounter, such as Social Security Administration. CMS also pointed out that some experts find such assessments to be misleading. Because we recognize that there is some controversy surrounding the use of reading levels, we included two additional assessments to supplement this readability analysis—the assessment of design and organization of the sample documents based on 60 commonly recognized communications guidelines and an examination of the usability of six sample documents, involving 11 beneficiaries and 5 advisers.

Second, CMS expressed concern about our examination of the usability of the six sample documents. The participating beneficiaries and advisers were called on to perform 18 specified tasks, after reading the selected materials, including a section of the Medicare & You handbook. CMS suggested that the task asking beneficiaries and advisers to calculate their out-of-pocket drug costs was inappropriate because there are many other tools that can be used to more effectively compare costs. We do not disagree with CMS that there are a number of ways beneficiaries may complete this calculation; however, we nonetheless believe that it is important that beneficiaries be able to complete this task on the basis of reading Medicare & You, which, as CMS points out, is widely disseminated to beneficiaries, reaching all beneficiary households each year. In addition, CMS noted that it was not able to examine our detailed methodology regarding the clarity of written materials—including assessments performed by one of our contractors concerning readability and document design and organization. We plan to share this information with CMS.

Finally, CMS took issue with one aspect of our evaluation of the 1-800- MEDICARE help line. Specifically, CMS said the 41 percent accuracy rate associated with one of the five questions we asked was misleading, because, according to CMS, we failed to analyze 35 of the 100 responses. However, we disagree. This question addressed which drug plan would cost the least for a beneficiary with certain specified prescription drug needs. We analyzed these 35 responses to this question and found the responses to be inappropriate. The CSRs would not provide us with the information we were seeking because we did not supply personal identifying information, such as the beneficiary’s Medicare number or date of birth. We considered such responses inappropriate because the CSRs could have answered this question without personal identifying information by using CMS’s Web-based prescription drug plan finder tool. Although CMS said that it has emphasized to CSRs, through training and broadcast messages, that it is permissible to provide the information we requested without requiring information that would personally identify a beneficiary, in these 35 instances, the CSR simply told us that our question could not be answered. CMS also said that the bulk of these inappropriate responses were related to our request that the CSR use only brand-name drugs. This is incorrect—none of these 35 responses were considered incorrect or inappropriate because of a request that the CSR use only brand-name drugs—as that was not part of our question.

This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. However, because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately.