CMS develops regulations and policies to implement the statutory provisions governing the Medicare program, and it communicates the information to providers primarily through its Medicare contractors. The contractors share information with providers through their Web sites, written bulletins, and carrier call centers. To respond to inquiries from providers, carriers operate 34 call centers. Most of the carrier call centers--31 of 34--are "blended," that is they respond to inquiries from beneficiaries as well as from providers. In most cases, CSRs at "blended" call centers answer calls from both providers and beneficiaries. More than 21 million inquiries were made to carrier call centers in fiscal year 2003. The vast majority of these were "status-oriented" calls. Typically status-oriented calls are relatively simple to answer and involve inquiries concerning the status of a claim or confirmation of an individual's eligibility for Medicare. Such calls generally do not require CSRs to provide callers with complex information. On the other end of the spectrum are "policy-oriented" questions, which involve more complicated issues, such as billing rules, covered services, and medical policies. The remaining calls include elements of both status-oriented and policy- oriented inquiries. For example, a provider may call to learn why a claim was denied. In some instances, the explanation may be simple, such as the claim form omitted necessary information. In others, the reason for the denial may be more complex and involve an assessment of whether the particular circumstance required to obtain Medicare payment was met. Although CMS requires that carriers report data that categorize calls by type, these categories are not standardized, and carriers differ in the criteria they use to define call type. CMS officials estimate the volume of calls received by CSRs involving policy-oriented questions to have been approximately 500,000 in fiscal year 2003. In fiscal year 2001, CMS required that all carrier call centers install automated voice response systems. The interactive voice response (IVR) unit allows providers to use their telephone keypads to respond to automated prompts and obtain status-oriented information without speaking to CSRs. Use of the IVR has been growing, and in fiscal year 2003, the automated system handled more than 52 percent of provider inquiries answered at carrier call centers. CSRs are available to respond to inquiries that providers believe are beyond the capability of the IVR, including policy-oriented calls. Generally, calls to CSRs are electronically routed by the carriers' automated systems based on CSR availability. The routing process does not consider the nature and complexity of the question or the expertise of the CSRs. Despite the diversion of many calls to the IVR, CSRs recently experienced an increase in the number of calls that they answer. According to CMS, the number of calls CSRs answered increased from 9 million in fiscal year 2002 to 10 million in fiscal year 2003. CSRs responding to status-oriented calls can typically access the relevant claims or enrollment information to respond to the inquiry. In addition, to assist CSRs, carriers have developed scripted responses to answer standard questions from beneficiaries, such as how to enroll in Medicare. To respond to inquiries from providers, CSRs may use FAQs posted on their carriers' Web sites. In addition, they may search a variety of other sources, including CMS's Web sites. If CSRs cannot locate information to respond to a provider's question, they may arrange to contact the provider--after conferring with a specialist. In addition, CMS requires each carrier to analyze provider inquiry data and develop a list of questions most frequently asked and areas of concern or confusion for providers. They must also tally problem areas identified when providers submit erroneous claims for payment. Each quarter, carriers report the 10 most frequent inquiries and claim submission errors to CMS and update the list of FAQs on their own Web sites. Because the nature of calls may vary by carrier, the types of questions posted on these Web sites may also vary. In addition, some carrier Web sites only list FAQs that are policy related, while others list routine questions about the mechanics of claims submissions and correcting billing errors. Carriers are required to monitor their own call centers and report to CMS on their performance, such as the average time that calls wait before being connected to a CSR and the percentage of provider calls that are abandoned before they reach a CSR. Carriers are also required to listen to, and rate, a selection of CSR calls on customer and knowledge skills, such as the manner in which they greet callers, conduct the call, offer additional assistance at the conclusion of the call, and whether their responses are correct and complete. Through CPEs, CMS also evaluates call center compliance with performance measures it establishes. Recently, CMS piloted remote call monitoring, which allows CMS staff to listen in on provider calls. CMS's administration of the Medicare program will undergo significant changes over the next several years as the Medicare Prescription Drug, Improvement, and Modernization Act of 2003 (MMA) is implemented. MMA provides CMS with increased flexibility in contracting with new entities to assist it in operating the Medicare program. Instead of primarily relying on the claims administration contractors to perform most of the key business functions of the program, the law authorizes CMS to enlist a variety of contractors to perform these tasks. For example, the MMA will allow CMS to use new contractors to communicate program information to its providers and deliver provider education and training. CMS is just beginning to develop plans to implement MMA's contracting reform provisions. Phase-in of certain provisions may begin as early as October 2005, and all carrier and fiscal intermediary contracts are scheduled to end by October 1, 2011. After this date, new contracts must be based on CMS's new authority. The agency expects to issue its implementation plan for contracting by October 1, 2004. CSRs at the carrier call centers we tested rarely provided correct and complete answers to our policy-oriented questions. Only 4 percent of the responses we received from CSRs were correct and complete. Our test suggested several factors that may account for poor performance, including fragmented information scattered among a variety of sources, confusing information that may be difficult for CSRs to understand and interpret, and difficulties in retaining CSRs. CMS is developing two initiatives that may improve CSRs' access to information; however, neither of these new tools is designed to provide a comprehensive source of information to support the CSRs who respond to providers' policy- oriented questions. We found that CSRs provided incorrect, partially correct, or incomplete responses to 96 percent of the 300 policy-oriented test calls we made to carrier call centers. The four questions we posed concerned a variety of circumstances on the proper way to bill Medicare in order to be paid for services rendered. Our questions specifically addressed the following topics: billing for beneficiaries transferred from one hospital to another, billing for services delivered by therapy students, billing for multiple surgeries on the same day, and billing for an office visit and services on the same day. The results of our test, which CMS Medicare coding and policy experts validated, are shown in table 1. Our analysis of CSR responses to test questions and discussions with CMS officials identified several factors that contributed to CSRs' errors. They include the following: Fragmentation of information: When responding to Medicare inquires from providers, CSRs rely on fragments of information from multiple electronic sources. In addition, many CSRs use printed Medicare program information, including policy changes, which CMS estimates at about 200 per year. For example, at one carrier call center, we observed that CSRs relied on electronic information from CMS and carrier Web sites, along with paper documents, including the Medicare carrier manual, program memorandums, and carrier bulletins. Further, CMS officials told us that the agency does not prepare scripted responses to follow when answering questions from providers as it does for CSRs responding to beneficiaries' inquiries. During the site visit, we also asked one CSR to demonstrate the process that would typically be followed to answer our four policy-oriented questions. In responding to one of our questions, we observed that this CSR did not have a source of comprehensive information that she could easily access to find the answer to our question. Instead, she accessed multiple information sources, including both electronic materials and paper documents, in an attempt to respond to our question. We further noted, as we toured the call center, that other CSRs also had access to, and appeared to be using, both paper and automated resources as they responded to calls. According to CMS and carrier officials, CSRs have to learn to operate multiple information systems, including CMS's claims information processing system and carrier developed information system. They also must access other sources, such as carrier or CMS Web sites, to respond to policy-oriented questions. Although CMS required that all CSRs have access to the Internet in fiscal year 2003, some CSRs continue to rely heavily on paper documents because of their familiarity with these materials. During our 300 test calls, CSRs referred us to a total of 13 different information sources when answering our second question regarding billing for services delivered by therapy students. Twelve of the references were either incorrect or did not include all of the information needed to give a correct and complete answer. Our review of the 13th document, which was structured in a question and answer format, included our specific test question but without the complete answer. Fragments of the answer, however, were located earlier in the document. We also found other parts of this answer on a different page, attached to a different but related question. It was evident to us that without reading the entire document, it would be plausible for the CSR to have read the test question and mistakenly given the caller the wrong answer, while assuming that the response given was correct and complete. Confusing information: Some of the information that CSRs must access to respond to policy-oriented questions is difficult to interpret. CMS officials acknowledged that some policies contain complex language. In addition, they told us that the agency's goal of quickly publishing a policy that is technically correct may sometimes overshadow its effort to develop a clear and understandable document. For example, we identified confusion among CSRs who responded to our second question concerning billing for services delivered by therapy students. Based on their exact responses, we were able to determine that 12 percent of the incorrect responses to this question were caused by the CSRs' confusion over a different Medicare policy, regarding the billing for services delegated by one professional provider to another, and not a student. CMS acknowledges that specialized training is required to understand the billing codes and modifiers that providers must include on their claims forms to receive payment from the program. Although CMS requires carriers to train CSRs, the agency has determined that answering providers' coding questions about specific claims is beyond the scope of CSRs' responsibilities. CMS also indicated that CSRs do not have the expertise to instruct a provider on the nuances of coding a service. CSRs, however, are permitted to respond to general questions about codes and modifiers, including coding definitions and explanations regarding the appropriate use of modifiers. We identified confusion among CSRs when they responded to such questions. In response to two of the four questions we posed, CSRs should have included specific modifiers with their answers because our questions were general and did not involve specific claims information. However, CSRs provided specific modifiers in only 16 of the 150 responses to these two questions. And, in most of these instances--9 of 16--the modifiers they cited were incorrect. For example, in one of our test questions, we asked for the extenuating circumstances for which carriers may pay the full amount for a second surgical procedure. In eight of the calls, CSRs responded that the provider should use either modifier "51" for multiple surgeries on the same day, modifier "59" denoting a distinct surgical procedure, modifier "58" for a staged procedure, modifier "76" indicating that a procedure was repeated by the same physician, or modifier "78" for a return to the operating room for a related procedure. The correct answer is modifier "22," indicating that the service performed was an unusual procedure. Without the correct modifier, the claim could be inappropriately denied or improperly paid. In addition, according to one CMS Medicare official--and confirmed by CSRs' responses to our test questions--CSRs may gravitate toward generic explanations and apply them to questions, even when they are not specific. For example, when answering our third question regarding billing for multiple surgeries to the same patient on the same day, 56 percent of the CSRs failed to address the extenuating circumstances in which a provider would be paid for multiple surgeries. Instead, their responses were based on their knowledge of general multiple surgery payment rules, which indicate that Medicare will pay 100 percent of costs for a beneficiary's first surgery and a reduced percentage for the second surgery. In addition, we categorized about 15 percent of CSRs' answers to our 300 test questions as vague and nonresponsive because, in these instances, CSRs simply responded that Medicare services must be medically necessary or that they cannot be preapproved. Although the vague answers may be correct, the responses did not address the specific elements contained in our questions. Difficulties in retaining CSRs: Difficulties in retaining staff limits carriers' ability to maintain a core of CSRs who are proficient on a range of complex, policy-oriented issues. CMS officials told us that retaining CSRs has been a major staffing problem. According to CMS officials, many of the most qualified CSRs are promoted to different positions within the call centers or resign to pursue better opportunities elsewhere. An internal CMS study found the turnover rate for carrier call center CSRs to be as high as 23 percent from calendar years 1999 through 2001 for all carrier call centers. This is significantly higher than the attrition rate for CMS's call centers for beneficiaries, 1-800-MEDICARE help lines, which one CMS official estimates is close to industry standards--about 10 percent. Although there are no more recent data, CMS officials view this as troubling. They explained that the CSR position is particularly challenging because, in addition to learning how to access and utilize multiple information systems, these employees must stay abreast of Medicare policy changes to answer the broad range of inquiries received by the carrier call centers. Although CMS is currently implementing two initiatives that may improve CSRs' access to information, neither of these new tools is designed to support the CSRs who respond to providers' policy-oriented questions. One is intended to enhance CSRs' accessibility to claims information; the other is aimed at clarifying information on new Medicare policies. First, CMS has begun deployment of a new computer application, Next Generation Desktop (NGD), to provide a single source of consolidated claims and related information to assist CSRs when they respond to status- oriented questions. Although NGD also contains some policy-oriented information, such as scripted responses for CSRs who respond to questions from beneficiaries, it does not improve CSRs' ability to respond to policy-oriented questions from providers. Recognizing the broader range of issues and the relative complexity of provider inquiries, CMS officials have not attempted to develop scripted responses to such questions. Although CMS is continuing to study the role of NGD in providing policy-oriented information, agency officials told us they are uncertain whether NGD is the appropriate mechanism to enhance the availability of such information to CSRs. Until CMS makes a final determination, CSRs can continue to access policy-oriented information on the agency's Web site. Second, CMS has developed a new strategy to clarify Medicare policy for providers, which CMS officials told us will also benefit the CSRs who respond to provider questions. CMS has retained a consulting firm to write explanatory articles about new Medicare policies. Although these articles may educate providers, they will be no more accessible to CSRs than the existing array of materials. For example, these articles are available to CSRs through CMS's Web site and carriers' Web-based bulletins. Although these articles contain citations to regulations and laws, for example, they are not electronically linked to the policies they describe. In addition, the policies they support are not annotated to reflect that an article exists, making it unlikely that CSRs can easily locate the clarifying information. Moreover, there are no plans to publish articles for the majority of existing policies. Like CMS-sponsored call centers, there are thousands of other call centers serving a large range of businesses and government agencies in the United States. Many of these centers rely on IVRs to route calls to the next available CSR. Other call centers have implemented systems that are more advanced than those used by carriers. Many of these centers triage incoming calls through a feature known as "skill-based routing." Skill- based routing systems are designed to enhance customer service by allowing the call center to first identify the nature of an incoming call and to then distribute the call to the CSR who is best qualified to respond to the caller's question. CSRs working in a skill-based routing environment develop expertise in key specialty areas so they can quickly and knowledgably respond to callers' questions. Although CMS has indicated it is committed to improving communications with providers, it has not taken steps that would enable it to identify the subject of providers' policy- oriented questions and route their calls to the most appropriate CSRs. CMS requires carriers to monitor the performance of their call centers. However, the performance standards that carriers are required to use, and the technology available to most of them, do not facilitate thorough assessments of whether CSRs provide correct and complete responses to policy-oriented questions. In addition, CMS's own monitoring efforts-- CPEs and remote monitoring of select calls--do not provide sufficiently detailed or meaningful information regarding CSR accuracy. CMS does not require carriers to monitor a sufficient number of calls to fully evaluate the CSRs' performance. CMS requires carriers to monitor their own call center performance by periodically listening to, and rating, a sample of each CSR's calls. On average, each CSR answers more than 1,700 calls a month. In fiscal year 2004, CMS required carriers to evaluate three calls per CSR per month. Furthermore, for the 31 "blended" call centers, which respond to inquiries from both providers and beneficiaries, carriers are only required to monitor one provider call per CSR per month. This falls short of one call center industry expert's recommendation to monitor a minimum of eight provider calls per CSR per month to obtain accurate statistics on CSR performance. It is also lower than the most frequent monitoring of calls per month from a survey of 735 North American call centers that represent help lines in various industries, including telecommunications, financial services, and health care. According to this study, there is a wide variance in the number of calls monitored per month per CSR. However, the most commonly reported monthly monitoring frequencies by survey respondents were 4 to 5 and 10 or more--exceeding CMS's requirement. In addition, CMS-developed quality standards used by carriers to conduct self-monitoring do not measure CSR performance in a meaningful way. CMS requires that CSRs be evaluated on customer skills--such as vocal tone, volume and politeness--and knowledge skills--including the accuracy and completeness of responses. However, we reported in 2002 that CMS's definition of what constitutes accuracy is neither clear nor specific. For example, according to CMS's standards, carriers should consider a response "accurate" if the CSR "gives an accurate response or referral" as opposed to providing necessary and complete information for the provider to bill the program correctly. Without such guidance or other criteria linked to measurable outcomes, the carrier has little basis to evaluate the correctness and completeness of CSRs' responses to policy- oriented questions. Although we recommended that CMS establish new performance standards for CSRs that emphasize providing correct and complete answers to provider inquiries, CMS has not revised the definition. Moreover, CMS has not instituted standard requirements for the technology used by carriers when conducting self-monitoring activities. As a result, there is a broad range of self-monitoring capabilities among the carriers, which, according to CMS officials, can affect a supervisor's ability to determine whether a CSR's response was correct and complete. For example, only 14 of the 34 carrier call centers have the capability of recording both the audio portion of monitored calls and the associated computer screens viewed by CSRs. This technology enables the supervisors monitoring calls to follow the actions taken by CSRs, step by step, as they respond to callers. Not only can these supervisors hear callers' questions and CSRs responses, but they can also view every computer screen accessed by CSRs during calls, enhancing their ability to determine whether CSRs supplied answers that were correct and complete. However, the remaining 20 call centers do not have this capability. While supervisors at 17 of them can record the audio portion of calls, they cannot view the computer screens accessed by the CSRs. Supervisors at 3 centers have no recording capability and are limited to listening to and evaluating "live" calls as they are received. Although CMS's principal oversight tools--CPEs--are designed to evaluate call centers' compliance with performance standards, they do not provide a comprehensive assessment of whether information provided by CSRs is correct and complete. As we noted in our previous report, these on-site evaluations, which are conducted by CMS staff and follow a structured protocol, focus on performance standards that address procedures. For example, in preparing that report, we observed a CPE review team concentrated on procedural items such as how long a caller was kept on hold, rather than on whether the information provided was correct and complete. Although CPE evaluators also review call center data and interview call center managers, these activities do not provide a method to measure the correctness and completeness of CSR responses. We found that the CPE evaluation criteria are not designed to verify that CSRs' responses to providers are accurate. Instead, they focus on evaluating whether carriers are appropriately adhering to CMS's self- monitoring requirements. For example, the CPE evaluators listen to a sample of calls self-monitored by the carrier to verify that the carrier is properly evaluating and documenting the CSRs' performance. In listening to these sampled calls, the CPE evaluators are not required to evaluate the correctness or completeness of responses provided by a CSR, rather they are expected to ensure that the carrier has a system in place to monitor calls. Although our earlier report included a recommendation that CMS employ expert teams to conduct more substantive reviews of calls to strengthen CPEs, CMS told us at that time that this was not feasible. CMS officials recently told us, however, that in many instances, CPE evaluators do not have the expertise to evaluate the accuracy of CSRs' responses. In addition, CPEs are not performed often enough to provide current feedback on either call center or CSR performance. In fiscal year 2002, only one carrier call center had a CPE covering provider telephone inquiries. Not one CPE was performed in fiscal year 2003. We also found that CPEs are based on an assessment of too few calls to provide meaningful data and are conducted too infrequently to provide current information on call center performance. The required CPE sample is too small to provide reliable results. CPE evaluators listen to a sample of 10 calls monitored by the carrier. If the call center is "blended"--as 31 of the 34 are--the CPE evaluators will listen to 5 provider calls. As a point of comparison, if a call center with the smallest number of CSRs monitors 216 provider calls from its CSRs annually, we found that CPE evaluators would have to draw a simple random sample of 138 provider calls annually to estimate the percentage of correct and complete calls within a margin of error of plus or minus 5 percent, with a 95 percent confidence level. CPE evaluators monitoring a call center with a larger number of CSRs would have to conduct more monitoring. For example, a call center that monitors 1,800 provider calls from its CSRs annually, would require the CPE evaluators to draw a simple random sample of 317 provider calls annually to reach this same confidence level. In July 2003, CMS introduced a pilot project that provides a second means of monitoring carrier call centers. CMS can now remotely monitor calls by dialing into carrier call centers and listening to calls as they occur. Remote monitoring provides CMS with an opportunity to hear providers' questions, as well as CSRs' responses, firsthand. Initially, CMS staff listened to 10 calls per month for each center, to develop a general understanding of provider inquiries and to contribute to developing the strategy for future monitoring. However, the staff responsible for remote monitoring reported being overwhelmed by the burden inherent in the task, and in January 2004, CMS reduced the number to five calls per center per month. Staff engaged in remote monitoring can only access the audio portion of calls, limiting their ability to thoroughly evaluate the correctness and completeness of CSRs' responses. In addition, CMS officials recognize that like CPE evaluators, staff engaged in remote monitoring lack the necessary understanding of substantive policy issues involved in the call to determine whether CSR responses were correct. Agency officials stated that they are studying how to best make use of this new capability. They told us that they are uncertain whether the pilot project will be expanded because they are not convinced that remote monitoring is the most appropriate vehicle for evaluating the correctness and completeness of CSR responses to provider questions. Ensuring that physicians and other providers receive correct and complete answers to their policy-oriented questions is critical to their ability to correctly bill Medicare for services rendered to the program's beneficiaries. Although policy-oriented questions may represent a small proportion of inquiries made to call centers, it is nonetheless important to ensure that providers can rely on the information they receive. Many call centers serving a variety of businesses have taken advantage of skill-based routing to identify a caller's specific question and direct the call to the CSR most qualified to respond. However, CMS has not done so, nor has it developed other strategies to improve the ability of CSRs to respond to inquiries from providers. Our test calls continue to show that carrier call centers do not adequately respond to policy-oriented questions. When responding to our questions, it was evident that CSRs lacked access to comprehensive materials that would facilitate correct and complete answers. Instead, CSRs relied on fragmented information sources and were also confused about Medicare policy issues. Furthermore, while CMS is attempting to enhance its monitoring efforts, it has not established a program that can sufficiently evaluate whether providers receive correct and complete answers to their policy-oriented questions. Now that CMS has been given new authority to contract with a variety of entities to assist it with managing the Medicare program, it should take the opportunity to improve its communications with providers. The new law gives CMS the opportunity to identify organizations that can best assist it with developing policy expertise among staff who respond to providers' questions, improving access to policy-oriented materials, and enhancing call center monitoring. In order to improve the accuracy and completeness of responses to policy- oriented inquiries from providers, we recommend that the Administrator of CMS take steps to ensure that all CSRs have the necessary tools to respond to such calls. Specifically, we recommend that the Administrator take the following three actions: Create a process to routinely screen and triage calls by routing complex policy-oriented questions to staff with the expertise to adequately address them. Develop clear and easily accessible policy-oriented materials to assist CSRs. The materials should be electronically searchable so that CSRs can expeditiously provide correct and complete responses to policy-oriented questions. Establish an effective monitoring program for call centers to assess CSRs' performance. The program should include the development of specific performance standards that will allow CMS to thoroughly and routinely measure the correctness and completeness of information given by CSRs in response to policy-oriented questions. In written comments on a draft of this report, CMS expressed its commitment to improving communications with providers and generally agreed with our recommendations. CMS agreed with our first recommendation to create a process to routinely screen and triage calls. CMS said it plans to establish a tiered system using specialty staff to respond to provider inquiries by fiscal year 2005. CMS also agreed with our second recommendation that clear and easily accessible policy-oriented materials should be available and electronically searchable for CSRs. However, they went further to state that clear and accessible information will also be available to specialty staff tasked with responding to complex policy-oriented questions. CMS also described its efforts to make information available to providers through customized Web pages and other educational materials. While CMS agreed with the concept of establishing an effective monitoring program--our third recommendation--it stated that it is in the process of determining how to do so once its new approach of triaging calls is implemented. CMS also said it is exploring other initiatives to enhance monitoring such as modifying its CPE requirements, developing performance-based standards for provider telephone inquiries, and surveying providers on their satisfaction with call centers' performance. We have reprinted CMS's letter in appendix III. CMS also provided us with technical comments, which we have incorporated as appropriate. As agreed with your office, unless you announce its contents earlier, we plan no further distribution of this report until 30 days after its issuance. At that time, we will send copies to the Administrator of CMS and other interested parties. We will then make copies available to others upon request. In addition, the report will be available at no charge on GAO's Web site at http://www.gao.gov. If you or your staff have any questions about this report, please call me at (312) 220-7600. An additional GAO contact and other staff who made contributions to this report are listed in appendix IV. To determine carriers' effectiveness in providing correct and complete responses, we placed 300 calls to 34 carrier call centers. We searched a variety of carriers' Web sites and compiled a group of 18 questions that physicians and other Part B providers frequently asked when contacting call centers. The questions represented common, policy-oriented questions concerning the proper way to bill Medicare in order to obtain payment from the program, as opposed to status-oriented claims inquiries. We asked Center for Medicare & Medicaid Services (CMS) officials to review our questions to determine whether they considered any of the 18 questions inappropriate for our test calls. For example, we did not want to pose a question that would unfairly test a customer service representative (CSR) knowledge and thus we did not want to include a question on a new or recently changed policy. We also did not want to pose questions that were the subject of an ongoing controversy. Based on input from CMS officials, we eliminated three questions. The first question we eliminated referred to pricing rules for multiple surgeries. CMS officials stated that the response to this question, as posted on a carrier's Web site, was unclear. The second question we excluded referred to reimbursements for nurse practitioner services. CMS officials also expressed hesitancy about the phrasing of this question and answer as shown on a carrier's Web site. Although CMS ultimately suggested language to rephrase this question and answer, we opted to remove the question from consideration. The third question we eliminated referred to an issue that CMS officials told us was the subject of an ongoing lawsuit. After obtaining CMS's input, we selected 4 questions (see app. II) from the remaining 15 questions. These questions addressed: (1) billing for beneficiaries transferred from one hospital to another, (2) billing for services delivered by therapy students, (3) billing for multiple surgeries for the same patient on the same day, and (4) billing for an office visit and procedure for the same patient on the same day. We did not inform CMS before making the calls of the final questions we selected. Each of the four questions was randomly assigned across the 34 carrier call centers and each question was posed 75 times. Calls were placed at different times of day and different days of the week from October 20 through November 3, 2003. Twenty-eight of the carrier call centers were called nine times and the remaining six call centers were each called eight times. To facilitate our calls, CMS officials informed call center managers of our test. They also agreed not to disclose any of the potential questions to carrier call center staff. During our calls, we identified ourselves as GAO representatives and asked each CSR to answer our question as if we were providers. To prevent us from biasing CSRs' responses and to ensure fairness, we read each question to CSRs without offering additional information or explanations. However, we repeated questions upon request. Prompts were only given if the CSR probed for more specific information or gave conditional responses that depended upon different circumstances. In those situations, we asked the CSR to provide the correct answer for each set of circumstances. Following the response, we asked the CSR if there was any additional information he or she would like to provide. We also told CSRs we were manually recording their responses verbatim. We analyzed the CSR responses and simultaneously submitted them to Medicare coding experts at CMS. Our assessment of CSR responses and the coding experts' verification of results relied on the following criteria: Correct and complete: The answer provided enough information to correctly bill the Medicare program, including (1) a correct explanation of how to apply the billing policy and (2) correct billing codes or a referral to specific documentation that provided coding information. Partially correct or incomplete: The answer provided some explanation, but (1) did not provide assistance in interpretation or warn about special circumstances that would affect billing; (2) provided interpretation but no directions to specific documentation; or (3) was correct, but not sufficiently complete to ensure that the claim would consistently pass claims processing edits. Incorrect: The answer contained fully or partially incorrect information, such that a physician might incorrectly bill or not file a claim for a billable service. Following CMS's verification, we discussed and resolved all discrepancies between our assessment of responses and CMS's verification. For example, when initially assessing CSRs' responses, we attempted to locate Web site documents that CSRs referred to during our call. If we found that the reference contained all the accurate information necessary to bill the program properly, we considered the CSRs' responses to be correct and complete even if they did not tell us the information themselves. Although CMS coding experts did not initially review these documents, and therefore may have considered the CSR's response to be incorrect and incomplete, they subsequently agreed that this was a fair and appropriate criterion to add to our assessment. The results from our 300 test calls are limited only to those calls and are not generalizable to the population of calls routinely made to call centers by providers. Although the four policy-oriented questions we posed were frequently asked questions obtained from carrier Web sites, they do not encompass all of the questions that providers might ask. We also interviewed carrier and CMS officials to determine what efforts the agency had in place to enhance CSRs' access to information. We reviewed information regarding the Next Generation Desktop (NGD) application, including videotape and internal documents outlining the phase-in schedule for the application. In addition, we observed a demonstration of the NGD application and monitored its capability to facilitate CSR's responses to providers' questions. To determine the efforts CMS has made to provide oversight, we identified CMS requirements for carrier call center operations and discussed with CMS staff the agency's oversight and monitoring of carrier call center activities. We reviewed the agency's protocol for its contractor performance evaluations (CPE) to determine the scope of work evaluators would perform and evaluated carrier call center performance standards to identify the types of problems found during their site visits. We also reviewed CPE reports from fiscal years 2001 and 2002. CMS did not perform any CPEs in fiscal year 2003. We visited two carrier call centers and consulted an industry expert on issues related to call center technology and standards. In addition, we observed carrier call centers' monitoring of calls for quality at one of the carrier call centers we visited. We also observed CMS regional staff performing remote monitoring of provider calls. We performed our work from September 2003 through June 2004 in accordance with generally accepted government auditing standards. The questions and answers we used to test the accuracy of carrier call center responses to policy-oriented questions are shown in table 2. Shaunessye Curry, Margaret Weber, Helen Chung, Mary Reich, and Marie Stetser made key contributions to this report. The Government Accountability Office, the audit, evaluation and investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO's commitment to good government is reflected in its core values of accountability, integrity, and reliability. The fastest and easiest way to obtain copies of GAO documents at no cost is through GAO's Web site (www.gao.gov). Each weekday, GAO posts newly released reports, testimony, and correspondence on its Web site. To have GAO e-mail you a list of newly posted products every afternoon, go to www.gao.gov and select "Subscribe to Updates."

Summary:
In 2002, GAO reported that the Centers for Medicare & Medicaid Services (CMS) needed to improve its communications with providers who deliver medical care to beneficiaries. GAO reported that 85 percent of the responses it received to 61 calls made to call centers operated by Medicare carriers--contractors that help manage the Medicare program--were incorrect or incomplete. GAO also found that CMS's primary oversight tools were insufficient to ensure accuracy in communication. GAO was asked whether call centers now provide correct and complete information to providers. GAO (1) reviewed carriers' effectiveness in providing correct and complete responses to policy-oriented telephone inquiries and CMS's efforts to improve communications with providers and (2) evaluated CMS's efforts to provide oversight of carrier call centers. Only 4 percent of the responses GAO received in 300 test calls to 34 call centers were correct and complete. GAO posed four policy-oriented questions 75 times each to carrier call centers. The level of correct and complete responses for each individual billing question ranged from 1 to 5 percent. The majority of remaining responses were incorrect, or partially correct or incomplete. Several factors, including fragmented sources of information, confusing policy information, and difficulties in retaining the CSRs responding to calls appear to account for the lack of correct and complete answers. There are many call centers serving other industries that triage incoming calls by first identifying the nature of the call and then distributing it to the CSR who is best qualified to respond. Although CMS has not adopted this approach, it is currently implementing two other initiatives that may improve CSRs' access to information. However, neither initiative is specifically designed to support CSRs responding to policy-oriented questions. In addition, CMS's efforts to provide oversight of carrier call centers are inadequate. Although CMS requires carriers to monitor the performance of their call centers, the standards used and the technological resources available to evaluate performance do not allow carriers to thoroughly assess whether CSRs' responses are correct and complete. In addition, CMS's own monitoring efforts are too infrequent. CMS only performed one contractor performance evaluation related to carrier telephone services in fiscal year 2002 and none were performed in fiscal year 2003. Moreover, when performed, these evaluations did not provide sufficiently detailed information to assess CSRs' performance.