Background

CMS develops regulations and policies to implement the statutory  provisions governing the Medicare program, and it communicates the  information to providers primarily through its Medicare contractors. The  contractors share information with providers through their Web sites,  written bulletins, and carrier call centers. To respond to inquiries from  providers, carriers operate 34 call centers. Most of the carrier call  centers—31 of 34—are “blended,” that is they respond to inquiries from  beneficiaries as well as from providers. In most cases, CSRs at “blended”  call centers answer calls from both providers and beneficiaries.
More than 21 million inquiries were made to carrier call centers in fiscal  year 2003. The vast majority of these were “status-oriented” calls. Typically  status-oriented calls are relatively simple to answer and involve inquiries  concerning the status of a claim or confirmation of an individual’s  eligibility for Medicare. Such calls generally do not require CSRs to  provide callers with complex information. On the other end of the  spectrum are “policy-oriented” questions, which involve more complicated  issues, such as billing rules, covered services, and medical policies. The  remaining calls include elements of both status-oriented and policy- oriented inquiries. For example, a provider may call to learn why a claim  was denied. In some instances, the explanation may be simple, such as the  claim form omitted necessary information. In others, the reason for the  denial may be more complex and involve an assessment of whether the  particular circumstance required to obtain Medicare payment was met.  Although CMS requires that carriers report data that categorize calls by  type, these categories are not standardized, and carriers differ in the  criteria they use to define call type. CMS officials estimate the volume of  calls received by CSRs involving policy-oriented questions to have been  approximately 500,000 in fiscal year 2003.
In fiscal year 2001, CMS required that all carrier call centers install  automated voice response systems. The interactive voice response (IVR)  unit allows providers to use their telephone keypads to respond to  automated prompts and obtain status-oriented information without  speaking to CSRs. Use of the IVR has been growing, and in fiscal year  2003, the automated system handled more than 52 percent of provider  inquiries answered at carrier call centers. CSRs are available to respond to  inquiries that providers believe are beyond the capability of the IVR,  including policy-oriented calls. Generally, calls to CSRs are electronically  routed by the carriers’ automated systems based on CSR availability. The  routing process does not consider the nature and complexity of the  question or the expertise of the CSRs. Despite the diversion of many calls  to the IVR, CSRs recently experienced an increase in the number of calls  that they answer. According to CMS, the number of calls CSRs answered  increased from 9 million in fiscal year 2002 to 10 million in fiscal year 2003.
CSRs responding to status-oriented calls can typically access the relevant  claims or enrollment information to respond to the inquiry. In addition, to  assist CSRs, carriers have developed scripted responses to answer  standard questions from beneficiaries, such as how to enroll in Medicare.  To respond to inquiries from providers, CSRs may use FAQs posted on  their carriers’ Web sites. In addition, they may search a variety of other  sources, including CMS’s Web sites. If CSRs cannot locate information to  respond to a provider’s question, they may arrange to contact the  provider—after conferring with a specialist.
In addition, CMS requires each carrier to analyze provider inquiry data and  develop a list of questions most frequently asked and areas of concern or  confusion for providers. They must also tally problem areas identified  when providers submit erroneous claims for payment. Each quarter,  carriers report the 10 most frequent inquiries and claim submission errors  to CMS and update the list of FAQs on their own Web sites. Because the  nature of calls may vary by carrier, the types of questions posted on these  Web sites may also vary. In addition, some carrier Web sites only list FAQs  that are policy related, while others list routine questions about the  mechanics of claims submissions and correcting billing errors.
Carriers are required to monitor their own call centers and report to CMS  on their performance, such as the average time that calls wait before being  connected to a CSR and the percentage of provider calls that are  abandoned before they reach a CSR. Carriers are also required to listen to,  and rate, a selection of CSR calls on customer and knowledge skills, such  as the manner in which they greet callers, conduct the call, offer additional  assistance at the conclusion of the call, and whether their responses are  correct and complete. Through CPEs, CMS also evaluates call center  compliance with performance measures it establishes. Recently, CMS  piloted remote call monitoring, which allows CMS staff to listen in on  provider calls.
CMS’s administration of the Medicare program will undergo significant  changes over the next several years as the Medicare Prescription Drug,  Improvement, and Modernization Act of 2003 (MMA) is implemented.  MMA provides CMS with increased flexibility in contracting with new  entities to assist it in operating the Medicare program. Instead of primarily  relying on the claims administration contractors to perform most of the  key business functions of the program, the law authorizes CMS to enlist a  variety of contractors to perform these tasks. For example, the MMA will  allow CMS to use new contractors to communicate program information  to its providers and deliver provider education and training. CMS is just  beginning to develop plans to implement MMA’s contracting reform  provisions. Phase-in of certain provisions may begin as early as October  2005, and all carrier and fiscal intermediary contracts are scheduled to end  by October 1, 2011. After this date, new contracts must be based on CMS’s  new authority. The agency expects to issue its implementation plan for  contracting by October 1, 2004.

	CSRs’ Responses to Policy-Oriented Questions Were Largely Incorrect

CSRs at the carrier call centers we tested rarely provided correct and  complete answers to our policy-oriented questions. Only 4 percent of the  responses we received from CSRs were correct and complete. Our test  suggested several factors that may account for poor performance,  including fragmented information scattered among a variety of sources,  confusing information that may be difficult for CSRs to understand and  interpret, and difficulties in retaining CSRs. CMS is developing two  initiatives that may improve CSRs’ access to information; however, neither  of these new tools is designed to provide a comprehensive source of  information to support the CSRs who respond to providers’ policy- oriented questions.

		CSRs Almost Never Provided Correct and Complete Responses to Our Policy Questions

We found that CSRs provided incorrect, partially correct, or incomplete  responses to 96 percent of the 300 policy-oriented test calls we made to  carrier call centers. The four questions we posed concerned a variety of  circumstances on the proper way to bill Medicare in order to be paid for  services rendered. Our questions specifically addressed the following  topics: billing for beneficiaries transferred from one hospital to another,  billing for services delivered by therapy students, billing for multiple  surgeries on the same day, and billing for an office visit and services on  the same day. The results of our test, which CMS Medicare coding and  policy experts validated, are shown in table 1.

		Variety of Factors Contributed to CSRs’ Incorrect and Incomplete Responses

Our analysis of CSR responses to test questions and discussions with CMS  officials identified several factors that contributed to CSRs’ errors. They  include the following:  Fragmentation of information: When responding to Medicare inquires  from providers, CSRs rely on fragments of information from multiple  electronic sources. In addition, many CSRs use printed Medicare program  information, including policy changes, which CMS estimates at about 200  per year. For example, at one carrier call center, we observed that CSRs  relied on electronic information from CMS and carrier Web sites, along  with paper documents, including the Medicare carrier manual, program  memorandums, and carrier bulletins. Further, CMS officials told us that  the agency does not prepare scripted responses to follow when answering  questions from providers as it does for CSRs responding to beneficiaries’  inquiries.
During the site visit, we also asked one CSR to demonstrate the process  that would typically be followed to answer our four policy-oriented  questions. In responding to one of our questions, we observed that this  CSR did not have a source of comprehensive information that she could  easily access to find the answer to our question. Instead, she accessed  multiple information sources, including both electronic materials and  paper documents, in an attempt to respond to our question. We further  noted, as we toured the call center, that other CSRs also had access to,  and appeared to be using, both paper and automated resources as they  responded to calls. According to CMS and carrier officials, CSRs have to  learn to operate multiple information systems, including CMS’s claims  information processing system and carrier developed information system.  They also must access other sources, such as carrier or CMS Web sites, to  respond to policy-oriented questions. Although CMS required that all CSRs  have access to the Internet in fiscal year 2003, some CSRs continue to rely  heavily on paper documents because of their familiarity with these  materials.
During our 300 test calls, CSRs referred us to a total of 13 different  information sources when answering our second question regarding billing  for services delivered by therapy students. Twelve of the references were  either incorrect or did not include all of the information needed to give a  correct and complete answer. Our review of the 13th document, which  was structured in a question and answer format, included our specific test  question but without the complete answer. Fragments of the answer,  however, were located earlier in the document. We also found other parts  of this answer on a different page, attached to a different but related  question. It was evident to us that without reading the entire document, it  would be plausible for the CSR to have read the test question and  mistakenly given the caller the wrong answer, while assuming that the  response given was correct and complete.
Confusing information: Some of the information that CSRs must access  to respond to policy-oriented questions is difficult to interpret. CMS  officials acknowledged that some policies contain complex language. In  addition, they told us that the agency’s goal of quickly publishing a policy  that is technically correct may sometimes overshadow its effort to develop  a clear and understandable document. For example, we identified  confusion among CSRs who responded to our second question concerning  billing for services delivered by therapy students. Based on their exact  responses, we were able to determine that 12 percent of the incorrect  responses to this question were caused by the CSRs’ confusion over a  different Medicare policy, regarding the billing for services delegated by  one professional provider to another, and not a student.
CMS acknowledges that specialized training is required to understand the  billing codes and modifiers that providers must include on their claims  forms to receive payment from the program. Although CMS requires  carriers to train CSRs, the agency has determined that answering  providers’ coding questions about specific claims is beyond the scope of  CSRs’ responsibilities. CMS also indicated that CSRs do not have the  expertise to instruct a provider on the nuances of coding a service. CSRs,  however, are permitted to respond to general questions about codes and  modifiers, including coding definitions and explanations regarding the  appropriate use of modifiers. We identified confusion among CSRs when  they responded to such questions. In response to two of the four questions  we posed, CSRs should have included specific modifiers with their  answers because our questions were general and did not involve specific  claims information. However, CSRs provided specific modifiers in only 16  of the 150 responses to these two questions. And, in most of these  instances—9 of 16—the modifiers they cited were incorrect. For example,  in one of our test questions, we asked for the extenuating circumstances  for which carriers may pay the full amount for a second surgical  procedure. In eight of the calls, CSRs responded that the provider should  use either modifier “51” for multiple surgeries on the same day, modifier  “59” denoting a distinct surgical procedure, modifier “58” for a staged  procedure, modifier “76” indicating that a procedure was repeated by the  same physician, or modifier “78” for a return to the operating room for a  related procedure. The correct answer is modifier “22,” indicating that the  service performed was an unusual procedure. Without the correct  modifier, the claim could be inappropriately denied or improperly paid.
In addition, according to one CMS Medicare official—and confirmed by  CSRs’ responses to our test questions—CSRs may gravitate toward generic  explanations and apply them to questions, even when they are not specific.  For example, when answering our third question regarding billing for  multiple surgeries to the same patient on the same day, 56 percent of the  CSRs failed to address the extenuating circumstances in which a provider  would be paid for multiple surgeries. Instead, their responses were based  on their knowledge of general multiple surgery payment rules, which  indicate that Medicare will pay 100 percent of costs for a beneficiary’s first  surgery and a reduced percentage for the second surgery. In addition, we  categorized about 15 percent of CSRs’ answers to our 300 test questions as  vague and nonresponsive because, in these instances, CSRs simply  responded that Medicare services must be medically necessary or that  they cannot be preapproved. Although the vague answers may be correct,  the responses did not address the specific elements contained in our  questions.
Difficulties in retaining CSRs: Difficulties in retaining staff limits  carriers’ ability to maintain a core of CSRs who are proficient on a range  of complex, policy-oriented issues. CMS officials told us that retaining  CSRs has been a major staffing problem. According to CMS officials, many  of the most qualified CSRs are promoted to different positions within the  call centers or resign to pursue better opportunities elsewhere.  An  internal CMS study found the turnover rate for carrier call center CSRs to  be as high as 23 percent from calendar years 1999 through 2001 for all  carrier call centers. This is significantly higher than the attrition rate for  CMS’s call centers for beneficiaries, 1-800-MEDICARE help lines, which  one CMS official estimates is close to industry standards—about 10  percent. Although there are no more recent data, CMS officials view this as  troubling. They explained that the CSR position is particularly challenging  because, in addition to learning how to access and utilize multiple  information systems, these employees must stay abreast of Medicare  policy changes to answer the broad range of inquiries received by the  carrier call centers.

		CMS’s Efforts Not Targeted to Supplying Policy- Oriented Information to CSRs

Although CMS is currently implementing two initiatives that may improve  CSRs’ access to information, neither of these new tools is designed to  support the CSRs who respond to providers’ policy-oriented questions.  One is intended to enhance CSRs’ accessibility to claims information; the  other is aimed at clarifying information on new Medicare policies.
First, CMS has begun deployment of a new computer application, Next  Generation Desktop (NGD), to provide a single source of consolidated  claims and related information to assist CSRs when they respond to status- oriented questions. Although NGD also contains some policy-oriented  information, such as scripted responses for CSRs who respond to  questions from beneficiaries, it does not improve CSRs’ ability to respond  to policy-oriented questions from providers. Recognizing the broader  range of issues and the relative complexity of provider inquiries, CMS  officials have not attempted to develop scripted responses to such  questions. Although CMS is continuing to study the role of NGD in  providing policy-oriented information, agency officials told us they are  uncertain whether NGD is the appropriate mechanism to enhance the  availability of such information to CSRs. Until CMS makes a final  determination, CSRs can continue to access policy-oriented information  on the agency’s Web site.
Second, CMS has developed a new strategy to clarify Medicare policy for  providers, which CMS officials told us will also benefit the CSRs who  respond to provider questions. CMS has retained a consulting firm to write  explanatory articles about new Medicare policies. Although these articles  may educate providers, they will be no more accessible to CSRs than the  existing array of materials. For example, these articles are available to  CSRs through CMS’s Web site and carriers’ Web-based bulletins. Although  these articles contain citations to regulations and laws, for example, they  are not electronically linked to the policies they describe. In addition, the  policies they support are not annotated to reflect that an article exists,  making it unlikely that CSRs can easily locate the clarifying information.  Moreover, there are no plans to publish articles for the majority of existing  policies.
Like CMS-sponsored call centers, there are thousands of other call centers  serving a large range of businesses and government agencies in the United  States. Many of these centers rely on IVRs to route calls to the next  available CSR. Other call centers have implemented systems that are more  advanced than those used by carriers. Many of these centers triage  incoming calls through a feature known as “skill-based routing.” Skill- based routing systems are designed to enhance customer service by  allowing the call center to first identify the nature of an incoming call and  to then distribute the call to the CSR who is best qualified to respond to  the caller’s question. CSRs working in a skill-based routing environment  develop expertise in key specialty areas so they can quickly and  knowledgably respond to callers’ questions. Although CMS has indicated it  is committed to improving communications with providers, it has not  taken steps that would enable it to identify the subject of providers’ policy- oriented questions and route their calls to the most appropriate CSRs.

	Call Center Oversight Does Not Adequately Assess CSRs’ Responses to Policy- Oriented Questions

CMS requires carriers to monitor the performance of their call centers.  However, the performance standards that carriers are required to use, and  the technology available to most of them, do not facilitate thorough  assessments of whether CSRs provide correct and complete responses to  policy-oriented questions. In addition, CMS’s own monitoring efforts— CPEs and remote monitoring of select calls—do not provide sufficiently  detailed or meaningful information regarding CSR accuracy.

		Carrier Self-Monitoring Does Not Effectively Evaluate CSRs’ Responses to Policy-Oriented Questions

CMS does not require carriers to monitor a sufficient number of calls to  fully evaluate the CSRs’ performance. CMS requires carriers to monitor  their own call center performance by periodically listening to, and rating, a  sample of each CSR’s calls. On average, each CSR answers more than  1,700 calls a month. In fiscal year 2004, CMS required carriers to evaluate  three calls per CSR per month. Furthermore, for the 31 “blended” call  centers, which respond to inquiries from both providers and beneficiaries,  carriers are only required to monitor one provider call per CSR per month.  This falls short of one call center industry expert’s recommendation to  monitor a minimum of eight provider calls per CSR per month to obtain  accurate statistics on CSR performance. It is also lower than the most  frequent monitoring of calls per month from a survey of 735 North  American call centers that represent help lines in various industries,  including telecommunications, financial services, and health care.  According to this study, there is a wide variance in the number of calls  monitored per month per CSR. However, the most commonly reported  monthly monitoring frequencies by survey respondents were 4 to 5 and 10  or more—exceeding CMS’s requirement.
In addition, CMS-developed quality standards used by carriers to conduct  self-monitoring do not measure CSR performance in a meaningful way.  CMS requires that CSRs be evaluated on customer skills—such as vocal  tone, volume and politeness—and knowledge skills—including the  accuracy and completeness of responses. However, we reported in 2002  that CMS’s definition of what constitutes accuracy is neither clear nor  specific. For example, according to CMS’s standards, carriers should  consider a response “accurate” if the CSR “gives an accurate response or  referral” as opposed to providing necessary and complete information for  the provider to bill the program correctly. Without such guidance or other  criteria linked to measurable outcomes, the carrier has little basis to  evaluate the correctness and completeness of CSRs’ responses to policy- oriented questions. Although we recommended that CMS establish new  performance standards for CSRs that emphasize providing correct and  complete answers to provider inquiries, CMS has not revised the  definition.
Moreover, CMS has not instituted standard requirements for the  technology used by carriers when conducting self-monitoring activities. As  a result, there is a broad range of self-monitoring capabilities among the  carriers, which, according to CMS officials, can affect a supervisor’s ability  to determine whether a CSR’s response was correct and complete. For  example, only 14 of the 34 carrier call centers have the capability of  recording both the audio portion of monitored calls and the associated  computer screens viewed by CSRs. This technology enables the  supervisors monitoring calls to follow the actions taken by CSRs, step by  step, as they respond to callers. Not only can these supervisors hear  callers’ questions and CSRs responses, but they can also view every  computer screen accessed by CSRs during calls, enhancing their ability to  determine whether CSRs supplied answers that were correct and  complete. However, the remaining 20 call centers do not have this  capability. While supervisors at 17 of them can record the audio portion of  calls, they cannot view the computer screens accessed by the CSRs.  Supervisors at 3 centers have no recording capability and are limited to  listening to and evaluating “live” calls as they are received.

		CMS Does Not Adequately Evaluate CSRs’ Responses to Policy-Oriented Questions

Although CMS’s principal oversight tools—CPEs—are designed to  evaluate call centers’ compliance with performance standards, they do not  provide a comprehensive assessment of whether information provided by  CSRs is correct and complete. As we noted in our previous report, these  on-site evaluations, which are conducted by CMS staff and follow a  structured protocol, focus on performance standards that address  procedures. For example, in preparing that report, we observed a CPE  review team concentrated on procedural items such as how long a caller  was kept on hold, rather than on whether the information provided was  correct and complete. Although CPE evaluators also review call center  data and interview call center managers, these activities do not provide a  method to measure the correctness and completeness of CSR responses.
We found that the CPE evaluation criteria are not designed to verify that  CSRs’ responses to providers are accurate. Instead, they focus on  evaluating whether carriers are appropriately adhering to CMS’s self- monitoring requirements. For example, the CPE evaluators listen to a  sample of calls self-monitored by the carrier to verify that the carrier is  properly evaluating and documenting the CSRs’ performance. In listening  to these sampled calls, the CPE evaluators are not required to evaluate the  correctness or completeness of responses provided by a CSR, rather they  are expected to ensure that the carrier has a system in place to monitor  calls. Although our earlier report included a recommendation that CMS  employ expert teams to conduct more substantive reviews of calls to  strengthen CPEs, CMS told us at that time that this was not feasible. CMS  officials recently told us, however, that in many instances, CPE evaluators  do not have the expertise to evaluate the accuracy of CSRs’ responses.
In addition, CPEs are not performed often enough to provide current  feedback on either call center or CSR performance. In fiscal year 2002,  only one carrier call center had a CPE covering provider telephone  inquiries. Not one CPE was performed in fiscal year 2003. We also found  that CPEs are based on an assessment of too few calls to provide  meaningful data and are conducted too infrequently to provide current  information on call center performance. The required CPE sample is too  small to provide reliable results. CPE evaluators listen to a sample of 10  calls monitored by the carrier. If the call center is “blended”—as 31 of the  34 are—the CPE evaluators will listen to 5 provider calls. As a point of  comparison, if a call center with the smallest number of CSRs monitors  216 provider calls from its CSRs annually, we found that CPE evaluators  would have to draw a simple random sample of 138 provider calls annually  to estimate the percentage of correct and complete calls within a margin  of error of plus or minus 5 percent, with a 95 percent confidence level.
CPE evaluators monitoring a call center with a larger number of CSRs  would have to conduct more monitoring. For example, a call center that  monitors 1,800 provider calls from its CSRs annually, would require the  CPE evaluators to draw a simple random sample of 317 provider calls  annually to reach this same confidence level.
In July 2003, CMS introduced a pilot project that provides a second means  of monitoring carrier call centers. CMS can now remotely monitor calls by  dialing into carrier call centers and listening to calls as they occur. Remote  monitoring provides CMS with an opportunity to hear providers’  questions, as well as CSRs’ responses, firsthand. Initially, CMS staff  listened to 10 calls per month for each center, to develop a general  understanding of provider inquiries and to contribute to developing the  strategy for future monitoring. However, the staff responsible for remote  monitoring reported being overwhelmed by the burden inherent in the  task, and in January 2004, CMS reduced the number to five calls per center  per month. Staff engaged in remote monitoring can only access the audio  portion of calls, limiting their ability to thoroughly evaluate the  correctness and completeness of CSRs’ responses. In addition, CMS  officials recognize that like CPE evaluators, staff engaged in remote  monitoring lack the necessary understanding of substantive policy issues  involved in the call to determine whether CSR responses were correct.  Agency officials stated that they are studying how to best make use of this  new capability. They told us that they are uncertain whether the pilot  project will be expanded because they are not convinced that remote  monitoring is the most appropriate vehicle for evaluating the correctness  and completeness of CSR responses to provider questions.

	Conclusions

Ensuring that physicians and other providers receive correct and complete  answers to their policy-oriented questions is critical to their ability to  correctly bill Medicare for services rendered to the program’s  beneficiaries. Although policy-oriented questions may represent a small  proportion of inquiries made to call centers, it is nonetheless important to  ensure that providers can rely on the information they receive. Many call  centers serving a variety of businesses have taken advantage of skill-based  routing to identify a caller’s specific question and direct the call to the CSR  most qualified to respond. However, CMS has not done so, nor has it  developed other strategies to improve the ability of CSRs to respond to  inquiries from providers. Our test calls continue to show that carrier call  centers do not adequately respond to policy-oriented questions. When  responding to our questions, it was evident that CSRs lacked access to  comprehensive materials that would facilitate correct and complete  answers. Instead, CSRs relied on fragmented information sources and  were also confused about Medicare policy issues. Furthermore, while CMS  is attempting to enhance its monitoring efforts, it has not established a  program that can sufficiently evaluate whether providers receive correct  and complete answers to their policy-oriented questions. Now that CMS  has been given new authority to contract with a variety of entities to assist  it with managing the Medicare program, it should take the opportunity to  improve its communications with providers. The new law gives CMS the  opportunity to identify organizations that can best assist it with developing  policy expertise among staff who respond to providers’ questions,  improving access to policy-oriented materials, and enhancing call center  monitoring.

	Recommendations for Executive Action

In order to improve the accuracy and completeness of responses to policy- oriented inquiries from providers, we recommend that the Administrator  of CMS take steps to ensure that all CSRs have the necessary tools to  respond to such calls. Specifically, we recommend that the Administrator  take the following three actions:    Create a process to routinely screen and triage calls by routing complex  policy-oriented questions to staff with the expertise to adequately address  them.
Develop clear and easily accessible policy-oriented materials to assist  CSRs. The materials should be electronically searchable so that CSRs can  expeditiously provide correct and complete responses to policy-oriented  questions.
Establish an effective monitoring program for call centers to assess CSRs’  performance. The program should include the development of specific  performance standards that will allow CMS to thoroughly and routinely  measure the correctness and completeness of information given by CSRs  in response to policy-oriented questions.

	Agency Comments

In written comments on a draft of this report, CMS expressed its  commitment to improving communications with providers and generally  agreed with our recommendations. CMS agreed with our first  recommendation to create a process to routinely screen and triage calls.  CMS said it plans to establish a tiered system using specialty staff to  respond to provider inquiries by fiscal year 2005. CMS also agreed with our  second recommendation that clear and easily accessible policy-oriented  materials should be available and electronically searchable for CSRs.  However, they went further to state that clear and accessible information  will also be available to specialty staff tasked with responding to complex  policy-oriented questions. CMS also described its efforts to make  information available to providers through customized Web pages and  other educational materials. While CMS agreed with the concept of  establishing an effective monitoring program—our third  recommendation—it stated that it is in the process of determining how to  do so once its new approach of triaging calls is implemented. CMS also  said it is exploring other initiatives to enhance monitoring such as  modifying its CPE requirements, developing performance-based standards  for provider telephone inquiries, and surveying providers on their  satisfaction with call centers’ performance.
We have reprinted CMS’s letter in appendix III. CMS also provided us with  technical comments, which we have incorporated as appropriate.
As agreed with your office, unless you announce its contents earlier, we  plan no further distribution of this report until 30 days after its issuance.  At that time, we will send copies to the Administrator of CMS and other  interested parties. We will then make copies available to others upon  request. In addition, the report will be available at no charge on GAO’s  Web site at http://www.gao.gov.
If you or your staff have any questions about this report, please call me at  (312) 220-7600. An additional GAO contact and other staff who made  contributions to this report are listed in appendix IV.

Appendix I: Scope and Methodology

To determine carriers’ effectiveness in providing correct and complete  responses, we placed 300 calls to 34 carrier call centers. We searched a  variety of carriers’ Web sites and compiled a group of 18 questions that  physicians and other Part B providers frequently asked when contacting  call centers. The questions represented common, policy-oriented questions  concerning the proper way to bill Medicare in order to obtain payment  from the program, as opposed to status-oriented claims inquiries. We  asked Center for Medicare & Medicaid Services (CMS) officials to review  our questions to determine whether they considered any of the 18  questions inappropriate for our test calls. For example, we did not want to  pose a question that would unfairly test a customer service representative  (CSR) knowledge and thus we did not want to include a question on a new  or recently changed policy. We also did not want to pose questions that  were the subject of an ongoing controversy. Based on input from CMS  officials, we eliminated three questions. The first question we eliminated  referred to pricing rules for multiple surgeries. CMS officials stated that  the response to this question, as posted on a carrier’s Web site, was  unclear. The second question we excluded referred to reimbursements for  nurse practitioner services. CMS officials also expressed hesitancy about  the phrasing of this question and answer as shown on a carrier’s Web site.  Although CMS ultimately suggested language to rephrase this question and  answer, we opted to remove the question from consideration. The third  question we eliminated referred to an issue that CMS officials told us was  the subject of an ongoing lawsuit.
After obtaining CMS’s input, we selected 4 questions (see app. II) from the  remaining 15 questions. These questions addressed: (1) billing for  beneficiaries transferred from one hospital to another, (2) billing for  services delivered by therapy students, (3) billing for multiple surgeries for  the same patient on the same day, and (4) billing for an office visit and  procedure for the same patient on the same day. We did not inform CMS  before making the calls of the final questions we selected. Each of the four  questions was randomly assigned across the 34 carrier call centers and  each question was posed 75 times. Calls were placed at different times of  day and different days of the week from October 20 through November 3,  2003. Twenty-eight of the carrier call centers were called nine times and  the remaining six call centers were each called eight times.
To facilitate our calls, CMS officials informed call center managers of our  test. They also agreed not to disclose any of the potential questions to  carrier call center staff. During our calls, we identified ourselves as GAO  representatives and asked each CSR to answer our question as if we were  providers. To prevent us from biasing CSRs’ responses and to ensure  fairness, we read each question to CSRs without offering additional  information or explanations. However, we repeated questions upon  request. Prompts were only given if the CSR probed for more specific  information or gave conditional responses that depended upon different  circumstances. In those situations, we asked the CSR to provide the  correct answer for each set of circumstances. Following the response, we  asked the CSR if there was any additional information he or she would like  to provide. We also told CSRs we were manually recording their responses  verbatim. We analyzed the CSR responses and simultaneously submitted  them to Medicare coding experts at CMS. Our assessment of CSR  responses and the coding experts’ verification of results relied on the  following criteria:    Correct and complete: The answer provided enough information to  correctly bill the Medicare program, including (1) a correct explanation of  how to apply the billing policy and (2) correct billing codes or a referral to  specific documentation that provided coding information.
Partially correct or incomplete: The answer provided some explanation,  but (1) did not provide assistance in interpretation or warn about special  circumstances that would affect billing; (2) provided interpretation but no  directions to specific documentation; or (3) was correct, but not  sufficiently complete to ensure that the claim would consistently pass  claims processing edits.  Incorrect: The answer contained fully or partially incorrect information,  such that a physician might incorrectly bill or not file a claim for a billable  service.
Following CMS’s verification, we discussed and resolved all discrepancies  between our assessment of responses and CMS’s verification. For  example, when initially assessing CSRs’ responses, we attempted to locate  Web site documents that CSRs referred to during our call. If we found that  the reference contained all the accurate information necessary to bill the  program properly, we considered the CSRs’ responses to be correct and  complete even if they did not tell us the information themselves. Although  CMS coding experts did not initially review these documents, and  therefore may have considered the CSR’s response to be incorrect and  incomplete, they subsequently agreed that this was a fair and appropriate  criterion to add to our assessment.
The results from our 300 test calls are limited only to those calls and are  not generalizable to the population of calls routinely made to call centers  by providers. Although the four policy-oriented questions we posed were  frequently asked questions obtained from carrier Web sites, they do not  encompass all of the questions that providers might ask.
We also interviewed carrier and CMS officials to determine what efforts  the agency had in place to enhance CSRs’ access to information. We  reviewed information regarding the Next Generation Desktop (NGD)  application, including videotape and internal documents outlining the  phase-in schedule for the application. In addition, we observed a  demonstration of the NGD application and monitored its capability to  facilitate CSR’s responses to providers’ questions.
To determine the efforts CMS has made to provide oversight, we identified  CMS requirements for carrier call center operations and discussed with  CMS staff the agency’s oversight and monitoring of carrier call center  activities. We reviewed the agency’s protocol for its contractor  performance evaluations (CPE) to determine the scope of work evaluators  would perform and evaluated carrier call center performance standards to  identify the types of problems found during their site visits. We also  reviewed CPE reports from fiscal years 2001 and 2002. CMS did not  perform any CPEs in fiscal year 2003. We visited two carrier call centers  and consulted an industry expert on issues related to call center  technology and standards. In addition, we observed carrier call centers’  monitoring of calls for quality at one of the carrier call centers we visited.  We also observed CMS regional staff performing remote monitoring of  provider calls. We performed our work from September 2003 through June  2004 in accordance with generally accepted government auditing  standards.

Appendix II: Carrier Call Center Accuracy Test Questions

The questions and answers we used to test the accuracy of carrier call  center responses to policy-oriented questions are shown in table 2.

Appendix III: Comments from the Centers for Medicare & Medicaid Services

Appendix IV: GAO Contact and Staff Acknowledgments

	GAO Contact

	Acknowledgments

Shaunessye Curry, Margaret Weber, Helen Chung, Mary Reich, and Marie  Stetser made key contributions to this report.

	GAO’s Mission

The Government Accountability Office, the audit, evaluation and  investigative arm of Congress, exists to support Congress in meeting its  constitutional responsibilities and to help improve the performance and  accountability of the federal government for the American people. GAO  examines the use of public funds; evaluates federal programs and policies;  and provides analyses, recommendations, and other assistance to help  Congress make informed oversight, policy, and funding decisions. GAO’s  commitment to good government is reflected in its core values of  accountability, integrity, and reliability.

	Obtaining Copies of GAO Reports and Testimony

The fastest and easiest way to obtain copies of GAO documents at no cost  is through GAO’s Web site (www.gao.gov). Each weekday, GAO posts  newly released reports, testimony, and correspondence on its Web site. To  have GAO e-mail you a list of newly posted products every afternoon, go to  www.gao.gov and select “Subscribe to Updates.”

		Order by Mail or Phone

	To Report Fraud, Waste, and Abuse in Federal Programs

	Congressional Relations

	Public Affairs

Summary:
Why GAO Did This Study

In 2002, GAO reported that the Centers for Medicare & Medicaid Services (CMS) needed to improve its communications with providers who deliver medical care to beneficiaries. GAO reported that 85 percent of the responses it received to 61 calls made to call centers operated by Medicare carriers--contractors that help manage the Medicare program--were incorrect or incomplete. GAO also found that CMS's primary oversight tools were insufficient to ensure accuracy in communication. GAO was asked whether call centers now provide correct and complete information to providers. GAO (1) reviewed carriers' effectiveness in providing correct and complete responses to policy-oriented telephone inquiries and CMS's efforts to improve communications with providers and (2) evaluated CMS's efforts to provide oversight of carrier call centers.

What GAO Found

Only 4 percent of the responses GAO received in 300 test calls to 34 call centers were correct and complete. GAO posed four policy-oriented questions 75 times each to carrier call centers. The level of correct and complete responses for each individual billing question ranged from 1 to 5 percent. The majority of remaining responses were incorrect, or partially correct or incomplete. Several factors, including fragmented sources of information, confusing policy information, and difficulties in retaining the CSRs responding to calls appear to account for the lack of correct and complete answers. There are many call centers serving other industries that triage incoming calls by first identifying the nature of the call and then distributing it to the CSR who is best qualified to respond. Although CMS has not adopted this approach, it is currently implementing two other initiatives that may improve CSRs' access to information. However, neither initiative is specifically designed to support CSRs responding to policy-oriented questions. In addition, CMS's efforts to provide oversight of carrier call centers are inadequate. Although CMS requires carriers to monitor the performance of their call centers, the standards used and the technological resources available to evaluate performance do not allow carriers to thoroughly assess whether CSRs' responses are correct and complete. In addition, CMS's own monitoring efforts are too infrequent. CMS only performed one contractor performance evaluation related to carrier telephone services in fiscal year 2002 and none were performed in fiscal year 2003. Moreover, when performed, these evaluations did not provide sufficiently detailed information to assess CSRs' performance.