Federal Contact Centers: Mechanism for Sharing Metrics and Oversight Practices along with Improved Data Needed
Published: 2006-02-08T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: Federal agencies have increasingly relied on contact centers--centers handling inquiries via multiple channels such as telephone, Web page, e-mail, and postal mail--as a key means of communicating with the public. Many of these centers are contractor-operated. Concerns exist about the accuracy of responses provided through contractor-operated centers. This report examines (1) the extent to which the contract terms and oversight practices for contact centers at selected agencies emphasize the importance of providing accurate information to the public, and (2) whether guidance for the operation of contact centers and basic information needed to provide general oversight exist. GAO reviewed one contractor-operated contact center at each of six agencies: the Centers for Disease Control and Prevention (CDC), General Services Administration (GSA), U.S. Postal Service (USPS), and the Departments of Defense, Labor, and Education (DOD, DOL, and Education).
section_title: What GAO Found
paragraphs: The contracts and oversight practices for the contact centers of the six agencies reviewed, which handle millions of inquiries annually, varied significantly regarding the emphasis they placed on providing accurate information to the public. Although federal policy for disseminating information to the public specifically emphasizes accuracy, only four of the six agencies include accuracy as a performance metric in their contracts. With respect to oversight, only two of the six agencies used all four of the accuracy-related oversight practices we identified--regular knowledge database reviews, regular contact monitoring, postcontact customer satisfaction surveys, and validation of contractor reports. Although each agency used some form of oversight to assess the accuracy of the information provided by its contact center, each agency differed regarding how it implemented these practices. There is no governmentwide guidance or standards for operating contact centers--including guidance on specifying accuracy as a contract performance metric or as a key focus for oversight. Some agencies indicated that had federal guidance been available, it would have helped them establish performance indicators and develop oversight policies and practices. Recognizing the need for operational standards for contact centers, an interagency working group recently proposed draft guidelines to OMB and other federal agencies, but OMB has no plans to issue these guidelines or any standards for use by agencies. Additionally, until recently the federal government had not collected data on the universe of federal contact centers. OMB and GSA attempted to collect data on the number, types, and costs of federal contact centers in 2004, but the data collected were incomplete. In addition, no governmentwide procurement information was reported to the Federal Procurement Data System (FPDS) in fiscal years 2000 through 2004 using the reporting code for telephone call centers, which OMB said is the appropriate code for contact centers. The five agencies we reviewed that report data to FPDS used a variety of different codes, some because they believe that the telephone call center code is too narrow to cover the services of their multichannel contact centers.

section_title: Letter
paragraphs: Each year millions of citizens contact the government for information  critical to their health, finances, or general well-being. In recent years,  federal agencies have increasingly relied on contact centers—centers  handling inquiries via multiple channels such as telephone, Web page, e- mail, and postal mail—as an important means of communicating with the  general public. This was demonstrated most recently during the response  to Hurricane Katrina. One of the principal ways the federal government  delivers results, according to the Office of Management and Budget  (OMB), is by providing timely and accurate information to citizens.  Improving access to high-quality government information and services is  an important focus for OMB’s current electronic government initiative,  which is an element of the President’s Management Agenda. Information  provided to the public can be evaluated by various measures, including  accuracy, timeliness, completeness, courtesy, and overall customer  service satisfaction. OMB guidance specifies accuracy as one of the basic  elements of quality for information provided by agencies to the public.  Incorrect information provided in a fast and courteous manner may be  satisfying initially, but in the long run, it may be of little use or could be  detrimental to the individual.
Many agencies have contracted with private sector firms to operate  contact centers. Recent GAO reports identified high error rates in the  accuracy of policy-oriented responses given to the public through one  contractor-operated contact center. In this context, you asked us to  review how agencies are overseeing their contractor-operated contact  centers to ensure that they provide accurate information to the public.  Specifically, we (1) identified the extent to which the contract terms and  oversight practices for selected agencies’ contractor-operated contact  centers emphasize the importance of providing accurate information to  the public, and (2) determined whether guidance for the operation of  contact centers and basic information needed to provide general oversight  exist.
To identify contract terms and oversight practices for contact centers, we  selected one contact center at each of six agencies. We selected centers  that handle over 1 million inquiries annually and provide information to  citizens that could significantly affect their finances, health, or safety.  Table 1 lists the contact centers we selected for our review and the main  public inquiry issues they handle.
For each of these centers, we interviewed agency officials and reviewed  the contract terms and performance measures, as well as the policies,  procedures, reports, and tools used to oversee and evaluate the  contractors’ operation of the contact centers. In addition, we visited four  contractor-operated centers to observe their operations and quality  control procedures. We did not independently assess the operations of the  contact centers or evaluate the effectiveness of the contractors’ quality  control processes. We identified industry practices for ensuring the  accuracy of information provided by contact centers and interviewed  representatives from two major contact center industry groups.
To determine the guidance provided to agencies and the information  gathered by the federal government about contact centers, we interviewed  OMB and GSA officials and reviewed OMB guidance, the results of OMB’s  request for information from federal agencies about their centers, GSA’s  baseline survey of a sample of agencies’ centers, and data from the Federal  Procurement Data System (FPDS). We also reviewed the report of the  GSA-sponsored interagency working group—the Citizen Service Levels  Interagency Committee—that developed potential standards for federal  contact centers. Appendix I includes a more detailed discussion of our  scope and methodology, and appendix II contains additional details on the  six contact centers we reviewed. Our work was conducted from February  through November 2005 in accordance with generally accepted  government auditing standards.
subsections: section_title: Background
paragraphs: In 2001, the President announced his management agenda for making the  government more focused on citizens and results, which included  expanding Electronic Government (E-Government). The President’s E- Government Strategy identified several governmentwide initiatives with a  goal of eliminating redundant systems and significantly improving the  government’s quality of customer service for citizens and businesses. The  expected results of the E-Government initiative include providing high- quality customer service regardless of whether a citizen contacts an  agency by phone, in person, or on the worldwide Web. The E-Government  Act of 2002 codified the President’s E-Government initiatives and  expanded OMB’s leadership role by establishing the Office of E- Government and Information Technology within OMB. The act also  requires that agencies comply with OMB E-Guidance. One of the 24  presidential E-Government initiatives is developing and deploying  governmentwide citizen customer service using industry best practices  that will provide citizens with timely, consistent responses about  government information and services via e-mail, telephone, Internet, and  publications.
By congressional direction, OMB also is responsible for establishing and  issuing governmentwide guidelines to federal agencies for ensuring the  quality of the information disseminated to the public. In response to this  direction, OMB issued guidance to agencies in February 2002 that defined  the quality of information to include accuracy as one of its fundamental  elements and directed agencies to develop procedures for reviewing and  substantiating the quality of their information before dissemination.
Contact centers are one method agencies use to disseminate information  to the public. In the past, public inquiries to the government were often  made by telephone and thus federal agencies began establishing call  centers.  With evolving technology, citizen inquiries to the government  now come through various channels such as e-mails, Web-based forms,  facsimiles, Web chat rooms, and traditional postal mail. As a result,  agencies have established multichannel contact centers to handle these  inquiries. Contact centers rely on automated and live telephone response  systems, Web site technologies, and trained customer service  representatives to provide information to the public. For contractor- operated contact centers, the agency typically provides either scripted  responses or the content from which the contractor creates its own  scripted responses. The scripts are used for the prerecorded telephone  response systems, Web pages, and preformatted responses given by the  customer service representatives. Contact centers are staffed in tiers by  generalist or specialist representatives or a combination of both. Usually,  Tier 1 staff handle general information inquiries and direct more complex  or personal issues to specialized Tier 2 or Tier 3 staff or to the agency’s  subject matter experts.
One method for obtaining information on the contact centers that are  operated by contractors on behalf of the government is to review data  from FPDS. FPDS is used to report individual procurement transactions,  which include the industrial classification of the goods and services  procured by the federal government. FPDS was implemented by OMB’s  Office of Federal Procurement Policy (OFPP) in 1978 in response to the  Office of Federal Procurement Policy Act of 1974 requirement of  establishing a system for collecting and developing information about  federal procurement contracts. Since 1982, GSA has administered FPDS  on OFPP’s behalf. In 2003, the system was revised and is now called FPDS- Next Generation.  A wide range of users, including those with the  executive and legislative branches, rely on FPDS data for information on  agency contracting actions, governmentwide procurement trends, and  achievement of goals related to small business.
section_title: Emphasis on Providing Accurate Information to the Public Varies
paragraphs: The six agencies we reviewed emphasized accuracy of contact center  information to varying degrees through the quality assurance mechanisms  of their contracts and various oversight practices. Four of the six included  a specific metric to measure contractor performance related to providing  accurate information to the public, but only one of the six used all four of  the oversight practices we identified—such as actively monitoring  contacts—to ensure that accurate information is provided to the public.
subsections: section_title: Most Agencies’ Contract Metrics Include Accuracy, but Some Did Not
paragraphs: Each of the six agencies we reviewed specified key performance metrics  that its contractor is required to meet. These performance metrics define  the minimum level of quality acceptable to the agency and provide the  basis against which the contractor is to be evaluated. We found that four  of the six agencies’ contracts included accuracy of information in one or  more of the key performance metrics. The remaining two agencies did not  have specific metrics that addressed the need to provide accurate  information to the public. Table 2 summarizes the key performance  metrics specified in the contracts we reviewed and indicates through  shading those that specifically address providing accurate information.
section_title: Agencies’ Contractor Oversight Practices Vary
paragraphs: The Federal Acquisition Regulation requires agencies to perform and  document oversight of their contractors’ performance to ensure the  government receives high-quality services as specified in the contract.  Agency oversight provides quality assurance independent of the  contractors’ own quality control processes. Although each agency  employed some oversight practices, only two of the six agencies we  reviewed used all four of the oversight practices we identified for ensuring  that accurate information is provided to the public. Each agency  emphasized accuracy of information to varying degrees within its  practices.
On the basis of our review of industry contact center practices and the  practices employed by the agencies considered leaders in government  contact centers, we identified four agency oversight practices related to  ensuring that accurate information is provided to the public via a  contractor-operated contact center.  Table 3 describes the four accuracy- related oversight practices.
The first two practices, knowledge database management and agency  contact monitoring, provide direct oversight regarding accuracy of  information, because they focus on detecting inaccuracies in the source  information used to provide responses to the public and in the actual  responses provided by the customer service representatives. The  remaining two practices, customer satisfaction surveys and validation of  contractor-prepared reports, are more indirect methods of ensuring  accuracy in that they review customers’ reactions to the information  provided and independent agency corroboration of the contractor’s  reporting on its own quality procedures. The agencies we reviewed varied  with respect to how they implemented these practices. This variance was  due to a number of factors, such as differences among the agencies in  staffing levels, funding, and the use of guidance specific to the agency.  Table 4 shows the extent to which the six agencies we reviewed employ  each of the accuracy-related oversight practices.
Most of the agencies we reviewed had a structured process for ensuring  accurate information is maintained in the knowledge database. DOL,  Education, GSA, and USPS approve contractor-developed information that  is created based on government-provided materials. These agencies then  perform periodic reviews of the information in the knowledge database.  CDC currently prepares all scripted responses and Web site information,  which the contractor is required to use, and plans to implement annual  reviews of the knowledge database, starting at the first anniversary of  operation in February 2006. TMA allows the contractor to develop  information based on material TMA provides, but does not review the  information used by the contractor to respond to public inquires. DOD  said that TMA relies on the expertise and skills of its contractor to provide  the required services.
subsections: section_title: Oversight Practice 2: Regular Agency Contact Monitoring
paragraphs: Almost all of the agencies we reviewed perform regular monitoring of the  contractor’s responses to the public to help assess whether accurate  information is provided. CDC, DOL, Education, GSA, and USPS each  monitor a number of contacts on a regular basis, although accuracy of  information is addressed to varying degrees in the score sheets. For  example, accuracy is clearly weighted as an important aspect of the call in  CDC’s score sheet. Therefore if an inaccurate answer is provided, the  contractor “fails” for that call and the customer service representative is  counseled. On the other hand, Education’s score sheet does not clearly  weight accuracy of information. Education and its contractor staff could  not explain how providing inaccurate information on a call would be  indicated on the monitoring score sheet. In addition to giving different  weights to accuracy, the five agencies also vary in terms of the frequency  with which they monitor their contacts. Education and USPS each employ  one full-time staff to monitor a selection of the contact centers’ contacts.  CDC has a third-party contractor monitor the contact center on a daily  basis and uses this assessment in the determination of the contractor’s  award fee. GSA staff monitor a sample of calls on a weekly basis and  started performing quarterly audits of the contractor’s monitoring efforts  in November 2005. The sixth center, TMA, only monitors calls on an ad  hoc basis when officials visit the contact center.
section_title: Oversight Practice 3: Postcontact Customer Satisfaction Surveys
paragraphs: Three of the agencies we reviewed conduct customer satisfaction surveys  subsequent to the initial contact from an individual. GSA, TMA, and USPS  conduct customer satisfaction surveys, which ask, to limited degrees,  questions that address the accuracy of information provided. While  providing some level of insight regarding accuracy, customer surveys may  not always provide a valid basis for oversight of the accuracy of  information, since they usually ask the individual’s opinion on the service  provided. If the survey is conducted too closely to the time of the inquiry,  the individual may not have had time to act upon the information to know  whether it is accurate or not. CDC plans to implement three types of  postcontact customer satisfaction surveys through a third-party contractor  beginning in June 2006. DOL does not conduct postcontact surveys  because it does not maintain personal information on the individuals that  contact the agency.
section_title: Oversight Practice 4: Validation of Contractor Reports
paragraphs: Three of the six agencies we reviewed take steps to validate the  information in the contractor-prepared reports related to contact center  performance. These reports generally include some aspects related to  accuracy of information provided to the public, such as the contractor’s  results of its monitoring of contacts. CDC and USPS validate to some  degree the reports provided by the contractor. GSA conducts quarterly  audits of its contractor’s supporting data. Although DOL, Education, and  TMA review their contractor reports, they rely upon the reports without  validation. According to GAO’s standards for internal control in the federal  government, good internal control practices require that agencies validate  the performance reports provided by the contractor to ensure the  information is valid.
section_title: Governmentwide Guidance and Information on Contact Centers Do Not Exist
paragraphs: The federal government does not have comprehensive, centralized  guidance for operating a contact center or for overseeing a contractor- operated center. Although operation and oversight of contact centers are  the responsibility of individual agencies, GSA, in consultation with OMB,  determined that governmentwide standards would be useful. GSA  sponsored an interagency committee that recently provided draft  guidelines for operating federal contact centers to OMB and other federal  agencies. However, OMB told us it does not plan to issue any  governmentwide guidance based on the committee’s recommended  guidelines at this time, because OMB has not identified the operation of  contact centers as an area of concern. Furthermore, until recently, no  governmentwide information specific to contact centers has been  collected. Initial attempts to gather governmentwide information about the  number and type of activities that agencies use to provide public  information proved to be inadequate for providing a comprehensive  governmentwide view of contact centers. In addition, officials from the  agencies we reviewed told us that no industry classification code in FPDS  currently covers the full range of services provided by a contact center.
subsections: section_title: Limited Federal Guidance for Operating and Overseeing Contractor- Operated Contact Centers
paragraphs: In its 2004 report on the electronic government initiative, OMB highlighted  the importance of delivering timely and accurate information to the public  and stated that there are opportunities to apply existing and emerging best  practices to achieve increases in productivity and delivery of services and  information. To date, however, OMB has established only limited  guidance on preferred practices at contact centers. The only OMB  guidance we found that specifically related to contact centers is focused  on the use of performance-based contracting for such services. This  guidance is dated and limited in its coverage and does not provide  guidance on performance metrics for contact centers or oversight  practices.
Because of the need for governmentwide standards for operating contact  centers, GSA in consultation with OMB, took the initiative to form an  interagency working group to propose guidelines to OMB and other  federal agencies. Formed in March 2005, the Citizen Service Levels  Interagency Committee is composed of 58 contact service representatives  from 33 executive branch agencies. In addition to relying on their  knowledge in running contact centers, the committee also had a  contractor perform two studies to provide insight on citizens’ expectations  when contacting government agencies for information and current  industry metrics, benchmarks, and best practices for operating contact  centers. The committee submitted a report with 37 proposed standards for  operating contact centers to OMB in September 2005, including four  standards specifically related to ensuring accuracy. The committee plans  to continue to work on additional contact center issues and to help  agencies implement any contact center standards that OMB might  endorse. In October 2005, OMB officials stated that they had reviewed the  committee’s report but did not plan to issue any governmentwide guidance  based on the committee’s recommended guidelines at this time, because  OMB has not identified the operation of contact centers as an area of  concern. OMB stated further that if agencies need additional guidance in  developing their standards, they can refer to the committee’s report.
The agencies we reviewed each performed independent research to  develop their contracts and formulate a management strategy for  operating their contact centers. Performing independent research resulted  in duplication of efforts across agencies, using limited resources and  taking valuable time. For example, to develop guidance, the Department of  Education performed market research, worked with a contractor on  customer services and related standards, and studied industry best  practices on a limited basis. Similarly, CDC sent out a request for  information to industry to gain insight on the technology available for  operating contact centers before it developed its contract. CDC then  performed market research, reviewed industry practices, and visited other  government contact centers, such as that of the Social Security  Administration and the Centers for Medicare and Medicaid Services, to  learn about the practices of government- and contractor-operated centers.
In 2004, GSA created a multiple-award contract called FirstContact to  assist agencies in contracting for contact center services. Under this  multiple-award contract, agencies can issue a task order to any of five  preapproved contractors to operate a contact center. Using FirstContact  will minimize the time and effort required of agencies to locate a  contractor to manage their centers. To date, the Department of Homeland  Security’s Federal Emergency Management Agency, GSA, and the  Department of Health and Human Services have placed six task orders  against this contract, and three other agencies are looking to place orders  as well. For example, the Federal Emergency Management Agency  recently used this contract vehicle to quickly provide contact center  services for the influx of calls and applications for government assistance  in the aftermath of Hurricane Katrina. Although agencies now have this  multiple-award contract as a mechanism to assist them in contracting for  contact center service, they still must develop specific performance  metrics and oversight practices specific to their center.
section_title: Governmentwide Data on Contact Centers Do Not Exist
paragraphs: Given the lack of governmentwide information on the activities that  provide information to the public, OMB made an initial attempt to collect  such data in 2004. For this effort, OMB issued a data request to all  executive branch agencies to obtain basic data, such as the contact center  name, volume of contacts, and whether performance and cost metrics are  collected, for every activity that provides information directly to the  public. OMB normally uses data requests as a census tool to acquire a  snapshot of the current budget environment of the government. Agencies  responded by self-identifying over 1,800 activities that currently provide  information to citizens using various communication channels such as  telephone, e-mail, and Internet Web sites. The individual activities  identified ranged in size from a couple of employees who answer  telephone calls as part of their duties to contact centers with staffs of  several hundred employees who handle millions of inquiries through  several channels. Of the 1,800 activities identified, over 500 categorized  themselves as contact centers. Since it was making a nonstandard data  request, OMB performed little follow-up on nonresponding agencies and  did not verify reported results. We noted that some large agencies, such as  DOD, did not report any activities that provide information to the public.
In an effort to expand on the information collected through the OMB data  request, GSA surveyed 360 activities—approximately a quarter of those  who responded to the data request—to develop a baseline snapshot of  governmentwide activities providing information to the public. However,  GSA’s survey methodology was flawed because the agency selected its  sample from an incomplete universe, had a low survey response rate, and  did not perform a nonresponder analysis. Thus, the survey results did not  provide a representative view of activities across the government. GSA  plans to conduct a follow-up survey of government activities in 2007.
While OMB and GSA information regarding the universe of federal contact  centers is incomplete, another potential source of information on those  contact centers that are contracted out by the government is the FPDS.  Since its inception in 1978, the FPDS has served as the governmentwide  system for collecting federal procurement data. Five of the six agencies we  reviewed, however, each used a different code to report their contact  center procurement actions to FPDS. Officials from agencies we  reviewed told us that no current North American Industry Classification  System (NAICS) code covers the full range of services provided by a  contact center. Table 5 lists the NAICS codes used by the five agencies  reporting data to FPDS.
The five agencies that reported to FPDS chose different NAICS codes for  different reasons. Officials from DOL and GSA stated that they chose  alternative NAICS codes because the definition provided for telephone call  center does not cover all of the activities handled in a contact center.  Education and DOD-TMA officials explained that they chose NAICS codes  that encompassed the main work of the contract, since the contact center  is only a portion of the work in a contract for a larger program. CDC chose  its NAICS code based upon the information technology services required  for creating its contact center.
No governmentwide procurement information was reported to FPDS using  the NAICS codes for telephone call centers in fiscal years 2000 through  2004. This category of NAICS codes—56142—is defined as establishments  primarily engaged in answering telephone calls and relaying messages or  in telemarketing activities. Although officials from three of the agencies  we reviewed expressed the opinion that the definition for telephone call  centers is too narrow to encompass all the work performed by a contact  center, OMB told us that the telephone call center code is the correct code  to use. Specifically, OMB stated that the subcode of 561422— telemarketing bureaus—was written with the intent to cover all the  functions of a contact center. OMB is considering issuing a clarification to  the description of the 56142 codes to explain that these codes include  more than telephones—such as Web sites, e-mails, facsimiles, and so  forth—in its next update to the NAICS manual in 2012.
section_title: Conclusions
paragraphs: Providing timely and accurate information is a key result area for the  federal government. Federal agencies that use contractor-operated contact  centers to meet the public’s demand for information assume the burden of  assuring that the information provided by the contractors is accurate.  While the agencies we reviewed have taken a variety of steps to ensure  that their contractor-operated contact centers address accuracy, at some  agencies accuracy clearly does not have the same priority as other  objectives, such as timeliness. Although agencies need flexibility in  meeting the needs of the individuals that contact them, they also can  benefit from the experience gained by their peers operating other  government contact centers. Short of mandating specific guidance,  increased sharing among agencies of successful practices for managing  contact centers may help improve their abilities to write and oversee  contracts for these centers and may avoid needless duplication of effort.  The guidelines proposed by the Citizen Service Levels Interagency  Committee are a step in this direction. However, whether this effort will  result in information sharing across agencies is uncertain. Leveraging  knowledge gained by other agencies through the sharing of effective  practices could be enhanced by governmentwide leadership. OMB’s  leadership of the electronic government initiative, its role in guiding  agency dissemination of public information, as well as its procurement  policy role, put it in an ideal position to facilitate the exchange of  information among agencies to ensure effective oversight of contractors in  meeting the public’s need for timely and accurate information.
While OMB and GSA have taken initial steps to enhance the oversight of  federal contact centers by gathering some information on the universe of  these centers, it is not clear whether the data collected provide enough  information for governmentwide oversight of contact center operations or  whether GSA’s planned data collection efforts will do so either. With  additional reliable information, OMB may be able to more quickly identify  and act on emerging problems and opportunities. In addition, FPDS can  also be more effective in identifying the number of contracts and dollars  obligated for contact centers across the government, but only if the  agencies consistently use the appropriate NAICS code for these services.
section_title: Recommendations for Executive Action
paragraphs: To facilitate the sharing of sound oversight practices for the operation of  contact centers, to help ensure that providing accurate information to the  public by contact centers is a priority outcome, and to improve the quality  of information gathered about these centers, we recommend that the  Director of the Office of Management and Budget take the following  actions:    Building on efforts begun by the GSA-sponsored interagency  committee, work with agencies to develop a mechanism for sharing  performance metrics and oversight practices for contact centers.  Continued efforts should stress that providing accurate information to  the public needs to be a key factor in the oversight of federal contact  centers.
Take steps to ensure consistent reporting on contact centers by  developing an industry category or specific code definition in NAICS  that encompasses all the services provided by contact centers or by  providing further instruction to agencies regarding the appropriate  NAICS code to use for contact centers.
To improve the quality of information about federal contact centers, we  recommend that the Administrator of General Services take the following  action:    Ensure that further efforts to develop governmentwide data on contact  center operations—such as the survey planned for next year—employ  sound methodologies to ensure that the resulting information is  representative of the activities across the government.
section_title: Agency Comments and Our Evaluation
paragraphs: We requested comments on a draft of this report from the Office of  Management and Budget and each of the six agencies we reviewed— Department of Defense, Department of Education, Department of Health  and Human Services, Department of Labor, General Services  Administration, and U.S. Postal Service. The Office of Management and  Budget provided oral comments in which they concurred with our findings  and recommendations. The Department of Defense, Department of Health  and Human Services, and the General Services Administration provided  written comments that are reproduced in appendices III, IV, and V,  respectively. OMB and most of the agencies also provided technical  comments, which we incorporated as appropriate.
The Department of Health and Human Services and the General Services  Administration also concurred with our findings and recommendations.  The Department of Defense did not concur with our draft report because it  believes the report does not fully reflect all the metrics and practices DOD  and its contractor use to ensure the accuracy of information provided to  TRICARE beneficiaries. In its comments, DOD emphasizes that its  approach to contracting for contact center operations relies on the  contractor to use industry standards for ensuring information accuracy.
DOD states that standards exist in its contract related to the accuracy of  information provided by telephone.  DOD also cites additional metrics it  uses for monitoring contractor performance.  In addition, DOD requires  the contractor to have a quality management program which must be  validated by a nationally recognized third-party organization.  DOD points  out that it receives monthly briefings on the operation of the contractor’s  quality management program and observes call center operations during  site visits.  Finally, DOD explained that it monitors the expertise and skills  of the contractor staff that perform the knowledge management function.
We recognize that DOD has decided to use what it calls the “audit the  auditor” approach to quality assurance.  It was not our objective, however,  to assess the merits of any particular approach to ensuring quality, but  rather to determine the extent to which contract terms and agency  oversight practices emphasize the importance of providing accurate  information to the public.  In this regard, while the contractor may use  specific standards for accuracy in its quality management program, we  found no specific metric related to accuracy in the TRICARE contact  center contract itself or in the additional metrics cited in DOD’s  comments.  For the most part, the additional quality control activities  listed by the Department are those of its contractor, not oversight  activities performed by the agency, which was the focus of our review.   While independent validation of the contractor’s quality control program  helps to ensure the contractor has a quality process in place for  monitoring its responses to the public, this does not substitute for DOD  oversight activities such as validating the contractor’s reports of its  monitoring efforts.  In addition, while DOD performs site visits to oversee  the contractor’s operations, it does so only on an ad hoc basis. Based on  DOD’s comments, we added additional language to the report regarding  DOD’s approach to knowledge management.
As arranged with your offices, unless you publicly announce its contents  earlier, we plan no further distribution of this report until 30 days after the  date of this report.  We will then send copies of this report to the Director  of the Office of Management and Budget, the Administrator of General  Services, the Postmaster General, and the Secretaries of the Department of  Defense, Department of Education, Department of Health and Human  Services, and Department of Labor. We will also make copies available to  others upon request. Contact points for our Office of Congressional  Relations and Public Affairs may be found on the last page of this report.   This report is available at no charge on GAO’s Web site at  http://www.gao.gov.
If you or your staff have any questions about this report, please call me at  (202) 512-4841. An additional GAO contact and staff who made  contributions to this report are listed in appendix VI.
section_title: Appendix I: Scope and Methodology
paragraphs: To assess the guidance provided to federal agencies and the information  gathered by the federal government about contact centers, we conducted  interviews with Office of Management and Budget (OMB) and General  Services Administration (GSA) officials, and reviewed related guidance  and results of their initial data collection efforts. We researched and  discussed with OMB the absence of guidance related to the operation and  oversight of contact centers. We also discussed the results of OMB’s 2004  request for information to federal agencies that asked for self- identification of any activities that provide information to the public. In  addition, we reviewed and discussed the results of GSA’s survey of a  sample of agency activities that responded to OMB’s request. We did not  assess the validity of the data gathered by OMB and GSA. However, a GAO  methodologist reviewed the GSA survey methodology and identified its  weaknesses. In addition, we monitored the progress of the GSA-sponsored  working group—the Citizen Service Levels Interagency Committee—as it  developed and recommended standards to OMB for federal contact  centers. We did not assess the committee’s recommendations as a whole,  but rather reviewed how accuracy of information was addressed within its  proposed standards. We reviewed data from the Federal Procurement  Data System for the past 5 fiscal years to determine if any contract actions  were reported using the code for telephone call center services.
To describe federal agencies’ efforts to ensure accurate information is  provided to the public by contractor-operated centers, we reviewed the  contract terms and oversight activities for one center at each of six  agencies. We selected centers that handle over 1 million inquiries annually  and provide information to citizens that could significantly affect their  finances, health, or safety. The contact centers selected for our review are    Department of Defense TriCare Management Activity (TMA) North  region—Healthnet’s contact center: provides general and personalized  medical benefit and coverage information and processes enrollments  and claims for military families in the North region;    Department of Education (Education)—Federal Student Aid  Information Center: provides general information about applications  and loan issues and personalized information on the status of  applications and loans to the public and academic community;    Department of Health and Human Services’ Centers for Disease  Control and Prevention (CDC)—CDC INFO contact center: provides  information about health and safety issues—including prevention,  detection, and outbreak control—to the public and medical  professionals;    Department of Labor (DOL)—National Contact Center: provides  general information and referrals regarding job issues, workplace  safety, and pension and health benefits to the public and employers;    General Services Administration—National Contact Center: provides  general information and referrals related to any agency or government  program; and    U.S. Postal Service (USPS)—National Contact Center: provides general  and individualized information on mail delivery and shipping issues to  the public and businesses.
To complete our review, we interviewed management and staff  responsible for oversight of the contractor-operated contact center at each  agency. We reviewed the performance metrics specified in the agency’s  contract as well as the related reports used to oversee and evaluate the  contractors’ operation of the contact centers. In addition to conducting  discussions with the agencies, we visited four contractor-operated centers  to observe their operations and quality control procedures. Specifically,  we visited locations for the GSA center operated by ICT Group, CDC and  Education centers operated by Pearson Government Solutions, and the  DOL center operated by Datatrac Information Services. At each center we  interviewed management and customer service representatives regarding  the oversight practices used to monitor the accuracy of information. We  did not test the contractors’ internal control procedures or validate any  data from their sample reports.
We identified industry practices for ensuring the accuracy of information  provided by contact centers, interviewed representatives from two major  contact center industry groups—the Society of Consumer Affairs  Professionals and the Incoming Calls Management Institute—and attended  the 2005 Government Customer Support Conference. In addition, we  reviewed prior GAO reports concerning contact centers. We also  discussed contact center issues with other GAO teams that were currently  reviewing or had recently reviewed other federal contact centers. Our  work was conducted from February through November 2005 in  accordance with generally accepted government auditing standards.
section_title: Appendix II: Contact Center Profiles for Six Agencies Reviewed
paragraphs: Medical benefits and  coverage issues,  enrollment, and claims  processing Daily 7:00 a.m. to 7:00 p.m.  (eastern time)
Languages supported  (beyond English)
Fiscal year 2005 call  volume  from February to September  2005  (estimated at 2.7 million  calls when fully operational)
Total value of contract at  award (in millions of  dollars) (the contact center is only a  portion of this contract)
Monday-Friday 8:00 a.m. to  midnight; Saturday 9:00 a.m. to  6:00 p.m. (eastern)
Monday-Friday 8:00 a.m. to 8:00  p.m. (eastern)
Monday-Friday 8:00 a.m. to  8:30 p.m.; Saturday 8:00 a.m.  to 6:00 p.m. (eastern time)
1-year base plus 9 1-year  options  1-year base plus 4 1-year  options  4-year base plus 6 1-year  options  $254.6 (for 4-year base only) (the contact center is only a  portion of this contract)
Firm fixed price plus award fee3  DOL provides service 24 hours a day for the Occupational Safety and Health Administration toll-free  number and provided service 24 hours a day during hurricane relief efforts; Education extends its  hours during student aid application season; GSA provides service 24 hours a day under emergency  situations.
The CDC contact center is in its second year of operation and is consolidating the work for 40  different toll-free numbers over a total period of 4 years.
The contact center is a portion of a larger service contract. The value shown here is for the entire  contract, as the agency could not provide a breakdown of the cost for the contact center alone.
section_title: Appendix III: Comments from the Department of Defense
section_title: Appendix IV: Comments from the General Services Administration
section_title: Appendix V: Comments from the Department of Health & Human Services
section_title: Appendix VI: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Acknowledgments
paragraphs: In addition to the contact named above, Ruth Eli DeVan, William McPhail,  Jean Lee, David Schilling, Nyankor Matthews, Robert Swierczek, John  Krump, Monica Wolford, and Karen O’Conor made key contributions to  this report.
section_title: Appendix VII: Related GAO Products Appendix VII: Related GAO Products
paragraphs: Improvements Needed to the Federal Procurement Data System-Next  Generation. GAO-05-960R. Washington, D.C.: September 27, 2005.
Social Security Administration: Additional Actions Needed in Ongoing  Efforts to Improve 800-Number Service. GAO-05-735. Washington, D.C.:  August 8, 2005.
Immigration Services: Better Contracting Practices Needed at Call  Centers. GAO-05-526. Washington, D.C.: June 30, 2005.
Federal Thrift Savings Plan: Customer Service Practices Adopted by  Private Sector Plan Managers Should be Considered. GAO-05-38.  Washington, D.C.: January 18, 2005.
Medicare: Accuracy of Responses from the 1-800-MEDICARE Help Line  Should Be Improved. GAO-05-130. Washington, D.C.: December 8, 2004.
Medicare: Call Centers Need to Improve Responses to Policy-Oriented  Questions from Providers. GAO-04-669. Washington, D.C.: July 16, 2004.
Reliability of Federal Procurement Data, GAO-04-295R. Washington, D.C.:  December 30, 2003.
Medicare: Communications with Physicians Can Be Improved.   GAO-02-249. Washington, D.C.: February 27, 2002.
IRS Telephone Assistance: Limited Progress and Missed Opportunities  to Analyze Performance in the 2001 Filing Season. GAO-02-212.  Washington, D.C.: December 7, 2001.
IRS Telephone Assistance: Quality of Service Mixed in the 2000 Filing  Season and below IRS’ Long-Term Goal. GAO-01-189. Washington, D.C.:  April 6, 2001.
IRS Telephone Assistance: Opportunities to Improve Human Capital  Management. GAO-01-144. Washington, D.C.: January 30, 2001.
Customer Service: Human Capital Management at Selected Public and  Private Call Centers. GAO/GGD-00-161. Washington, D.C.: August 22, 2000.
Social Security Administration: Information on Monitoring 800  Number Telephone Calls. GAO/HEHS-98-56R. Washington, D.C.:   December 8, 1997.
Social Security Administration: More Cost-Effective Approaches Exist to  Further Improve 800-Number Service. GAO/HEHS-97-79. Washington,  D.C.: June 11, 1997.
This is a work of the U.S. government and is not subject to copyright protection in the  United States. It may be reproduced and distributed in its entirety without further  permission from GAO. However, because this work may contain copyrighted images or  other material, permission from the copyright holder may be necessary if you wish to  reproduce this material separately.