Office of Workers' Compensation Programs: Goals and Monitoring Are Needed to Further Improve Customer Communications
Published: 2000-10-03T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: This testimony discusses the Department of Labor's Office of Workers' Compensation Programs (OWCP). GAO reviewed how OWCP communicates with injured federal workers, agencies who employ these persons, and medical and other service providers who treat them. To evaluate OWCP's system, GAO used criteria suggested by the National Partnership for Reinventing Government (NPR). This report summarizes GAO's findings on NPR's study of private sector practices for providing telephone customer service, which included: (1) setting challenging goals for meeting callers' needs for timely and accurate information; (2) collecting credible performance data to measure progress in attaining those goals; and (3) improving telephone service by using the performance data and results to periodic surveys of customers and stakeholders to determine levels of satisfaction.
section_title: What GAO Found
paragraphs: GAO found that OWCP provided consistent customer service regardless of where injured workers live. GAO made 2,400 telephone calls to OWCP's 12 district offices. To compare OWCP's goals and practices for telephone communication with those of model organizations, GAO surveyed three agencies that have won awards for their telephone communication practices: the Social Security Administration, the Department of Veterans Affairs' Benefits Administration, and Ohio's Bureau of Workers' Compensation.

section_title: Letter
paragraphs: Thank you for the opportunity to testify on the Department of Labor’s Office of Workers’ Compensation Programs (OWCP). You asked us to review how OWCP communicates with injured federal workers, agencies who employ these individuals, and medical and other service providers who are involved in their treatment. In general, our review focused on how well OWCP’s performance management system was used to respond to claimants’ and other customers’ inquiries, both over the telephone and in written correspondence. This statement responds to your request by describing how we conducted our review and what we found.
To evaluate OWCP’s system, we used criteria suggested by the National Partnership for Reinventing Government (NPR). NPR performed an extensive study of high performance customer service organizations in the private sector and their best practices for providing telephone service. NPR stated that the level of service a customer receives should not vary significantly across an organization. NPR also found that model customer service organizations in the private sector generally follow three consistent approaches or best practices for telephone communications. First, they set challenging goals for meeting callers’ needs for timely and accurate information. Second, they collect credible performance data to measure progress in attaining those goals. Third, they continuously improve telephone service by using the performance data and results of periodic surveys of customers and stakeholders to determine levels of satisfaction.
subsections: section_title: Background
paragraphs: OWCP is responsible for adjudicating and administering claims of work- related injuries and illnesses as authorized by the Federal Employees’ Compensation Act (FECA) (5 U.S.C. 8101 et seq., as amended).The FECA program covers nearly 3 million active duty civilian federal employees, providing benefits to those it determines sustain an injury or illness in the performance of duty worldwide.
During fiscal year 1999, FECA’s costs totaled about $1.9 billion in compensation, medical, and death benefits, and federal employees filed about 167,000 injury notices.  At the end of fiscal year 1999, OWCP was administering about 243,000 ongoing injury cases, including from previous years, for partial or total disability.
According to OWCP officials, they receive an estimated 2.6 million phone calls and 5.5 million pieces of mail each year from customers--claimants, medical providers, agencies, and others. Some mail requires a response— for example, congressional inquiries on behalf of constituents. However, district office officials said they believed that most of the mail does not require a response. For example, medical reports are used to assist claims examiners in adjudicating cases but do not usually require a response. Although OWCP did not know what proportion of its mail requires a response, district office officials’ estimates ranged from 1 percent to 7 percent.
The telephone calls and written correspondence are handled primarily by 12 OWCP district offices nationwide, which had a total of about 900 employees as of December 1, 1999. These district offices operate under the authority and guidance of OWCP headquarters and are responsible for adjudicating claims from injured workers, approving wage loss claims, paying medical bills, and responding to inquiries from customers. Each district office is responsible for providing services to claimants living in several states. expanding automated voice response systems to allow pharmacy staff to verify claimant’s eligibility and the amounts of drug payments authorized; beginning the process of converting incoming medical bills and other correspondence to a computerized format to make the information available to district office representatives via their computer terminals, enabling them to answer more queries during initial calls; giving federal agencies, unions, and congressional staff direct computer access to information they need to deal with their employees’ or constituents’ cases; and initiating a communications redesign project last year--which included establishing a redesign team comprised of union members and management to propose standards, reengineer practices, and make other improvements in OWCP’s communications--and investigating best practices in public and private organizations.
OWCP has also taken actions when its monitoring systems have indicated that district offices have failed to meet goals for responsiveness to telephone inquiries. For example, when district offices failed to meet goals for responding to telephone inquiries for one or more quarters of a fiscal year, OWCP’s national office counseled the district directors and required plans for improvement.
In addition, OWCP’s budget request for fiscal year 2001 requested funding for a toll free 800 telephone number for medical authorizations, for telephone system hardware upgrades, for additional communication specialists, and for expanded access to automated information for injured workers. As of September 22, 2000, the House and Senate appropriation committees for OWCP had decided not to fund this request.
section_title: Telephone Customer Service Levels Varied Widely Across District Offices
paragraphs: Although NPR has stated that the level of service a customer receives should not vary significantly across an organization, we found that service levels varied widely for those attempting to reach OWCP representatives by phone. As figure 1 shows, the extent to which we were unable to access district offices’ telephone systems on our 2,400 calls—that is, where there was a busy signal, no answer after 1 minute, or a message erroneously stating that the phone number was invalid—ranged from 0 percent in Boston to 54 percent in Jacksonville.
Washington, D.C., office had purchased an additional eight telephone lines in late July 2000 and that he believes this will increase the system’s accessibility.
We also found that our ability to speak to an OWCP employee varied significantly across districts. Of the 2,400 calls we made, 1,200 calls were to either an office phone number designated for contacting an employee or a central phone number that gives callers an option for contacting a representative. As figure 2 shows, the rates at which we were unable to reach any employee within 5 minutes ranged from 13 percent to 97 percent of the calls. In three offices—Jacksonville, Dallas, and New York—we were unable to access an employee on 97, 86, and 80 percent of the calls, respectively. offices with the lowest telephone access rates by giving them part of another office’s staff allocation.
When we made our 2,400 telephone calls, we also attempted on 1,200 of those calls to compare the information on actual injured workers’ claims provided to us by OWCP headquarters officials with the same information available on that claimant through district offices’ telephone systems. For example, if OWCP headquarters told us that claimant Mary Smith was mailed a compensation check of $550, would the district office where Mary’s claim was handled provide us with this same information?
We did not include 604 of the 1,200 calls in our analysis: 138 calls where we could not access the phone system for various reasons (e.g., busy signal); 43 calls where we could access the phone system but not the interactive voice response system for various reasons (e.g., claim number was different from that provided by headquarters), and 423 calls where we could not compare the information because it had been updated after OWCP headquarters provided it to us.
For the remaining 596 calls, the extent to which district offices provided us with consistent claims information ranged from 88 percent to 100 percent. (See appendix II for information on the accuracy of each district office’s interactive voice system.)  Most of the inconsistent information involved the dates and amounts of claimants’ compensation checks.
Other communication practices also varied significantly across district offices:
The Dallas office, unlike most others, used e-mail for medical authorizations, congressional contacts, and general inquiries. The four other district offices we visited did not use e-mail because of Privacy Act concerns.
The national office and four district offices have taken steps to provide customers information through the Internet, while others have not.  The Internet-linked offices have established a World Wide Web page to provide information about the workers’ compensation program and the district offices’ procedures and practices.
Most district offices had representatives available to answer the phone 7 or more hours per day, but three offices—New York, Philadelphia, and Boston--were available by phone 6 hours or less per day, and one of these offices—Boston—had representatives available for only 4.5 hours.
section_title: OWCP Did Not Have Goals for Some Important Customer Service Areas
paragraphs: OWCP had not set any goals in some important areas of telephone communications and the goals it did set for telephone and written communications allowed OWCP more time to provide responses to customers than NPR suggests for telephone communications or that other organizations we surveyed allowed.  For example, NPR suggests the following telephone service goals:
99 percent of callers access the telephone system;
98 percent of callers reach a customer service representative, and the time waiting on line be no more than 30 seconds; and
85 percent of callers’ inquiries should be resolved during the first call.
These three basic goals focus on meeting callers’ needs for timely and accurate information. The other agencies we contacted--SSA, VBA, and Ohio’s BWC--varied in whether they established goals for these measures. Three had goals for telephone access, two had goals for the portion of callers reaching representatives and the time they have to wait on line, and one had a goal for resolving inquiries on the first call.
OWCP had not set goals that conform to any of these three goals. OWCP did have a goal to return 90 percent of phone calls not related to medical authorizations within 3 days to persons who leave messages. That goal could be met by OWCP’s calling the people within 3 days, giving them the status of their claims, and saying that the answer to their questions would follow at a later time. However, OWCP’s Acting Director noted that this is the only response possible for many calls when OWCP lacks information, such as doctors’ reports, needed to resolve the caller inquiry. OWCP also had a separate goal established in fiscal year 1999 to return 95 percent of calls related to medical authorizations within 3 days. whose employees’ only responsibility is answering telephone calls. He explained that because district offices have many other responsibilities in addition to answering calls, some district offices prefer to direct most calls to voice mail and respond at a later time.  Several district directors told us that, if there were such a goal, assigning additional employees to answer calls would take time away from their adjudication of claims.
The scope of NPR’s study did not include identifying what goals the private sector has for responding to written inquiries, as it did for telephone communications.  Thus, we could not compare the goals that OWCP has established for the timeliness of written communications with an NPR suggested standard.
Nevertheless, for nonpriority mail requiring a response, OWCP had a goal of responding to 85 percent within 30 days. OWCP also had goals for responding to priority mail from Congress: 90 percent within 14 days and 98 percent within 30 days. These goals do not compare favorably to VBA’s goal of responding to all written benefit inquiries within 10 workdays and Ohio’s BWC’s goal of responding to written requests the same day, or within 24 hours of the request’s being referred to another section of the Bureau.
OWCP also did not have a national goal for responding to requests for medical authorizations in the mail, through e-mail, or by fax. Nonetheless, the five district offices we visited gave written medical authorization requests received by mail the same priority that OWCP gave congressional correspondence. Several of these five offices have also established their own goals for medical authorizations received by e-mail or fax. For example, the Dallas district office encourages claimants to use e-mail for medical authorizations. Dallas had a goal of responding to 90 percent of e- mails within 24 hours. The Chicago district office received 95 percent of its medical authorization requests by telephone. The district office has chosen to use the goal for medical authorizations received by phone—95 percent within 3 days—for authorization requests received by fax.
section_title: OWCP Lacked Valid Information to Measure Progress in Achieving Goals
paragraphs: Often, OWCP did not collect credible performance data to gauge progress in attaining its goals.  Credible performance information is essential for accurately assessing agencies’ progress toward meeting existing goals and for setting new goals. Decisionmakers must have assurance that the program and financial data being used will produce complete, credible, useful, and consistent data in a timely manner if these data are to inform decisionmaking.
subsections: section_title: Progress Measures for Telephone Timeliness Were Invalid
paragraphs: OWCP’s system for measuring its goal of 3 days for returning phone calls to those who left a message that required a response did not yield valid timeliness measurements. It could do this by either creating a record of all such calls or of a statistically valid sample. While all calls may not require a response, district office officials have stated that most callers have inquiries that require a response. The national office suggested--but did not require--that district employees use a standardized computer program (CA- 110) to make a recording of all calls requiring a response, as well as of those in which the content is relevant to adjudicating decisions.  OWCP did not require the use of this program for all calls because some district office employees have complained about taking time away from their other tasks to record the information, such as the date and nature of the call.
We found that all 12 district offices used the CA-110 system to some extent. However, two of the five offices we visited—Dallas and Seattle-- told us that they entered only about 15 percent or fewer of all calls requiring a response and did not enter calls in the systematic manner that would be necessary to yield valid results. The other three offices— Chicago, San Francisco, and Washington, D.C.--estimated that they entered about 75, 75, and 95 percent of the calls requiring a response, respectively. However, our analysis of the number of calls received and the number of calls recorded in the CA-110 system suggests that these estimates are high.For example, San Francisco estimated that it entered 75 percent, but for a 3-month period in fiscal year 2000, San Francisco received 76,238 calls and entered 14,502, or 19 percent, in the CA-110 system. sampling plan approved by OWCP headquarters when recording information. However, the OWCP Acting Director said that each of these four offices had developed a modified version of the sampling plan and that their plans--while approved by OWCP--were probably not statistically valid. We reviewed the national office’s sampling plan for the data to be entered into these systems and also believe that this plan would not yield statistically valid results even if implemented as designed.
The telephone response rates developed using the CA-110 and other systems indicated that the district offices were generally meeting their timeliness goals.  However, because the performance data were not statistically valid or collected in enough cases, OWCP could not determine whether these goals were being met.
section_title: Progress Measures for Timeliness of Written Correspondence Were Also Invalid
paragraphs: We also found problems in the methods that OWCP used to measure timeliness goals for written correspondence that undermine the usefulness of the data. Each district office was required to take a statistical sample of incoming general, nonpriority correspondence, and then record and track the correspondence to determine whether it was responded to within 30 days. The results of these samples were to be reported to the national office on a quarterly basis. Four of the five district offices we visited each had a different approach for sampling such correspondence, and each stated that its approach was not scientifically developed or developed in a manner that would produce valid results if projected to the universe of all responses. For example, the Dallas district office required each claims examiner to provide his or her supervisor four pieces of written correspondence requiring a response per month to determine whether the response was provided within 30 days. We are concerned about the validity of this approach because, among other things, the potential exists for the claims examiner to give the supervisor only those letters to which the response was timely.
The Seattle district office required claims examiners to log in all general correspondence received every Wednesday that required a response. Supervisors were to review the claimants’ files for these letters after 30 days to determine whether a response had been sent within the 30-day goal. days. Although the performance data showed that this timeliness goal for fiscal year 1999 was generally met by all district offices, the results are probably not statistically reliable.
Conversely, OWCP did seem to have a valid and reliable system for tracking responses to priority mail involving congressional requests. For example, the date of receipt of all congressional correspondence was to be recorded in the Priority Correspondence Tracking System. This system provides reports that track each piece of correspondence until a response is provided.
section_title: OWCP Often Lacked Customer Satisfaction Data to Know How and Where Improvements Were Needed
paragraphs: As I said earlier, the third approach that NPR found model organizations followed was to continuously improve customer service by using performance data—including surveys of important customers and stakeholders-- to identify how and where improvements are needed.  We found that OWCP’s efforts in this area fell well short of the best practices that NPR found in the private sector and in the three agencies we surveyed.
Let me first state the obvious--OWCP did not measure progress toward goals that it did not establish in the first place. OWCP did not have goals— nor did it have related measures—for three basic areas of telephone communications: the percentage of callers able to access the telephone system, the percentage of callers who can reach a customer service representative and their time waiting on line, and the percentage of callers’ inquiries that are resolved during the first call.
Officials from SSA and VBA told us that setting goals for those areas (e.g., telephone access) for their agencies and measuring the results has proven useful in identifying areas where customer satisfaction levels needed improving. For example, data from SSA’s customer satisfaction surveys in 1993 showed that access was the single biggest factor affecting customer satisfaction. According to an SSA official, SSA began collecting access data and found that callers attempting to reach SSA at the busiest times were getting busy signals 50 percent of the time. SSA established a telephone access goal and continued to collect access data, explore new technologies, and acquire additional telephone capacity. By 1996, SSA said, it was able to set and achieve a goal of 95 percent of the callers reaching the system within 5 minutes, and customer satisfaction scores improved accordingly.
My point in using this example is that without goals or measures in these three basic high priority areas, OWCP was not in a position to know what levels of customer service it was providing and where and how telephone services needed to be improved.
For the goals that OWCP did establish, we found that it often did not collect credible performance data from surveys of (1) injured workers, (2) medical providers, or (3) employees that was sufficiently reliable or done in a timely manner to measure progress or to set goals for improving customer service.
subsections: section_title: OWCP Did Little to Gauge Customer Satisfaction Levels
paragraphs: NPR found that model telephone service organizations in the private sector survey (1) their customers frequently to determine how satisfied they are with the services provided and (2) their employees who are answering the phones for job satisfaction levels and ideas to improve their services. In addition, Executive Order 12862, issued September 11, 1993, directs departments and agencies to survey customers to determine the kind and quality of services they want and their level of satisfaction with existing services.
Every organization must decide how frequently it can survey its customers in a cost-effective manner. However, we found that OWCP (1) did not obtain information on injured workers’ satisfaction with services as frequently and by utilizing as many techniques as other model organizations do; (2) did not survey other important stakeholders, such as medical providers and federal agencies; and (3) did not survey the OWCP employees who are answering the phones.
OWCP did do customer satisfaction surveys of injured workers. Since 1996, OWCP has hired a contractor to conduct customer satisfaction surveys about once each year to determine claimants’ perceptions of several aspects of the FECA program, including overall service, the timeliness of responses to telephone inquiries, and the timeliness, thoroughness, and accuracy of written responses to claimants’ inquiries.The claimants are selected on a random sample basis.  During these surveys, injured workers are asked to recall situations that occurred up to 1 year in the past. and to gather requests for new services. Ohio’s BWC surveys approximately 600 injured workers weekly by mail.
The OWCP Acting Director said that OWCP does not conduct a customer satisfaction survey of medical providers because it had difficulty doing so in the past. That is, OWCP officials said they had previously attempted to survey medical providers; however, when they called the representative in the medical provider’s office often could not identify the individual who had previously called OWCP.
Some district office directors and other officials cited outreach programs that, while not necessarily systematic, were an attempt to gain more input from a broader selection of customers. For example, the Seattle district office, in May 2000, initiated a program to begin calling a sample of all telephone callers within the same week they called. The district director said that the office asked the callers whether the response was appropriate and if the caller was satisfied with the representative’s response. OWCP national office officials said that these efforts did not ensure that consistent questions were used or that a random sample of all district office customers was surveyed.
OWCP also has not surveyed its own employees regarding customer service or employee satisfaction within the last 5 years. According to NPR, employee satisfaction is measured as routinely as customer satisfaction in model customer service organizations. NPR cited as benefits of such employee surveys: obtaining information on how to improve the work processes that lead to improved customer service, as well as identifying employee morale issues that could lead to customer service problems. In addition, Executive Order 12862 directed agencies to survey front-line employees on barriers to, and ideas for, matching the best in business. Officials from Ohio’s BWC told us that they have systems in place, such as an annual employee survey that captures information about employee morale.
section_title: OWCP Followed Fewer NPR Best Practices Than the Other Three Agencies We Visited
paragraphs: included related activities, such as establishing a dedicated customer relations team to receive complex customer issues and complaints and recording caller complaints for use in identifying the root causes of problems. While no organization would be expected to apply all of these best practices, we wanted to determine the extent to which OWCP was applying these practices in comparison to SSA, VBA, and Ohio’s BWC. Consequently, we asked these organizations to characterize to what extent—“all,” “some,” or “none”--they followed the activities within each of these practices.
Of the 20 best practices related to measuring performance, OWCP stated that two best practices were not applicable because they applied to call center operations and OWCP does not have any call centers. Of the 18 that OWCP said were applicable, it reported that it performed “all” or “some” of 9 of the 18 practices, or 50 percent. This compares to SSA, VBA, and Ohio’s BWC, who reported that they performed 20, 17, and 20, or 100, 85, and 100 percent, of all 20 NPR best practices, respectively. Of the 9 practices OWCP indicated it did not perform, SSA, VBA, and Ohio’s BWC responded that they applied 8 of them to all or some extent. Examples of these 8 performance measuring practices follow.
Call monitoring: Senior managers regularly listen in on live calls in order to stay in touch with the customer. Team leaders participate in group monitoring sessions to ensure consistency of measurement.
Accessible to customers: A customer feedback loop is built into every phase of the customer service delivery process. It is convenient and easy for customers to contact world-class organizations.
Of the 68 best practices for improving customer service, OWCP said that 6 practices were not applicable because they applied to call centers. Of the 62 practices OWCP said were applicable, OWCP reported that they generally followed to all or some extent 31 of the 62 practices, or 50 percent. This compares to SSA, VBA, and Ohio’s BWC, who reported that they followed 65, 51, and 66, or 96, 75, and 97 percent, of the 68 NPR best practices to all or some extent, respectively. Of the 31 practices OWCP indicated it did not perform, SSA, VBA, and Ohio’s BWC each responded that they applied 18 of them to all or some extent. Examples of these 18 practices follow.
Information queuing: Callers waiting in the queue are provided with information as to the expected length of delay, allowing them to choose whether to stay in queue or hang up.
Resource allocation strategies: Continuous evaluations of key performance indicators help to ensure the appropriate alignment of resource allocations with planning objectives. Well-established benchmarking programs help identify improvement opportunities for cross-functional teams.
section_title: Conclusions
paragraphs: OWCP has been concerned about the level of services that it provides to its customers—injured workers, medical service providers, and agencies.  The annual surveys of injured workers that OWCP has contracted for have surfaced issues, like access to service representatives, that OWCP has taken actions to address.  For instance, the installation of the automated response systems in district offices have given customers an alternative means of getting answers to common questions.  OWCP has also begun to set goals and use data about goal achievement to manage the customer service aspect of district office operations. written communications can be made more exacting, and beginning to reliably measure both customer satisfaction and goal achievement, OWCP can lay the foundation for better serving its customers’ needs for timely and accurate information.
section_title: Recommendations for Executive Action
paragraphs: We recommend that the Secretary of Labor require the Director of OWCP to establish goals for all important areas of OWCP’s telephone and written communications with injured workers and other customers and revise as appropriate existing goals to better ensure that customers’ needs for accurate and timely information are met; collect credible performance data on progress toward these goals, including timely periodic surveys of injured workers’, medical providers’, and agencies’ satisfaction with OWCP’s services and surveys of OWCP employees to gauge their job satisfaction and to gather ideas on how to improve services; and use these performance data and survey results to identify areas needing improvement and to develop strategies for achieving those improvements, including new and revised goals where appropriate.
OWCP’s Acting Director said that he agreed with our recommendations and would continue to explore ways in which to improve customer communications.
That concludes my statement, Mr. Chairman. I would be happy to answer any questions that you or other Members of the Subcommittee may have.
section_title: Contacts and Acknowledgements
paragraphs: For further contacts regarding this testimony, please contact Michael Brostek at (202) 512-9039 or Alan Stapleton at (202) 512-3418. Individuals making key contributions to this testimony included Jeanne Barger, Thomas Davies Jr., James Turkett, Michael Valle, and Cleofas Zapata Jr.
section_title: Scope and Methodology
paragraphs: To determine how OWCP communicates with injured federal workers, agencies who employed these workers, and medical and other service providers who are involved in their treatment, we performed the following audit steps.
We placed 2,400 telephone calls to OWCP’s 12 district offices (200 per office), to assess the accessibility of telephone representatives at each of the 12 OWCP district offices and whether each office’s automated integrated voice response (IVR) system’s data was consistent with the national office’s claim data. We generated listings for our callers which provided the dates, times, and phone numbers to be called, and when applicable, the claim information to be accessed. OWCP provided information about the breakdown of IVR call types for four of the twelve OWCP district offices (Chicago, Philadelphia, Kansas City, and Seattle) during a 2-month period.  We used this information to build a nongeneralizable profile of the distribution of IVR calls across the following six types of claim information for each district office (number of calls per district office in parentheses): claimant calling about a bill payment ( 47), medical provider calling about a bill payment  (8), medical provider calling about a periodic roll payment  (24), claimant calling about a compensation check  (10), claimant calling about physical therapy authorization  (2), and medical provider calling about physical therapy authorization  (9).
For the IVR calls, we were provided with identifiers, such as claim numbers and employer identification numbers, that enabled us to enter the IVR and identify specific transactions on each office’s automated database. We worked with OWCP to identify a time period of earlier transactions from the national database that would still exist in each district office’s IVR system during our test period in June and July 2000.  OWCP sampled records from their national database for this time period until they obtained the required number of records of each type, for each district office. OWCP’s selection of IVR cases did not strictly constitute a random sample of cases for each office from the specified time period, since the cases were selected in case number order, which generally reflected a chronological order, until the required number of cases was obtained. However, since we had no reason to believe that time ordering of the cases was associated with whether a district office’s IVR data would match the information in the national database, we accepted OWCP’s selections for use in our test.
All of our test calls attempted to reach an OWCP representative, either directly or after accessing the IVR system.  We therefore randomly scheduled all our test calls only during the hours that each office told us that a “live” representative should be available to respond to customer inquiries.  Saturdays, Sundays, and Tuesday, July 4, 2000, were excluded from our test days.
For district offices that had the same telephone number for the automated voice response system and for a representative, we called that number 200 times and attempted to reach a representative immediately upon accessing the system on 100 of those calls and attempted to access claim information on the automated system and then access a representative on the other 100 calls. For offices that had separate numbers for the automated voice response system and the representative, we called each number 100 times. As before, upon completion of the automated voice response system, we attempted to reach a representative. All representatives contacted were informed that they had participated in a GAO telephone survey. When attempting to access an OWCP office’s telephone system, we let the telephone ring 15 times (over 1 minute) before making the determination that the telephone system did not answer.  And, in waiting for a representative to answer the line, we waited at least 5 minutes after making the selection to speak to a representative before determining that the call was not answered. We used the 5 minute time period because we wanted to be more conservative than VBA’s goal of a caller’s accessing a representative within 3 minutes and NPR’s guidance and Ohio’s BWC’s goal of 30 seconds. For each call, we recorded how successful the district office was in providing services an injured worker or medical provider might desire. The information recorded included busy signals, no answer after 1 minute, whether we reached an office representative, whether we reached the automated voice response system, and whether consistent information was provided about the claimant by that system, such as the amount of a medical payment or the status of medical authorization for physical therapy. We conducted this telephone survey over a 6-week period in June and July 2000, and attempted the calls throughout the business hours listed for each district office.
If the test calls we made were considered random samples of customers’ telephone experiences during the test period, the following statements could be made about the precision of the estimates:
Estimates of the proportion of calls in each district in which customers were unable to access the telephone systems, and the proportion of calls in each district in which customers were unable to reach an employee within five minutes, have sampling errors of no more than 10 percentage points.
Estimates of the proportion of IVR calls with consistent data (among IVR calls for which transactions could be tested) have sampling errors of no more than 10 percentage points unless otherwise noted in table II.1.
The data we collected, however, were from test calls rather than “actual” customer calls.  Characteristics of the test calls that might affect the outcomes we measured, such as time of day, day of week, or subject matter, might not have mirrored the profile of these characteristics among “actual” customers’ phone calls in any district during the test period. Therefore, the results displayed for individual districts might differ from the ones we might have obtained by sampling “actual” customer calls, by amounts larger than the stated sampling errors.
At OWCP’s headquarters in Washington, D.C., we interviewed knowledgeable officials, reviewed strategic and operational plans for fiscal years 1995 through 2000 to identify goals and measures related to responding to customer inquiries, and obtained communication reports showing data that would indicate OWCP’s performance for the same period.  We also discussed with these officials methods for testing the communications at OWCP district offices, and obtained claimant and medical provider information so that we could perform a telephone survey of the 12 district offices.
We visited 5 of the 12 OWCP district offices—Chicago, Dallas, San Francisco, Seattle, and Washington, D.C. At these five offices, we conducted an in-depth review; we interviewed regional directors, district directors, claims managers, claims examiners, and workers compensation assistants to gain an understanding of the communications at OWCP from various perspectives. We obtained available communication reports, accountability review reports, reports measuring written and telephone communication performance, and other applicable communication data. We selected the five offices based on several factors to obtain a mixture of offices of differing sizes and levels of performance. The primary factors were (1) the number of employees and the number of cases managed at these offices, which ranged from among the lowest to among the highest of all the district offices; (2) OWCP’s telephone and written responsiveness measures, which indicated that some offices had and some had not met national performance goals; and (3) the proximity of two of the offices to our Washington, D.C., and Dallas Regional Office staff.
We surveyed the other seven district offices--Boston, Cleveland, Denver, Jacksonville, Kansas City, New York, and Philadelphia--via a questionnaire and obtained general information about their communication practices, such as the number of employees each assigned to respond to telephone calls.
To compare OWCP’s goals and practices for telephone communication with those of leading organizations, we also surveyed three agencies that have won awards for their telephone communication practices: the Social Security Administration, Department of Veterans Affairs’ Bureau of Benefits Administration, and state of Ohio’s Bureau of Workers’ Compensation (BWC). We asked OWCP and the three agencies to identify which of 95 telephone “best practices” they used that NPR identified in a 1995 study. We did not attempt to verify or validate their responses. In addition, we interviewed and obtained documents on communication performance goals and practices from officials at the three organizations and compared OWCP’s performance goals and measures with those of private sector organizations identified in the NPR study and those of the three organizations we surveyed.
We did our work between January and September 2000 in accordance with generally accepted government auditing standards.
section_title: Access Rates For And Accuracy Of OWCP District Offices’ Automated Interactive Voice Response Systems
paragraphs: Number (percentage) of transactions with consistent data55(93) 48(94) 49(91)48(94) 56(93) 28(88)58(98) 47(90)55(93) 42(100) 561(94)
The district office had separate telephone numbers for accessing the automated interactive voice response system and accessing an OWCP representative. The other district offices had only one main number.
An average for all OWCP district offices cannot be provided, because OWCP could not provide the total number of telephone calls each office received, and thus we could not weight the sample to accurately reflect the impact of each district office’s performance on a national average.
Viewing GAO Reports on the Internet For information on how to access GAO reports on the INTERNET, send e-mail message with “info” in the body to: or visit GAO’s World Wide Web Home Page at: Reporting Fraud, Waste, and Abuse in Federal Programs To contact GAO's Fraud Hotline use: Web site: http://www.gao.gov/fraudnet/fraudnet.htm E-Mail: fraudnet@gao.gov Telephone: 1-800-424-5454 (automated answering system)