For many years, the Congress has expressed concerns about the public's telephone access to SSA. Efforts to improve this access have resulted in a dual system of telephone service (a nationwide 800 number and local office service at more than 800 of SSA's field offices) and also led to the current demonstration project. In 1989, SSA established a nationwide toll-free 800 number to replace its local office telephone service. With the implementation of this service, SSA directed its local offices to remove their general inquiry telephone numbers from local phone directories. In their place, the offices listed the new toll-free 800 number. In establishing this toll-free network, SSA intended to provide all of its customers with equal and toll-free access to program services. SSA envisioned that the public would call the 800 number with basic questions about the program, when reporting changes in benefit status, with problems or concerns specific to Social Security records, or to make appointments with local field office staff. The public could continue to contact local office staff when necessary by requesting the unpublished telephone number for any office from SSA's 800 number staff. The establishment of a national toll-free telephone network was planned to facilitate an agencywide 20-percent staff reduction that occurred between 1985 and 1990. By transferring a large workload from its field offices to the 800 number, SSA hoped the downsized offices would be better able to conduct nontelephone business. SSA had start-up problems when the 800 number went on-line nationwide. It had underestimated the volume of calls that would be made to the 800 number and was not able to staff the service adequately, especially when call volumes were heaviest. High busy-signal rates made it difficult for the public to reach SSA, generating complaints to SSA and to the Congress. In response, SSA took several steps to expand its capacity to handle the volume of 800 number calls. These included actions to increase staff devoted to handling calls during the heaviest calling periods, converting additional facilities to 800 number phone centers, and increasing the number of telephone lines devoted to 800 number calls. Even with these actions, busy-signal rates remained high because the number of calls placed to the 800 number continued to grow rapidly. For example, in 1990, callers placed 85 million calls to SSA, and the overall busy-signal rate was 34 percent. In 1994, callers placed almost 117 million calls to the 800 number, and the overall busy-signal rate grew to about 45 percent. During the start-up of the 800 number, these problems concerned the Congress so much that, in 1990, it required SSA to restore telephone access to local offices. As a result, SSA reinstated direct local telephone service to about 830 of its more than 1,300 local offices by publishing their telephone numbers in local directories in addition to the 800 number. However, the Congress did not provide any additional resources for SSA to either purchase telephone equipment or increase staff to handle the reinstated workload. Because it had fewer field office staff due to its downsizing in the late 1980s, SSA chose to implement the local office telephone service with a minimum number of telephone lines and staff. In June 1992, the House Committee on Ways and Means asked us to evaluate the public's ability to access local offices that offered local phone service. In March 1993, we reported that the busy-signal rate at local offices averaged 47.3 percent during the month tested. In October 1993, SSA advised the Congress about its plans to conduct a demonstration project to enhance local office operations and perhaps improve telephone access to its local offices. To improve the public's telephone access to its local offices, SSA is conducting a demonstration project to test telephone equipment known as automated attendant and voice mail. SSA's demonstration project involves 30 of its field offices and three different configurations of the automated attendant and voice mail equipment (referred to as methods A, B, and C in this report). SSA wanted local offices from each of its 10 regions involved in the project, and it allowed the regions to select these offices on the basis of the type of telephone equipment they were already using and their willingness to participate in the project. Each method being tested in the demonstration project represents a different configuration of equipment. In method A offices, SSA added automated attendant and question-and-answer mail boxes to its general inquiry lines. In addition, it added voice mail to staff member extensions. A caller to method A offices hears a recorded greeting that identifies the agency, office hours, and address. This basic information answers caller questions in many cases. Callers seeking other types of assistance have other options: Callers may press the extension number of a particular employee with whom they may be working on a claim or other matter. If not already working with an SSA representative, callers may also select an automated service menu for routine matters such as reporting changes in address, making an appointment to file for benefits, or requesting an original or duplicate Social Security card. These services are provided without direct staff intervention through the use of question-and-answer voice mail messages. Finally, if callers wish to speak to an SSA representative, they can choose to hold the line until one becomes available. Method B offices operate the same way as method A offices except that one additional feature is available. Method B offices have an additional general inquiry telephone line to play a message that advises callers that all available lines are busy. This message also states that the caller should either call at a later time or may call SSA's toll-free 800 number. Callers are only connected to this line when all the other general inquiry lines are already in use. For the demonstration, method C offices do not have any additional telephone lines, automated attendant, or the related question-and-answer mailboxes on their general inquiry lines. They have only voice mail capability at the desks of staff members. The underlying objective of the demonstration project is to improve the public's access by making more telephone lines available to handle phone calls at local offices. The demonstration project equipment configurations have also extended service hours for method A and B offices because, with automated attendant, after-hours calls can be answered and callers can leave voice mail messages. Most method A and B offices received additional general inquiry telephone lines when SSA installed the new equipment in their offices. Local managers in some participating offices, however, did not want additional lines because they believed that they could not handle additional telephone calls without increased staffing. Table 1 shows each method A and B office and the number of general inquiry lines each had before and after equipment was installed for the demonstration project. As shown, five method A and eight method B offices received at least one additional general inquiry line. We found statistically significant improvement in access under method B, while method A showed no statistically significant change in access. Under method B, busy-signal rates dropped greatly, but more calls were being placed on hold. Because method C did not involve any change to the general inquiry lines, we did not consider its effect on access to the local lines. When examining how telephone access changed by the individual offices in the demonstration, we found mixed results among both methods A and B. We also found that SSA staff in the demonstration offices strongly believe that the voice mail equipment on their desk phones enhanced efficiency and public service. To measure changes in access for evaluation purposes, we grouped the call outcomes into two categories: access and no access. We considered access to consist of two call outcomes: calls in which we spoke to an SSA employee without spending any time on hold and calls in which we were on hold for less than 2 minutes before speaking to an SSA employee. We considered no access to consist of five different call outcomes: busy signals, no answer after the phone rang 10 times, recorded messages directing us to call at a later time, calls that were disconnected before we had a chance to speak with an SSA representative, and all calls in which we were placed on hold for more than 2 minutes. We selected 2 minutes as the time we would wait on hold before hanging up because we thought it was a reasonable expectation. In addition, our definition is consistent with information SSA obtained from a survey of its clients. In July 1994, SSA reported that 90 percent of the respondents who used the 800 number said that being on hold for no more than 2.3 minutes would be good service. Table 2 compares how telephone access changed with the installation of new equipment at method A and B offices. It shows that method B offices had an improvement of 23 percentage points in the calls reaching SSA and that this change was large enough to be statistically significant. The method A configuration did not produce a statistically significant change in access under our test. Examining the results of our analysis by call outcomes provides a better understanding of the changes occurring under the demonstration project. As shown in table 3, the installation of the new equipment and additional telephone lines has resulted in a large drop in busy signals. After installation, busy signals dropped at method B offices by 55.2 percentage points. The large increase in the number of callers receiving the "call later message" after installation of the new equipment probably accounts, in part, for the drop in busy-signal rates. The other substantial change shown in table 3 relates to calls placed on hold. The table shows two categories for calls placed on hold: on hold less than 2 minutes and on hold more than 2 minutes. The percent of calls in both of these categories increased greatly under the demonstration. With newer equipment, more telephone lines, and a constant level of staff assigned to answer these calls, the additional calls reaching SSA are being placed on hold until staff can answer them. Examining how access changed at each office within methods A and B showed mixed results. For example, tables 4 and 5 show that 3 of the 10 method A offices and 4 of the 9 method B offices showed statistically significant improvement in access. However, five of the method A offices and the five remaining method B offices showed no significant change in access. Furthermore, two method A offices also showed statistically significant declines in telephone access rates. Local factors such as call volumes, the number of telephone lines available, and staffing issues may account for the wide variation in access rates at the office level. We recognize that a caller placed on hold (rather than receiving a busy signal) can be considered successful access to SSA. In fact, SSA considers access to its 800 number successful when a caller is connected to SSA regardless of whether the caller has spoken with a representative, heard a recorded message, spent a long period of time on hold, or hung up while on hold. Analyzing our data using this broader interpretation of access, we found that statistically significant improvement occurred under both methods A and B. These results are shown in table 6. Using this definition of access, on an office level, we noted additional improvements. Among method A offices, significant improvement in access occurred in 1 more office--4 of the 10 offices instead of 3 of the 10 offices improved. Among method B offices, significant improvement in access occurred in three additional offices--seven of the nine method B offices improved instead of four of the nine offices. Staff at all demonstration offices had voice mail installed on their desk telephones. We visited 12 of the 30 demonstration offices and met with office managers and staff using the new equipment. Overall, we heard almost universal praise about how the voice mail feature improved office operations and enhanced customer service. All 12 of the office managers we interviewed were enthusiastic about the new equipment's voice mail feature. Seven of the 12 managers told us that the voice mail equipment increased their claims representatives' efficiency. Other managers told us that the voice mail equipment added flexibility to their offices and improved customer service. Finally, all of these managers told us that feedback they have received from the public about the new voice mail equipment has been positive. We also interviewed 71 staff members who use the voice mail equipment. Most of these staff members told us that the new equipment has improved service to the public by making it easier to reach SSA. They said that when a caller tries to reach a specific SSA representative who is not at his or her desk, the caller can leave a message on the staff person's voice mail. Furthermore, many of the staff members we interviewed told us that voice mail has enabled them to manage their workload better and has increased their productivity. Some of these staff also told us that they no longer worry about losing messages or receiving inaccurately recorded messages. Others said that with voice mail, callers can leave messages and information needed for processing a claim. This eliminates the need for repeated calls between SSA and the public, speeding up the claims process. Two separate SSA organizational entities are evaluating the telephone demonstration project. SSA's Office of Workforce Analysis (OWA) is evaluating the equipment's effect on office productivity and employee reactions. The Office of Program Integrity Reviews (OPIR) is evaluating public reaction to the equipment. Neither SSA study had been finished as of early February 1996. OWA's study has two basic objectives, determining the equipment's effect on productivity levels and identifying employee experiences and reactions to using the equipment. To measure the new equipment's effect on productivity, OWA planned to gather and compare certain data. For example, OWA planned to examine how busy-signal rates and call volumes have changed using data obtained from the telephone companies servicing the demonstration offices. OWA also planned to measure the amount of work generated by callers using the automated services option (reporting address changes or missing checks). It has directed local offices to prepare weekly reports on the number of callers using these services. To examine employee reactions, OWA has planned to have field office managers and staff who answer the telephones fill out a short questionnaire. The questionnaire is soliciting information about how well the system has performed and respondents' views on ease of use and training adequacy. To obtain information about the public's reaction to the new equipment, OPIR planned to install caller ID equipment at 19 of the 30 demonstration offices. Offices with caller ID are to record the phone numbers on certain dates. OPIR prepared several different questionnaires for its staff to use when contacting callers. OPIR planned to contact 1,500 callers, 500 for each equipment configuration but has encountered complications. Its report is to be finished in February 1996. Overall, the addition of new equipment and telephone lines has demonstrated that access to SSA offices can be improved. Even if SSA does not devote additional staff to answering telephones in local offices, this technology may help improve the efficiency and effectiveness of the agency's service to the public. To fully evaluate whether to install the demonstration phone equipment in other locations, however, an important consideration for SSA will be the public's and SSA employees' views along with the equipment's relative costs and contributions to meeting SSA's public service goals. SSA commented on a draft of this report in a letter dated January 29, 1996 (see app. II). SSA agreed with our findings that enhanced technology has increased the public's telephone access to field offices. It also agreed with our view that a full evaluation of productivity issues and employee acceptance of and public reaction to the new equipment is needed before installation of this equipment across the board. SSA noted that its internal studies on these issues will be completed by the end of February 1996. Copies of this report are being sent today to SSA and parties interested in Social Security matters. Copies will be made available to others upon request. If you have any further questions, please contact me on (202) 512-7215. GAO contacts and staff who prepared this report are listed in appendix III. The objective of our review was to determine if the installation of the new telephone equipment has improved the public's access to the participating offices in SSA's demonstration project. To do this, we placed phone calls to offices before and after installation of the new equipment being tested and recorded outcomes of these calls (busy signal, placed on hold, and the like). From these outcomes, we then calculated access rates. As noted earlier in this report, SSA installed two types of new equipment at 30 field offices: automated attendant and voice mail. The equipment was installed in three different configurations. We labeled these configurations methods A, B, and C. SSA designated 10 offices to test each method. Table I.1 shows these office locations. Table I.1: SSA Offices Participating in the Demonstration Project by Method Attleboro, Mass. Bangor, Me. Albany, N.Y. Geneva, N.Y. Petersburg, Va. Reading, Penn. Asheville, N.C. Charleston, S.C. West Indianapolis, Ind. Cedartown, Ga. El Dorado, Ark. Champaign, Ill. Norfolk, Neb. Oklahoma City, Okla. Roswell, N.M. Stockton, Cal. Winfield, Kans. Pocatello, Ida. Las Vegas, Nev. We conducted the preinstallation phase of the test from mid-January through the end of February 1995, placing our calls on what we believed to be the 8 busiest days during that period. We reasoned that the best way to measure changes in phone access was to test performance on the busiest calling days rather than on average calling days. To identify the busiest calling days, we used information on telephone call volume to the 800 number during the same period in 1994. SSA has information that tracks the busy-signal rate for the 800 number. Using these data, we identified the 8 busiest days from mid-January through the end of February in 1994. We chose this period because SSA began installation of the new equipment at the 30 offices during the last week of February 1995. The busiest days tended to be Mondays, Fridays, the third of the month (when Social Security checks are normally delivered), and the day after a holiday. The exact days we chose for study were January 17 and 30 and February 1, 3, 6, 7, 21, and 27. SSA had planned to complete installation of the phone service by June 1995. However, it encountered several installation problems. By late July, only one office did not yet have the equipment installed. We decided to give the field offices some time to become acquainted with the equipment. By using the 1994 call log for SSA's 800 number, we selected the following 8 days on which to conduct the postinstallation phase calls: August 22, 29, and 30 and September 5, 6, 8, 11, and 13. We designed the test using statistical sampling principles so that calls would be randomly distributed throughout the day and across the 30 SSA offices during each of the two 8-day test periods. To provide an adequate level of precision for our estimates of the busy-signal rates, we made 350 preinstallation calls and 350 postinstallation calls for each of the three methods being tested. To determine the time of the calls, we divided the workday into 28 15-minute segments (beginning at 9 a.m. and ending at 4 p.m.). This created 224 time periods over the 8-day test period (8 days times 28 time periods per day). Since 10 locations could be called during each of the 224 time periods, we had a total of 2,240 possible time/location combinations, with each representing a possible telephone call. We numbered these combinations 1 through 2,240, with number 1 assigned to the combination of the first location and the first time period (9:00 to 9:15 a.m.) of the first of the 8 days, and number 2,240 assigned to the combination of the tenth location and the last time period (3:45 to 4:00 p.m.) on the eighth day. We then picked at random 350 of the numbers from 1 to 2,240. For each number picked, we looked up the corresponding time/location combination that had been assigned that number and placed a telephone call at that time to that location. For example, one of the random numbers we picked was 572. We had assigned that number to location number 2 during the 9:15 to 9:30 a.m. period on the third day. As shown in table I.1, location number 2 for method A is the Flatbush office. Therefore, we placed a call to Flatbush during the 9:15 to 9:30 a.m. period on the third day. We also placed calls during the same period on the same day to the Albany and Geneva offices, locations number 2 under methods B and C. For the postinstallation period, we placed an identical set of calls, in time and location, to those placed to estimate the busy-signal rates before installation of the new equipment. For example, since we had picked the number 572 we again placed calls to the Flatbush, Albany, and Geneva offices during the 9:15 to 9:30 a.m. period on the third day of our postinstallation test. We used the same set of 350 random numbers for both our pre- and postinstallation tests of the equipment to make our comparisons of changes in the three methods' access rates as fair as possible. By placing the preinstallation test calls on the same days and at the same times to each of the three groups of 10 locations, we hoped to minimize the effect on our estimates of variation among locations in the volume of calls received on particular days or during particular hours. Similarly, by placing our postinstallation test calls to the same locations and at the same times as those of our preinstallation test calls, we attempted to minimize the effect of variation among locations in the general call volume between the mid-January through February period and the period of our postinstallation test in August and September. Several events arose during our analysis that necessitated adjusting the data for study purposes. Table I.2 summarizes these adjustments. Table I.2: Adjustments to SSA Offices Participating in the Demonstration Project by Method Attleboro, Mass. Bangor, Me. Albany, N.Y. Geneva, N.Y. Petersburg, Va. Reading, Penn. Asheville, N.C. Charleston, W.Va. West Indianapolis, Ind. Cedartown, Ga. El Dorado, Ark. Champaign, Ill. Norfolk, Neb. Oklahoma City, Okla. Roswell, N.M. Stockton, Cal. Winfield, Kans. Pocatello, Ida. Las Vegas, Nev. Affected demonstration office. Due to unforeseen events, we could not complete our comparison exactly as planned. Some of the field offices had to be dropped from the study or moved to another method. SSA did not install new equipment in the St. Paul or Bangor field offices as had been planned. Therefore, we excluded St. Paul and Bangor from our study. We also discovered that the phone number we had used in the first phase of the study for the Murray field office was incorrect so we excluded this office from our analysis. Finally, the Las Vegas field office, which was to receive equipment for method C, instead received the equipment for method A. These adjustments resulted in 10 field offices using method A, 9 field offices using method B, and 8 field offices using method C in our analyses. For each method, we estimated the proportion of times that the public would have accessed SSA when calling the offices in the test during the 8 days on which we placed calls. Because our estimates--which apply only to the 8 days on which we placed calls--are based on a limited number of phone calls, each estimate has an associated sampling error. At the 95-percent confidence level, sampling errors for our estimates of access rates under each method (both pre- and postinstallation) are about 5 percentage points. Sampling errors for our estimates of changes in access rates under each method are about 7 percentage points. In many instances, sampling errors for estimates of access rates at individual offices are substantially higher. We designed a simple computer-assisted telephone interview to collect the data on the outcome of each telephone call attempt. The information collected included whether (1) we got a busy signal, (2) the phone rang without being answered (we hung up after 10 rings), (3) a person answered, (4) we were placed on hold (we waited 2 minutes before hanging up), and (5) we were disconnected. In addition to those named above, the following individuals made important contributions to this report: Jim Wright and Jay Smale developed our sample design and the computer-assisted interview instrument used to record telephone call outcomes; Inez Azcona, and Jeffrey Bernstein collected the data, visited local SSA offices and helped prepare this report; Wayne Turowski and Steve Machlin did the computer programming and analysis of data. The first copy of each GAO report and testimony is free. Additional copies are $2 each. Orders should be sent to the following address, accompanied by a check or money order made out to the Superintendent of Documents, when necessary. VISA and MasterCard credit cards are accepted, also. Orders for 100 or more copies to be mailed to a single address are discounted 25 percent. U.S. General Accounting Office P.O. Box 6015 Gaithersburg, MD 20884-6015 Room 1100 700 4th St. NW (corner of 4th and G Sts. NW) U.S. General Accounting Office Washington, DC Orders may also be placed by calling (202) 512-6000 or by using fax number (301) 258-4066, or TDD (301) 413-0006. Each day, GAO issues a list of newly available reports and testimony. To receive facsimile copies of the daily list or any list from the past 30 days, please call (202) 512-6000 using a touchtone phone. A recorded menu will provide information on how to obtain these lists.

Summary:
Pursuant to a legislative requirement, GAO reviewed the Social Security Administration's (SSA) telephone access demonstration project. GAO found that: (1) under one of the two telephone demonstration project configurations tested, telephone access improved significantly, busy-signal rates dropped, and the number of callers reaching SSA improved; (2) although improvements were statistically significant under a broader access definition, more callers were placed on hold because staffing had not increased; (3) configuration results varied among SSA offices; (4) SSA field office staff believe that the installation of voice mail equipment has improved office efficiency and public service; (5) SSA expects to complete its internal evaluations of the project's effect on local operations, employees, and public relations by the end of February 1996; and (6) the equipment tested in the demonstration project has the potential to further SSA public service goals, but SSA must assess the costs and contributions of the equipment before installing it systemwide.