In the 2001 tax filing season, IRS received more than 70.7 million calls on its three toll-free assistance numbers and answered over 50.5 million calls—assistors answered 22.7 million calls and automated systems answered 27.8 million calls. As in previous years, IRS had three toll-free telephone numbers that taxpayers could call with questions about tax law, taxpayer accounts, and refunds. Located at 26 call sites, IRS has about 10,000 assistors that help taxpayers with a variety of questions ranging from the applicability of tax laws to the status of their accounts. IRS’ call sites are supervised by 10 field directors, each of whom oversees two to three sites. IRS has four measures to evaluate the extent to which taxpayers are provided with accessible telephone assistance, and four to evaluate the extent to which taxpayers are provided with accurate telephone assistance. (For more information on IRS’ telephone assistance access and accuracy measures see app. I.) IRS’ measures of access are based on actual counts of calls using data collected by IRS’ telephone system. IRS’ measures of the accuracy of assistance, the quality and correct response measures, are estimates based on representative samples of nationwide calls that quality assurance staff monitor and score for accuracy. IRS began collecting data on correct responses in June 2000, so there are no data for the 2000 tax filing season to compare with 2001. Over the years, IRS has studied its telephone performance and made changes designed to improve it. For example, in 1999, IRS extended its hours of service to 24 hours a day, 7 days a week. By providing around-the- clock service, IRS expected to distribute demand more evenly and thus improve taxpayers’ access to service. With the increased use of call- routing technology in 1999, IRS began to manage its telephone operations centrally at the Joint Operations Center in Atlanta. Routing calls to the first available assistor who had the necessary skills to answer the taxpayer’s question was expected to improve taxpayers’ access to service and lessen the disparity in the level of service across sites. However, the level of service declined in 1999, and the quality of service was mixed in the 2000 tax filing season and below IRS’ long-term goal of providing world-class customer service. According to IRS, some of the key factors that affected performance in the 2000 tax filing season were the demand for assistance, staffing levels, assistor productivity, assistor skills, and IRS’ guidance for assistors. As we discussed in a previous report, IRS’ analyses did not cover all key management decisions or other key factors that could have affected telephone performance. Additionally, determining how each factor affected performance was made even more difficult because many of the factors are interrelated; changes in one can affect another. The IRS Commissioner has recognized the complex interrelationships within the telephone-operating environment and has stated that years of sustained effort will be required for IRS to achieve its goal of providing world-class telephone service. To address our objectives, we interviewed IRS officials involved in managing toll-free telephone operations and obtained and analyzed supporting documentation as follows: To assess IRS’ performance in responding to calls on the three main telephone assistance toll-free numbers, we compared the 2001 tax filing season performance for accessibility and accuracy measures with IRS’ performance in the 2000 tax filing season and its 2001 performance targets. To assess IRS’ efforts to determine the factors that affected performance in the 2001 tax filing season, including actions it took to improve performance, we used as criteria GPRA and IRS’ own guidance on analyzing performance data. We interviewed IRS officials in the Wage and Investment and Small Business and Self-Employed Divisions, and the Joint Operations Center. We also analyzed various documents, including reports on IRS’ efforts to determine the factors that affect telephone performance and the results of actions to improve performance. In addition, we used a questionnaire to obtain information from the 10 field directors about their efforts to identify the factors that affected performance and assess the effectiveness of actions taken to improve performance. While we did not independently assess the accuracy of IRS’ performance data, we verified that IRS had procedures in place intended to ensure data reliability. We did our work from February 2001 through October 2001 in accordance with generally accepted government auditing standards. IRS made limited progress in the 2001 tax filing season toward its long- term goal of providing world-class telephone service. When compared with the 2000 tax filing season, access and accuracy performance improved by 2 percentage points or less in three of the six comparable measures. The quality of responses to account inquiries increased 10 percentage points, and there was a 4 percentage point decline in callers who hung up while waiting to speak with an assistor; however, taxpayers waited 15-percent longer to speak with an assistor. When compared with 2001 performance targets, IRS did not meet any of its accessibility goals. These targets were intended to move IRS toward its goal of providing world-class service. Although it met or exceeded the 2001 quality targets, IRS did not meet the current year’s higher targets for providing taxpayers with correct responses. Table 1 compares IRS’ actual 2000 performance levels with its 2001 performance levels and targets. (See app. I for more information on the measures.) According to IRS officials, the access measures are similar to those commonly used by world-class customer service organizations. They are designed to focus efforts on enhancing taxpayers’ experience in getting access to assistance. For example, the “assistor level of service” measureis intended to show IRS’ effectiveness in providing callers with access to an assistor. The “assistor response level” is to measure the percentage of taxpayers that waited 30 seconds or less to speak with an assistor. The “abandon rate” measure is to show the percentage of taxpayers who hang up while waiting to speak with an assistor, while the “average speed of answer” measure is to show the average number of seconds taxpayers wait to speak to an assistor. IRS’ accuracy measures are designed to gauge the taxpayers’ experience in getting accurate assistance. The “quality” measures are to show, for a representative sample of calls, the percentage for which assistors followed all procedures, such as properly identifying themselves at the beginning of calls, doing appropriate research on taxpayers’ accounts, and providing accurate information to taxpayers. The new “correct response rate” measures are intended to show the percentage of calls for which IRS assistors provided correct responses to inquiries without taking into account procedural errors that would not affect the accuracy of the information given the taxpayer. IRS began collecting these data in June 2000, so there were no data for the 2000 filing season to compare with 2001. IRS also measures its performance in answering calls through the use of automation. However, we did not consider this measure—“automated service completion rate”—in assessing IRS’ performance because it assumes that callers who get through to TeleTax are served. The TeleTax system does not have data on how many callers hung up before completing an automated service. Although IRS officials recognize that the measure had limitations, according to them, routing refund status calls to TeleTax allowed IRS to answer about 11.3 million more calls made to its three main toll-free assistance numbers as compared with the 2000 tax filing season. We are continuing to assess many of IRS’ new performance measures, including those used to evaluate telephone assistance. IRS missed some opportunities to better understand the factors that affected performance and to plan evaluations of the actions it took to improve performance. GPRA and IRS guidance outline the benefits of first gathering and then analyzing data to help managers understand the reasons for performance. To this end, IRS collected performance data on access and conducted some analyses. Even so, IRS missed opportunities to do other analyses of the factors affecting performance including actions to improvement it. Contributing to the missed opportunities was a lack of planning for the evaluation of those actions. IRS has a variety of systems in place to make data on the access and accuracy of telephone assistance available to managers. Two of these systems are its Joint Operations Center in Atlanta and Centralized Quality Review System (CQRS) in Philadelphia. Managers at call sites also collect data on factors affecting access and accuracy performance. IRS’ Joint Operations Center in Atlanta manages the activities of the 26 call sites, including monitoring access data and routing calls to the next available assistor anywhere in the country. The Center collects data on various accessibility measures and makes those data available daily to IRS managers through an internal Web site. According to IRS officials, IRS improved the collection of performance data in the 2001 tax filing season. For example, IRS implemented the Enterprise Telephone Database to provide a central call information database. The database was designed to provide IRS analysts and management with the most accurate information for analysis and program decision-making by centralizing data collection and producing a standard set of management reports. IRS’ CQRS staff in Philadelphia are responsible for collecting data on the accuracy of telephone assistance. CQRS provides call-site officials with daily access through its internal Web site on the results of the sample of calls answered by their sites. It also provides weekly and monthly reports on the quality of sites’ responses to taxpayers’ questions about tax law or about their accounts—two of the accuracy performance measures. These data show the call sites what errors assistors are making so site managers can quickly take action to reduce these errors. IRS officials told us that they made better use of the data in the 2001 filing season. They said that Wage and Investment Division and site officials developed strategies to reduce assistor errors based on CQRS reports. IRS call sites collect data on factors affecting access and accuracy in various ways. For example, supervisors use real-time data and historical reports available at the call sites on how assistors spent their time, including average handle time—the time an assistor spends talking with the taxpayer, keeping the taxpayer on hold, and finishing the call and indicating readiness to receive another call. Also, local staff monitor calls to provide more detailed information on what errors assistors are making and in what units the errors are being made.