As we and others have reported, taxpayers often have problems obtaining the needed information from IRS to file their tax returns and resolve problems with their accounts. Not only do taxpayers have difficulty in reaching IRS by telephone, but once a taxpayer reaches a CSR, that CSR does not always have easy access to the information needed to resolve the taxpayer’s problems. One of TSM’s major goals is quick and easy access to the data needed by CSRs and other employees to provide better customer service and improve voluntary compliance. Several systems are being developed or are planned to address IRS’ critical data needs. IRS considers ICP to be one of the most important of these undertakings. Information on taxpayers and their accounts are contained in a variety of IRS databases. Until 1995, information on IRS’ primary database for taxpayer account, which is used for assisting taxpayers—known as the Integrated Data Retrieval System (IDRS)—was stored at the service center where the taxpayers filed their returns and could be accessed by employees at the service center, connected district offices, and customer service sites. If the taxpayers called a service center other than that at which their returns were filed, the CSR would be unable to answer questions about their accounts. Either the taxpayers were told to call a different service center or the questions would be written down and referred to the appropriate service center for resolution. Early in 1995, IRS implemented a networking capability among the service centers, district offices, and customer service sites, so that employees could have access to IDRS data nationwide. This networking capability is referred to as Universal IDRS; however, this is only a partial solution to IRS’ data accessibility problems. Although Universal IDRS gives IRS employees access to taxpayer account information nationwide, it does not always provide complete information on a taxpayer’s account. Other information needed to help the taxpayer may be contained in different systems that are not linked to IDRS. Generally, the CSR must access each of the different systems independently. For example, an IRS employee using IDRS will know that a taxpayer was sent a notice of underreported income but would not have access to the actual notice. That notice is contained in IRS’ Automated Underreporter System (AUR). AUR would provide additional information, such as the amount of unreported income and information from the tax return that may indicate, for example, the amount of dividend or interest reported by financial institutions but not by the taxpayer. To obtain these data, the IRS employee must be able to access the AUR database using a different computer terminal. However, the employee may not have access capability. As a result, the employee would have to either (1) refer the taxpayer to another office, (2) research the problem and return the taxpayer’s call, or (3) tell the taxpayer to call back later. With ICP, IRS envisions that customer service staff would have all relevant information from a number of important databases available to them to assist the taxpayer. IRS plans to use ICP to integrate and obtain access to information from each of the existing IRS functional databases that contain taxpayer information. The primary databases include IDRS, AUR, Corporate Files On Line (CFOL), and the Automated Collection System (ACS). ICP was intended to resolve the data accessibility problems by integrating the information from various databases used by CSRs and providing a single computer terminal to do the task. Using a taxpayer’s Social Security number to obtain case information, the ICP software is expected to automatically assemble the relevant information on a computer terminal, provide questions and prompts for CSRs, and perform calculations for updating the account. ICP, as originally envisioned, was expected to support both IRS’ customer service vision and district office compliance operations. It was to be developed and implemented in stages using a multirelease approach. Each release was to build upon the previous release, providing a related set of software, hardware, and telecommunication tools that were to provide incremental improvements in customer service. The first series of ICP releases, commonly referred to as releases 1.0/1.5, 2.0, and 2.5, were intended to meet the needs of IRS’ customer service employees. Later releases are expected to support district office compliance operations, but they have been delayed until sometime after 2000 due to IRS’ recent rescoping of the TSM program. The first release, 1.0/1.5, primarily provided computer hardware and software that eliminated the need for CSRs to use multiple workstations to access data on various databases. It was designed to allow CSRs to use one computer terminal to access the various databases that contain information on taxpayers’ accounts. For example, using an ICP workstation, CSRs could access information stored on IRS’ three major databases—IDRS, ACS, and AUR as well as some smaller databases. It also provided some features that made the existing systems easier to use, such as a summary screen of taxpayer information, menus to look up command codes, and automated forms ordering. The next ICP software release, 2.0, is designed to provide CSRs with a single view of taxpayer data. It is expected to eliminate the need for a CSR to access the separate databases. Instead, information is to be assembled from various databases onto a standard screen. Release 2.0 is expected to also provide CSRs with new tools to enhance their ability to offer taxpayers one-stop service. It is also expected to provide a call-routing feature that would route taxpayers’ calls to the next available representative who would be most skilled at addressing the taxpayer’s question or issue. Some of the tools expected from ICP 2.0 include on-line display of and adjustments to Form 1040 returns and associated schedules including automated tax, interest, and penalty computation; automated installment agreement preparation; automated payment tracer capability; automated refund inquiries; data directed routing; and enhanced history generation. Additionally, ICP 2.0 is expected to eliminate the need for CSRs to remember numerous command codes, which are needed to access and update taxpayer account information. For example, both ACS and IDRS have their own language of command codes, requiring significant training and adequate time to learn. IDRS alone has many codes, requiring two large handbooks of explanation. Not surprisingly, few IRS employees have mastered both systems. ICP 2.0 would eliminate the need for CSRs to know any ACS command codes and most of the IDRS command codes. IRS expects ICP 2.0 to provide improved service to taxpayers. Currently, to answer taxpayers’ questions about whether payments have been properly credited to their accounts, CSRs must access up to five separate databases, searching for payment transaction codes or payment offset codes. This procedure is known as a “payment tracer.” CSRs must then locate the missing payment and manually prepare a credit transfer to move the payment to the proper account. Often the CSR is unable to complete the search while the taxpayer is on the telephone. With ICP, CSRs are expected to complete the payment tracer and resolve the taxpayer’s question while the taxpayer is still on the telephone. Instead of entering five separate search commands, CSRs would simply input the amount of the payment. ICP would automatically search the databases and provide CSRs with the information needed to determine whether the taxpayer’s account had been properly credited for the payment. ICP would also provide CSRs with an easier way to transfer payments between accounts. Release 2.5 is expected to provide CSRs this same level of access to information for business taxpayers. Over the past decade, we have issued several reports and testified before congressional committees on IRS’ costs and difficulties in modernizing its information systems. From 1986 through fiscal year 1995, IRS estimated that it had invested about $2.5 billion in TSM. IRS projects to spend over $8 billion on TSM. By any measure, this is an enormous information systems development effort, much larger than most other organizations have ever undertaken. In September 1993, IRS assessed its software development capability using Carnegie Mellon University’s Software Engineering Institute’s (SEI) Capability Maturity Model (CMM). This model is the generally accepted standard in both industry and government for assessing an organization’s ability to develop software in accordance with modern software engineering methods. This tool focuses on the maturity of certain software development processes called “key process areas (KPA).” The five KPAs are: requirements management, software project planning, software project tracking and oversight, software quality assurance, and software configuration management. The model ranks organizations on a scale of 1 to 5. IRS’ self-assessment placed its software development capability at the lowest level, CMM level 1, because its assessment showed significant weaknesses in all KPAs prescribed for an organization to reach a level 2 capability. Each of the CMM levels are described in appendix I. In February 1995, TSM was added to our list of high-risk areas as a critical information systems project that is vulnerable to schedule delays, cost over-runs, and potential failure to meet mission goals. In July 1995, we issued a comprehensive report on the effectiveness of IRS’ efforts to modernize tax processing. The report discussed pervasive management and technical weaknesses that must be corrected if TSM is to succeed and made over a dozen specific recommendations. In this regard, we reported that unless IRS improved its software development ability, it is unlikely to build TSM in a timely or economically manner, and systems are unlikely to perform as intended. Reflecting continued congressional concern with TSM, the Treasury, Postal Service, and General Government Appropriations Act of 1996, required the Secretary of the Treasury to provide a report to the House and Senate Appropriations Committees regarding the management and implementation of TSM. This report was provided to the Committees in May 1996. As directed by the same legislation that required the report, in June 1996, we reported on our assessment of IRS actions taken to correct its management and technical weaknesses. We found that while IRS had taken some actions, none responded to any of our recommendations in total. As a result, IRS was not in any appreciably better position to ensure Congress that the money spent on TSM would deliver the promised capability, on time, and within budget. Because IRS had not made adequate progress to correct its weaknesses, we suggested that Congress should consider limiting TSM spending to only cost-effective modernization efforts that (1) support ongoing operations and maintenance; (2) correct IRS’ pervasive management and technical weaknesses; (3) are small, represent low technical risk, and can be delivered in a relatively short time frame; and (4) involve deploying already developed systems—only if these systems have been fully tested, are not premature given the lack of a completed architecture, and produce a proven, verifiable business value. Our objectives were to (1) evaluate IRS’ assessment of ICP costs and benefits and obtain users’ perceptions on the system’s benefits, (2) analyze IRS’ testing of ICP, (3) assess IRS’ ongoing efforts to redesign its customer-service work processes to fully utilize ICP capabilities, and (4) assess IRS’ software development processes being used for ICP. To evaluate IRS’ assessment of ICP costs and benefits, we reviewed two IRS studies that were developed to assess the expected costs and benefits of ICP. The first document, known as the Unified Business Case, was developed by IRS in January 1995. During our review, IRS conducted a second analysis of ICP costs and benefits. This ICP Business Case was issued in July 1996. We reviewed both documents for completeness and compared them against IRS’ criteria for business cases, as detailed in its Business Case Handbook. To obtain user views on ICP benefits, we randomly selected and conducted structured interviews with 193 CSRs, 37 customer service managers and 11 system administrators at the Nashville, Cincinnati, and Atlanta customer service sites. We chose these three sites because (1) Nashville was the prototype site for testing ICP and new work processes and (2) Atlanta and Cincinnati were two of the initial sites to receive ICP. To analyze IRS’ testing of ICP, we reviewed the results of the initial pilot test of ICP version 1.5. We met with IRS officials at the National Office, the ICP program office, and the Customer Service Site Executive’s Office to discuss the limitations of the test that IRS identified. We also discussed with IRS officials their plans for a more thorough test of the next ICP version, 2.0, including visiting the Integrated Test and Control Center facility where ICP 2.0 was being tested. We were unable to review specific plans for the pilot test because they had not been completed during our audit work. To assess IRS’ ongoing efforts to redesign its customer service work processes to fully utilize ICP capabilities, we met with IRS officials in charge of efforts to develop new work processes for CSRs. We reviewed documents, such as the Customer Service Work System Design document, that discussed the results of the initial efforts to broaden the scope of telephone assistors’ work. We also reviewed draft reports on the results of recent studies that make further recommendations for redesigning work processes. To assess IRS’s software development processes used to develop ICP 2.0, our fourth objective, an SEI-trained team of GAO specialists used SEI’s Software Capability evaluation (SCE) method. The details of our scope and methodology for this objective are discussed in appendix I. We conducted our work from August 1995 through August 1996 in accordance with generally accepted government auditing standards. We requested comments on a draft of this report from the Commissioner of IRS or her designee. On November 21, 1996, IRS officials, including the Customer Service Site Executive and the National Director, Customer Service Planning and Systems Division, provided us with oral comments. These comments were supplemented by a memorandum from the National Director, Customer Service Planning and Systems Division, and the Deputy Chief Information Officer (Systems Development) on November 26, 1996. Their comments are summarized on pages 22-24 and incorporated elsewhere in the report where appropriate. Through fiscal year 1995, IRS had invested over $150 million in ICP and, according to data provided to us after a May 6, 1996, Treasury report to the House and Senate Appropriations Committees, IRS had plans to invest about $77 million and $112 million in fiscal years 1996 and 1997. That would bring the total investment to about $340 million, or about 53 percent of the $641.1 million budgeted for ICP through 2000. However budget cuts have caused IRS to reduce planned expenditures for ICP and to reassess how to move forward to meet the needs of front-line assistors. Despite this sizable investment, ICP costs and benefits remain uncertain because the scheduled rollout of ICP and its capabilities continue to change. Since ICP began in 1993, the milestone dates for tasks have slipped, and most recently the testing of software release 2.0 has been delayed at least 3 months. Also, the capabilities of software release 2.0 may be less than originally planned. Finally, the original business case on ICP was never accepted. While a more recent business case indicates that IRS will update projections for cost and benefits as necessary, IRS has made no revisions to the business case, even though changes are expected to the rollout date and to the software capabilities for release 2.0. IRS is reassessing its plans for release 2.0 and plans to revise its business case after a proposal is made to and approved by the Investment Review Board. ICP began in late 1993, and the capability of ICP was to be rolled out incrementally in four phases and was to be completed by 1997. In March 1995, changes in the scheduled rollout date took place. The revised date for ICP being operational was extended to November 1998. As of June 1996, the first increment of ICP was partially deployed at 14 of the 23 customer service centers. There were about 2,500 ICP workstations operating at these sites. IRS was expecting to purchase additional workstations in 1996 and 1997. The latest IRS schedule calls for ICP to be fully deployed by fiscal year 2000, but this may be delayed. For example, pilot testing of release 2.0 was scheduled to begin on September 30, 1996, with initial deployment in April 1997. However, the development team has been unable to deliver the software as scheduled. As a result, the pilot test was delayed, and a risk assessment of the entire ICP project was initiated. The contractor’s interim report on the risk assessment states that the pilot test on ICP release 2.0 should be delayed at least 3 months. The testing and deployment of release 2.0 may be delayed longer than 3 months, because the contractor stated that the number of problems identified during software testing continue to increase and are “not likely to be fixed in near term.” IRS does not know the impact on costs of these delays, but it seems these delays, especially any long delay with release 2.0, will likely increase costs. IRS has spent about $150 million to date for ICP 1.0 /1.5 and to develop ICP 2.0, but IRS officials told us that they never projected any revenue or productivity gain for the early releases of ICP. IRS officials said that ICP activities to date have provided the foundation for development of ICP 2.0 and have put in place the hardware, telecommunications, and other infrastructure components required to implement the customer service vision; and they noted that the real benefit gains of ICP will come from ICP release 2.0. In 1995, IRS’ Information Systems Division developed a “Unified Business Case” for the systems supporting IRS’ customer service and district office operations. The costs and benefits were projected to be $3.2 billion and $5.2 billion, respectively. IRS customer service officials said that this cost and benefit analysis was never accepted by their office because, by the time the analysis was completed, the projects being evaluated were not consistent with their new business vision and no longer represented the scope of ICP. In July 1996, IRS completed another business case for Customer Service/ICP. ICP costs and benefits were estimated to be $774 million and $2.9 billion, respectively. This business case was intended to justify the costs of ICP, including the necessary physical infrastructure, such as real estate, telecommunications, computer equipment, and furniture. Most of the users we interviewed said that ICP 1.5 had provided some advantages. At the time of our review, however, IRS had not taken steps to measure the extent to which ICP has improved service to taxpayers. More than 91 percent of the employees that responded to this question said ICP improved their ability to serve taxpayers at least to some extent, when compared with what they used before development of ICP. About 89 percent of those who responded told us that ICP increased their productivity while 85 percent said it increased their ability to resolve the taxpayers’ questions on the initial contact at least to some extent. While the results of our survey of CSRs were generally positive, IRS had not attempted to measure the extent to which ICP had affected the services provided to taxpayers. Appendix II shows CSRs’ opinions on the extent to which ICP release 1.5 has allowed them to improve customer service and improved their ability to do their jobs. The testing of ICP 1.5 was too limited and did not measure ICP’s impact on business operations. Also, IRS discounted system downtime when analyzing the results of the test. IRS officials recognized the limitations of the ICP 1.5 testing and told us that testing of ICP 2.0 would be more comprehensive. IRS conducted its test of ICP 1.5 at the Nashville customer service site during a 4-week period in July and August 1995. Nashville, IRS’ prototype customer service site, had been using ICP for approximately 9 months before the test. The test was done during a nonpeak period, when IRS is not typically as busy as during the tax season months of January through April. Testing during a nonpeak period may not stress the system’s capacity. IRS officials said that testing was limited because ICP 1.5 was only intended to provide the data access foundation for developing ICP 2.0 and to put in place the hardware, telecommunications, and other infrastructure components required to implement the customer service vision. The National Research Council also reported that the ICP test was too limited to “yield the analytical results needed to appraise ICP in a full site-production mode.” The Council’s report also states that ICP “was tested during only one tax season, on a limited basis, before being deployed to other sites.” The purpose of a pilot test is to evaluate the performance of a system in one location before deciding whether to implement the system at other locations. IRS uses the pilot test to certify that the system is meeting its program or business objectives. IRS refers to this process as the “Business Certification.” During the pilot test, IRS was to collect data on the performance of the system and compare the data against established performance goals to certify that the system is performing as expected. To measure ICP’s impact on business operations, IRS examined six quality indicators—productivity, accuracy, timeliness, revenues, initial contact resolution, and customer satisfaction. IRS had difficulty measuring four of these six indicators, and its measure of the remaining two indicators was very limited in scope. Additionally, IRS based its measure of another indicator—quality of the workplace—on focus group discussions. Despite difficulties in measuring the impact on business, IRS officials decided to roll out the system to other sites because comments on the quality of the system from the workplace focus groups had been generally favorable. IRS discounted the results of its testing of accuracy and revenues collected because it could not isolate the impact of ICP from that of other changes in work processes. For example, during the certification test period, accuracy varied from 85 percent for questions on tax law and procedures to 36 percent for account questions. The national standard is 87 percent. The evaluation team concluded that the results of the accuracy and revenue tests were not comparable to national results because Nashville was “blending” certain collection and taxpayer service work and cross-training its employees to work both areas. IRS used very narrow measures to gauge the system’s effect on timeliness and productivity at the Nashville site. Timeliness and productivity measurements were limited to measuring gains made from a more timely process of ordering forms. According to the test results, ICP reduced the amount of time it took to order forms by 1 day and saved $118.68 per day, compared with fiscal year 1994 costs for direct labor and mail. However, ordering forms is only a small part of customer service. IRS did not measure the timeliness of handling taxpayers’ calls for other services, such as refund inquiries or the productivity of CSRs—concerning the number of calls they were able to answer. IRS had no baseline measures for customer satisfaction and initial contact resolution. This prevented IRS from measuring improvements over the status quo. The certification report gives no results for customer satisfaction and notes that surveys on customer satisfaction were not done. IRS reported the results of measures on initial contact resolution—the percentage of calls that IRS resolved in one contact. However, the rate—43 percent—is much lower than the goal of 95 percent. Nonetheless, IRS gave the system a “pass” mark on that indicator stating that the “increased functionality expected in future releases of ICP should increase the overall ICR rate.” Furthermore, IRS’ measurement of how ICP 1.5 affected the quality of work life was limited to holding focus group discussions. Thirty-one of the 311 employees out of the Nashville office participated in the focus groups. The 3 focus groups were made up of 14 experienced CSRs, 12 inexperienced CSRs, and 5 managers. According to the certification report, the experienced CSRs were “excited about the ICP system,” but they expressed several concerns ranging from technological problems to lack of training. The report cautioned that “unless their concerns are addressed, the impression of the ICP system will turn into that of a curse rather than the now perceived blessing.” Similarly, the managers said the system offered many promises, but they too were concerned about the technical problems associated with the system. The inexperienced CSRs were not as enthusiastic about the system as the experienced CSRs and managers. While they had concerns similar to the experienced CSRs, they were very concerned about the amount of system downtime. In our July 1995 report on TSM, we said that although IRS recognized the importance of testing, it had not yet developed a complete and comprehensive testing plan for TSM. We said that individual TSM systems were developing their own test plans, which IRS described as rudimentary and inadequate. If systems like ICP are not adequately tested, design and development errors may go undetected, leading to performance shortfalls. Similar to ICP, IRS failed to thoroughly test its Service Center Recognition Image Processing System (SCRIPS). The pilot test of SCRIPS was incomplete because it (1) did not certify all software applications that were to be used during 1995 and (2) did not test SCRIPS ability to handle peak processing volumes. Many of the problems IRS experienced with SCRIPS, such as slow processing rates and system failures, might have been anticipated had IRS thoroughly tested the system before placing it into operation. IRS’ technical certification report stated that the system was available to users more than 98 percent of the time during the 19-day test period. However, it reached that percentage by excluding 3 days in which the system was down. IRS excluded the downtime from the test results because officials said they believed they had corrected the technical problem and that it would not recur. Had the 3 days been included, the system would have been available to the users about 95 percent of the time. Downtime may be caused by problems with system elements related to ICP, but not directly measured in the ICP test. Most of the CSRs we interviewed in December 1995 and March 1996 said downtime was a problem at their site. Of the 185 CSRs who responded to this question, 82 percent said downtime had disrupted customer service, at least to some extent. The representatives considered downtime to be those times when they were unable to access information through their workstation, regardless of the cause. They told us that when the system went down they were unable to provide customer service. They said they either called the taxpayer back when the system was back up, or they told the taxpayer to call back later, anticipating that the system would be back in operation when the taxpayer called. IRS officials in Nashville said the downtime stemmed from power outages, telecommunications problems, and connectivity problems with the systems from which ICP pulls data. Both the Cincinnati and Atlanta customer service centers experienced similar problems with connectivity to these old systems. IRS officials told us that they plan to conduct a more thorough pilot test of the next release of ICP. Software acceptance testing began in May 1996 and was scheduled to be completed in September 1996. ICP 2.0 was then to be subjected to 3 months of operational testing, using 40 CSRs at the Fresno customer service center beginning on September 30, 1996. It was to be expanded using about 160 CSRs in January 1997 and then rolled out to other customer service centers beginning in May 1997. However, on July 31, 1996, in a memorandum to the Commissioner and Deputy Commissioner, the Associate Commissioner for Modernization cancelled the September 30, 1996, pilot start date on the recommendation of the Business Site Executive. The Associate Commissioner noted that continued slippage in milestone dates for software programming and testing had jeopardized the pilot start date. To address concerns about the project, IRS hired a contractor to perform a risk assessment of the entire project. We believe that this decision is a positive indication of IRS’ desire to ensure that the system is sound before it is tested in production. The final results of this risk assessment were submitted in October 1996. IRS has prepared a draft evaluation plan that it believes will enable them to make a sound business decision about further investment in ICP. In an interim report dated August 21, 1996, the contractor recommended that the pilot test be delayed at least 3 months. The contractor cited various reasons why the test should be delayed. For example, during software testing, ICP failed to recognize certain data or taxpayer issues when such data or issues existed, and it failed to shut down when data were entered into certain accounts that were supposed to be protected from additional data entry. The contractor also cited problems with (1) the accuracy of data and with the updating of taxpayers’ IDRS accounts; (2) the definition of the user requirements for ICP 2.0; and (3) hardware differences among development, test, and production sites. The contractor stated that the number of problems identified during software testing continue to increase and were not likely to be fixed in the near term. According to IRS’ Customer Service Vision, ICP was expected to be the vehicle to provide CSRs access to the information they would need to answer all types of calls coming from taxpayers. Also, IRS planned to combine a phase of the collection process with customer service.However, according to IRS officials, after experimentation at the Nashville customer service center prototype, IRS is now reconsidering the extent to which CSRs will be able to answer the broad range of taxpayer questions, which are anticipated if IRS reduces the current level of employee specialization and combines the customer service and some of the collection functions. Modifications to ICP and/or subsequent investments in information technology may be required as the roles and responsibilities of a CSR continue to evolve. IRS hired a contractor and formed a team in February 1996 to examine the customer-service work processes and duties of CSRs. IRS acknowledges that many questions still need to be resolved on future job scope and structure of the customer service position. The contractor has been focusing on the redesign of current operations, systems, and organizations and the design of the CSR’s position. The contractor’s task is to develop a quality oriented, workable customer service system that furthers IRS’ objectives, enhances employee and customer satisfaction, and maximizes efficient use of resources. The draft design report was issued to IRS for review and comment on August 30, 1996. IRS has traditionally operated its telephone activities along functional lines, with employees specializing in specific areas. As such, an IRS telephone representative does not handle a broad range of inquiries. For example, if a business taxpayer called IRS regarding a balance due, the taxpayer would be routed to an IRS employee who specialized in handling business accounts and who handled only business account calls. As originally envisioned, ICP was to allow CSRs to perform a wide range of tasks, rather than have specific areas of expertise. ICP was to consolidate data from multiple databases and eliminate the complex command codes that IRS employees are now required to know in order to access and update taxpayer account information. Also, IRS planned to combine its initial efforts to collect taxes owed with its traditional customer service work. Essentially, with this blending of work, a CSR would be expected to answer all types of taxpayer calls. For example, a representative could receive a call from an individual taxpayer inquiring about a refund, and the next call could be an income tax preparer asking questions about IRS procedures or tax law. The CSR described in the vision would require a far broader knowledge base and much more extensive training than under the traditional telephone operations. IRS is reconsidering the extent to which CSRs will be able to answer the broad range of questions. IRS officials said they tested the blending concept at the Nashville prototype site and concluded that blending all the duties into one position was not feasible. The work systems design team is expected to decide how the work will be performed and define the duties of CSRs. Senior executives say they are committed to merging the taxpayer service and compliance functions. They acknowledge that certain issues must be resolved, such as how much tax knowledge a CSR needs to have, the proper skill level, and what authority the position should have to make certain decisions about a taxpayer account. CSRs that we talked with had mixed views on the extent to which blending has improved IRS’ ability to serve taxpayers. Twenty-two percent of the CSRs said blending improved their ability to assist taxpayers to little or no extent while another 22 percent said it improved their ability to assist taxpayers to a great extent. Some CSRs said blending allows them to provide one-stop service to the taxpayer without transferring them to other CSRs, while others said that one CSR cannot be responsible for performing multiple jobs. Some CSRs also said blending causes inaccuracy and requires more time per call because CSRs are less proficient when performing multiple jobs. IRS officials said that ICP 2.0 has been designed to provide CSRs with the most basic capabilities that both traditional telephone assistors and collection staff would find useful. Indeed, many of the capabilities expected from ICP 2.0 would provide clear advantages over IRS’ existing systems. However, until the role of the CSR is defined, it is unlikely that IRS will be able to provide information technology solutions that maximize productivity and customer service. As we have previously reported, organizations that successfully develop systems and achieve significant operational improvements do so only after analyzing and redesigning critical business processes. At the time of our review, the process of designing, testing, and implementing the role and processes surrounding the CSR was still not complete. Until IRS completes its work systems design effort, the information technology requirements to support CSRs will not be fully understood. The information CSRs need and the presentation of data might change from IRS’ initial vision and current ICP requirements because of the results of the work systems design effort. Therefore, CSRs may not require the same capabilities from ICP, as previously envisioned, in order to provide customer service. At the time of our visits, some sites were not using all of ICP’s capabilities because current duties did not require those capabilities. For example, CSRs at the Atlanta and Cincinnati customer service centers were using ICP to access IDRS when responding to inquiries from taxpayers who had received collection notices from IRS. They did not use ICP’s capabilities to access additional databases such as ACS and AUR. Customer service center officials reported that they were not servicing the kinds of calls that require access to either ACS or AUR. None of the ICP software development projects reviewed fully satisfy any of the KPAs that the SEI’s CMM requires to classify as a CMM level 2 rating or “a repeatable software development process.” In this regard, we found that three IRS organizations developing ICP software are extremely weak in the following KPAs: requirements management, software project planning, software project tracking and oversight, software quality assurance, and software configuration management. As a result, successful delivery of ICP 2.0 software is unlikely. Each of the five KPAs, along with examples of how the software development organizations compare to the KPA goals, is summarized below. Appendix III details how well each of the three organizations performed the KPA goals. Requirements Management - The purpose of requirements management is to establish a common understanding and agreement between the customer and the software project management on the customer’s requirements that are to be addressed through the software. One of the two goals of this KPA states that, “software plans, products, and activities are kept consistent with the system requirements allocated to software.” While IRS produces a number of documents—for example, (1) the configuration item list; (2) administrative request for information services; (3) the system architectural description; and (4) the concept of operations, which contains varying levels of detail on customer requirements—IRS does not update these documents, as requirements change, to ensure that these document are complete, consistent, or current. As a result, IRS has no assurance that the code being written and tested is traceable to customer requirements. Software Project Planning - The purpose of software project planning is to establish reasonable plans for performing the software engineering and for managing the software project. One of the three goals within this KPA states that, “software project activities and commitments are planned and documented.” IRS does not have a defined process governing software project planning. Moreover, the ICP software projects do not have documented software plans. Without these plans, IRS cannot effectively measure and monitor software development progress and take appropriate action when needed. Software Project Tracking and Oversight - The software project tracking and oversight process provides insight into actual project progress so that management can take effective actions when the software project’s performance deviates significantly from the software plans. One of the three goals within this KPA is that “actual results and performances are tracked against the software plans.” As noted above, IRS does not have ICP software development plans, and while it tracks the software project against schedules, these schedules are not derived using generally accepted government or industry software engineering methods. As a result, management cannot tell when actual progress warrants corrective action. Software Quality Assurance - The purpose of software quality assurance is to enable management to assess the quality of the process being used by the software project and of the products being built. Two of the four goals within this KPA emphasize that (1) “software quality assurance activities are planned” and (2) “adherence of software products and activities to applicable standards, procedures, and requirements is verified objectively.” The ICP software projects do not have software quality assurance plans. In addition, a software quality assurance group does not participate in certain required software quality assurance functions, such as the preparation, review, and audit of projects’ software development plans, standards, and procedures. As a result, IRS has no assurance that the ICP software is being developed in a quality fashion and will perform as intended. Software Configuration Management - The purpose of software configuration management is to establish and maintain the integrity of products of the software project throughout the project’s software life cycle. Two of the four goals of the configuration management KPA require that (1) “software configuration management activities be planned” and (2) “software work products be identified, controlled, and available.” The ICP software projects reviewed do not have software configuration management plans. In addition, although IRS controls changes to source code using a tool called Source Code Control System, the requirements within this KPA require change control to all software products created within the entire software life cycle. Specifically, IRS has not identified software work products—other than source code—such as requirements documentation, design specifications, test plans and results that need to be placed under configuration management. As a result, IRS does not know whether all its software products are complete, consistent, and current. Modernizing IRS’ systems is critical to IRS reaching its Customer Service Vision. As envisioned, ICP is planned to offer some clear advantages over IRS’ existing information systems and could improve taxpayer services. However, the success of ICP may be at risk because IRS has made substantial investments in the system without having (1) validated the costs and benefits by thoroughly testing the ICP system, (2) finalized the redesign of work processes that ICP will support, and (3) achieved the software development maturity needed to successfully build the envisioned capabilities within planned cost and milestones. Some of these problems are evident in recent slippage in milestone dates for software programming and testing that forced the cancellation of the September 30, 1996, pilot start date for ICP release 2.0. The contractor’s interim report assessing the risks associated with ICP development also supports delaying the pilot test. IRS has invested millions of dollars in ICP without having the cost and benefit data needed to fully assess the program, including analyzing program risks and making the most appropriate investment decisions. Furthermore, IRS’ testing of ICP 1.5 was limited and lacked baseline measures to gauge the success of this and future releases. Until IRS settles outstanding issues with its work processes, such as the scope of the duties of CSRs, it will not be in a position to adequately project whether ICP will provide the necessary capabilities or be the best system for customer service. The views of CSRs were generally supportive of an early version of ICP. However, continuing with plans to develop and deploy ICP to support unmeasurable benefits is risky. In this regard, until IRS implements a way to measure benefits, the extent to which ICP is likely to improve customer service and provide a positive return on investment cannot be determined. IRS is unnecessarily risking hundreds of millions of dollars by attempting to develop ICP software without having the requisite processes for doing so. Concurrent with the risk assessment being performed by the contractor, we recommend that the IRS Commissioner immediately limit deployment of ICP workstations to those already purchased until (1) projected costs and benefits are better known and can be validated by testing the system in a realistic operational environment, using baseline performance measures and (2) decisions are made on work processes, including the blending of collection and service work and specific duties of CSRs. We also recommend that expedient steps be taken to better position IRS to develop its software successfully and to protect its software investments. Specifically, we recommend that the IRS Commissioner take the following actions: Develop and implement an action plan to ensure that ICP software is developed by an organization(s) with at least a level 2 CMM rating. Delay any major investment in ICP software until the action plan is implemented. We requested comments on a draft of this report from the Commissioner of Internal Revenue or her designated representative. Responsible IRS officials, including the Customer Service Site Executive and the National Director, Customer Service Planning and Systems Division, provided IRS’ comments in a November 21, 1996, meeting. These comments were supplemented by a November 26, 1996, memorandum from the National Director, Customer Service Planning and Systems Division, and the Deputy Chief Information Officer (Systems Development) that addressed our recommendations and clarified remarks made during our discussion. We considered IRS’ comments and modified this report where appropriate. IRS officials agreed with our recommendation to limit further deployment of ICP workstations to those already purchased. They are currently considering several alternatives for reevaluating ICP. They said these alternatives, along with a recommendation, will be presented to IRS’ Investment Review Board in the near future. Additionally, IRS officials generally agreed with our assessment of ICP software development processes and agreed with our recommendation that they need at least a CMM level 2 capability to develop ICP software. The officials added that future ICP development is to be done using CMM level 2 processes and that, as we recommended, major investments in ICP will be delayed until this level of capability is achieved. They also added that their plan for achieving this level of capability involved two options—software development process improvements and heavy reliance on software development contractors. With respect to the former, the officials cited examples of improvement initiatives under way and planned, such as use of a requirements traceability matrix and software quality assurance program. We believe IRS’ two proposed actions to improve software development capabilities are not totally responsive to our recommendation. First, while the software process improvements cited are a step in the right direction, these actions should be part of a complete and comprehensive action plan for process improvement, as we recommended, which is rooted in SEI’s CMM level 2 KPA requirements. Second, to effectively acquire software using development contractors, IRS must have at least SEI defined CMM level 2 software acquisition processes. Moreover, it must ensure that its development contractors have at least level 2 development capabilities. Accordingly, IRS’ action plan for ICP should specify how this goal will be accomplished before it relies on contractors to develop ICP. IRS officials also stated that some ICP software had been developed using nationally recognized standards. For example, they cited software for computer screens, developed by IRS for use on multiple systems, including ICP. However, as stated in the objectives, scope, and methodology section of this report, our software capability assessment addressed those IRS organizations responsible for developing ICP applications software. IRS officials raised concerns about the amount of money cited in our report as spent on ICP through fiscal year 1995. Rather than $150 million, they now believe the investment in ICP through 1995 is about $73 million. Throughout our review, we had difficulty determining the amount spent on ICP. At one point, IRS officials told us that $171.5 million had been spent on ICP through fiscal year 1995, as reported in the May 6, 1996, Treasury report to the House and Senate Appropriations Committees. They later told us they found errors in that estimate, and the actual investment in ICP through 1995 was about $150 million. Now, they believe the $150 million cost projection was overstated because it included costs for the Aspect Automated Call Distributor System, which are not directly attributable to ICP. While we agree that some equipment costs are included in the $150 million figure, we are uncertain how much is attributable to the Aspect system because we did not validate the accuracy of IRS’ estimates. Accordingly, the $150 million was retained in this report. We are sending copies of this report to the Ranking Minority Member of your Subcommittee, the Chairman and Ranking Minority Member of the Senate Committee on Finance and other appropriate congressional committees, the Secretary of the Treasury, the Commissioner of Internal Revenue, and other interested parties. Major contributors to this report are listed in appendix IV. If you or your staff have any questions concerning this report, please call me on (202) 512-8633. This section describes the methodology we used to evaluate the software development capabilities of the organizations that are developing ICP software. The Software Capability Evaluation (SCE) is a method for evaluating agencies’ and contractors’ software development processes against the Software Engineering Institutes’s (SEI) five-level software Capability Maturity Model (CMM), as shown in table I.1. These levels, the key process areas (KPA) described within each level, and the goals within each KPA, define an organization’s ability to develop software and can be used to guide software development process improvement activities. The findings generated from an SCE identify (1) process strengths that mitigate risks, (2) process weaknesses that increase risks, and (3) improvement activities that indicate potential mitigation of risks. Continuous process improvement is enabled by quantitative feedback from the process and from piloting innovative ideas and technologies. Detailed measures of the software process and product quality are collected. Both the software process and products are quantitatively understood and controlled. The software process for both management and engineering activities is documented, standardized, and integrated into a standard software process for the organization. All projects use an approved, tailored version of the organization’s standard software process for developing and maintaining software. Basic project management processes are established to track cost, schedule, and functionality. The necessary process discipline is in place to repeat earlier successes on projects with similar applications. The software process is characterized as ad hoc, and occasionally even chaotic. Few processes are defined, and success depends on individual effort. In our July 1995 report, we reported that IRS was a CMM level 1 software development organization and that unless IRS improved its software development capability, it was unlikely to build Tax Systems Modernization (TSM) systems timely or economically. In June 1996, we reported that IRS had begun to act on our recommendations in this area, however, none of the actions were complete or institutionalized. At that time, IRS’ Chief Information Officer agreed that IRS was not yet institutionally a CMM level 2, but stated that some CMM level 2 processes were being used to develop Integrated Case Processing (ICP). Therefore, we evaluated ICP software development organizations that were said to be using CMM level 2 requirements. Specifically, we evaluated two ICP version 2 subsystems that are being developed in three locations—Dallas, Texas; Austin, Texas; and Fresno, California. We evaluated the software development processes used on these projects, focusing on KPAs necessary to achieve a “repeatable” capability or CMM level 2. According to SEI, organizations that have a repeatable software development process have been able to significantly improve their productivity and return on investment. In contrast, organizations that have not developed the process discipline necessary to better manage and control their projects at the repeatable level incur greater risk of schedule delay, cost overruns, and poor quality software.These organizations rely solely upon the variable capabilities of individuals, rather than on institutionalized processes considered basic to software development. According to SEI, KPAs for a repeatable capability are considered the most basic in establishing discipline and control in software development and are crucial steps for any project to mitigate risks associated with cost, schedule, and quality. These KPAs are identified and described in table I.2. Defining, validating, and prioritizing requirements, such as functions, performance, and delivery dates. Developing estimates for the work to be performed, establishing the necessary commitments, and defining the plan to perform the work. Tracking and reviewing software accomplishments and results against documented estimates, commitments, and plans and adjusting these based on the actual accomplishments and results. Selecting qualified contractors and managing them effectively. Reviewing and auditing the software products and activities to ensure that they comply with the applicable processes, standards, and procedures and providing the staff and managers with the results of their reviews and audits. Selecting project baseline items, such as specifications; systematically controlling these items and changes to them; and recording and reporting status and change activity for these items. Table II.1: CSRs’ Views on the Extent That ICP Has Allowed Them to Improve Customer Service in Selected Areas (as a Percentage of All Comments). Decreasing Case Inventory Delivery System cycle time Table II.2: CSRs’ Views on the Extent That Various ICP Capabilities Have Improved Their Ability to Do Their Jobs (as a Percentage of All Comments) To a very great extent Servicewide Electronic Research Project (SERP) is IRS’ automated system for researching IRS publications. Table III.1 summarizes our detailed findings from our software capability evaluation at three of IRS’ ICP development centers. As mentioned in appendix I, we evaluated the software development processes used on ICP software development projects at three centers, focusing on the key process areas (KPA) necessary to achieve a Capability Maturity Model (CMM) level 2 rating. CMM level 2 is achieved by satisfying all of the five KPAs under this level. To satisfy a given KPA, all of that area’s goals must be satisfied. Satisfying a goal, in turn, requires effectively meeting all of the activities associated with that goal. Table III.1 identifies whether each of the IRS development centers satisfied the KPAs, the associated goals, and activities. In accordance with the Software Engineering Institute’s (SEI) CMM assessment methodology, the activities within the respective goals are characterized as (1) a “strength” if IRS’ implementation of the activity was effective, (2) a “weakness” if IRS’ implementation of the CMM activity was ineffective, or IRS failed to implement an acceptable alternative, and (3) “not applicable” if the activity does not apply to the center’s software development environment. Therefore, in table III.1, a goal is classified as “not satisfied” when any associated activity is classified as a “weakness” and a KPA is classified as “not satisfied” when any associated goal is classified as “not satisfied.” Requirements management: to establish a common understanding between the customer and the software project of the customer’s requirements that will be addressed by the software project. Goal 1 System requirements allocated to software are controlled to establish a baseline for software engineering and management use. Goal 2 Software plans, products, and activities are kept consistent with the system requirements allocated to software. Software configuration management: to establish and maintain the integrity of products of the software project throughout the project’s software life cycle. Goal 1 Software configuration management activities are planned. Goal 2 Selected software work products are identified, controlled, and available. The software engineering group reviews the allocated requirements before they are incorporated into the software project. The software engineering group uses the allocated requirements as a basis for software plans, work products, and activities. Changes to the allocated requirements are reviewed and incorporated into the software project. A software configuration management plan is prepared for each software project according to a documented procedure. A documented and approved software configuration management plan is used as a basis for performing software configuration management activities. A documented and approved software configuration management plan is used as a basis for performing software configuration management activities. A configuration management library system is established as a repository for the software baselines. The software work products to be placed under configuration management are identified. Products form the software baseline library are created and their release is controlled according to a documented procedure. (continued) Goal 3 Changes to identified software work products are controlled. Goal 4 Affected groups and individuals are informed of the status and content of software baselines. Software quality assurance: to provide management with appropriate visibility into the process being used by the software project and of the products being built. Goal 1 Software quality assurance activities are planned. Goal 2 Adherence of software products and activities to the applicable standards, procedures, and requirements is verified objectively. Change requests and problem reports for all configuration items/units are initiated, recorded, approved, and tracked according to a documented procedure. Changes to baselines are controlled according to a documented procedure. The status of configuration items/units is recorded according to a documented procedure. Standard reports documenting the Software Configuration Management activities and the contents of the software baseline are developed and made available to affected groups and individuals. Software baseline audits are conducted according to documented procedures. A software quality assurance plan is prepared for the software project according to a documented procedure. Software quality assurance group’s activities are performed in accordance with the software quality assurance plan. Software quality assurance group’s activities are performed in accordance with the software quality assurance plan. Software quality assurance group participates in the preparation and review of the project’s software development plan, standards, and procedures. Software quality assurance group reviews the software engineering activities to verify compliance. Software quality assurance group audits designated software work products to verify compliance. (continued) Goal 3 Affected groups and individuals are informed of software quality assurance activities and results. Goal 4 Noncompliance issues that cannot be resolved within the software project are addressed by senior management. Software project planning: to establish reasonable plans for performing the software engineering and for managing the software project. Goal 1 Software estimates are documented for use in planning and tracking the software project. Goal 2 Software project activities and commitments are planned and documented. Software quality assurance group periodically reports the results of its activities to the software engineering group. Deviations identified in the software activities and software work products are documented and handled according to a documented procedure. Software quality assurance group conducts periodic reviews of its activities and findings with the customer’s software quality assurance personnel, as appropriate. Deviations identified in the software activities and software work products are documented and handled according to a documented procedure. Estimates for the size of software work products(or changes to the size of the software work products) are derived according to a documented procedure. Estimates for the software project’s effort and cost are derived according to a documented procedure. Estimates for the project’s critical computer resources are derived according to a documented procedure. The project’s software schedule is derived according to a documented procedure. Software project planning is initiated in the early stages of, and in parallel with, the overall project planning. A software life cycle with predefined stages of manageable size is identified or defined. (continued) Goal 3 Affected groups and individuals agree to their commitments related to the software project. Software project tracking and oversight: to provide adequate visibility into actual progress so that management can take effective actions when the software project’s performance deviates significantly from the software plans. Goal 1 Actual results and performances are tracked against the software plans. The project’s software development plan is developed according to a documented procedure. The plan for the software project is documented. Software work products that are needed to establish and maintain control of the software project are identified. The software risks associated with the cost, resource, schedule, and technical aspects of the project are identified, assessed, and documented. Plans for the project’s software engineering facilities and support tools are prepared. The software engineering group participates on the project proposal team. The software engineering group participates with other affected groups in the overall project planning throughout the project life cycle. Software project commitments made to individuals and groups external to the organization are reviewed with senior management according to a documented procedure. A documented software development plan is used for tracking the software activities and communicating status. The size of the software work products(or size of the changes to the software work products) are tracked, and corrective actions are taken as necessary. The project’s software effort and costs are tracked, and corrective actions are taken as necessary. The project’s critical computer resources are tracked, and corrective actions are taken as necessary. The project’s software schedule is tracked, and corrective actions are taken as necessary. The software engineering technical activities are tracked, and corrective actions are taken as necessary. The software risks associated with cost, resource, schedule, and technical aspects of the project are tracked. (continued) Goal 2 Corrective actions are taken and managed to closure when actual results and performance deviate significantly from the software plans. Goal 3 Changes to software commitments are agreed to by the affected groups and individuals. Actual measurement and replanning data for the software project are recorded. The software engineering group conducts periodic internal reviews to track technical progress, plans, performance, and issues. Formal reviews to address the accomplishments and results of the software project are conducted at selected project milestones according to a documented procedure. The project’s software development plan is revised according to a documented procedure. The size of the software work products are tracked, and corrective actions are taken as necessary. The project’s software effort and costs are tracked, and corrective actions are taken as necessary. The project’s critical computer resources are tracked, and corrective actions are taken as necessary. The project’s software schedule is tracked, and corrective actions are taken as necessary. The software engineering technical activities are tracked, and corrective actions are taken as necessary. Actual measurement and replanning data for the software project are recorded. Software project commitments and changes to commitments made to individuals and groups external to the organization are reviewed with senior management according to a documented procedure. Approved changes to commitments that affect the software project are communicated to the members of the software engineering group and other software related groups. Leonard Baptiste, Jr., Senior Assistant Director Kelly A. Wolslayer, Senior Information Systems Analyst Madhav S. Panwar, SCE Team Leader David Chao, SCE Team Member Nancy M. Donnellan, Information Systems Analyst Leonard J. Latham, SCE Team Member K. Alan Merrill, SCE Team Member Paul Silverman, SCE Team Member The first copy of each GAO report and testimony is free. Additional copies are $2 each. Orders should be sent to the following address, accompanied by a check or money order made out to the Superintendent of Documents, when necessary. VISA and MasterCard credit cards are accepted, also. Orders for 100 or more copies to be mailed to a single address are discounted 25 percent. U.S. General Accounting Office P.O. Box 6015 Gaithersburg, MD 20884-6015 Room 1100 700 4th St. NW (corner of 4th and G Sts. NW) U.S. General Accounting Office Washington, DC Orders may also be placed by calling (202) 512-6000 or by using fax number (301) 258-4066, or TDD (301) 413-0006. Each day, GAO issues a list of newly available reports and testimony. To receive facsimile copies of the daily list or any list from the past 30 days, please call (202) 512-6000 using a touchtone phone. A recorded menu will provide information on how to obtain these lists.

Summary: Pursuant to a congressional request, GAO reviewed the Internal Revenue Service's (IRS) Integrated Case Processing (ICP) systems development effort, focusing on: (1) IRS' assessment of ICP costs and benefits and users' perceptions of the system's benefits; (2) IRS' testing of ICP; (3) IRS' ongoing efforts to redesign its customer service work processes to fully use ICP capabilities; and (4) the software development processes being used for ICP. GAO found that: (1) improving service to taxpayers is an important goal that IRS' Customer Service Vision shows promise in addressing, but the promise anticipated by the vision is unlikely to be fulfilled unless changes are made in the development and deployment of ICP; (2) IRS estimated that about $150 million was spent on ICP from 1993 to 1995 and that an additional $77 million will be spent through 1996; (3) overall, IRS plans to spend about $641 million on ICP through fiscal year 2000; (4) despite this sizable investment, costs and benefits remain uncertain because: (a) the scheduled rollout of ICP workstations continues to change; (b) the ICP capabilities have not been finalized; (c) certain benefits are still to be determined; and (d) the software is still being developed; (5) IRS planned that certain ICP capabilities being developed would be pilot tested beginning on September 30, 1996, but in a memorandum dated July 31, 1996, the Associate Commissioner for Modernization postponed the pilot test indefinitely; (6) IRS developed and initiated a limited deployment of the initial ICP version; (7) the test results provided little insight on the potential benefits of the system, because IRS did not adequately measure ICP's impact on business operations; (8) IRS officials recognized the limitations of the testing and told GAO that testing of the next software release would be more comprehensive; (9) it is unclear how this and future versions will support new work processes that are being designed; (10) according to IRS' Customer Service Vision, ICP was expected to be the vehicle to provide customer service representatives with access to information that would enable IRS to combine a phase of the tax collection process with customer service; (11) IRS is now reconsidering the extent to which the collection process can be combined with customer service and is reconsidering the range of tasks a customer service representative can be expected to perform; and (12) the software development processes in place at IRS organizations responsible for developing ICP software are extremely weak, making the likelihood of their producing quality ICP software on time and within budget very low.