Since May 1999, IRS has submitted two spending plans and two interim, stopgap plans. On June 15, 1999, we reported on IRS' first spending plan, stating that the $35 million requested satisfied the legislative conditions for the use of ITIA funds and was consistent with our open recommendations. We also said that it was an appropriate first step but that the key to success would be effective implementation of the plan. We recommended that future plans specify progress against prior plan commitments and that the next plan clarify the roles and responsibilities of IRS, its prime systems integration services contractor (PRIME) for modernization, and other modernization support contractors. Because IRS was unable to submit its second plan on time, it submitted in December 1999 a stopgap request for $33 million and additional time as an interim measure. We reviewed the request and raised concerns about projects that were scheduled to begin detailed design and software development before, among other things, the enterprise architecture was completed and the ELC was defined and implemented. Later that December, the appropriations subcommittees approved IRS' $33 million funding measure but, in discussions and correspondence, directed IRS to (1) expedite completion of the architecture and implementation of the ELC and (2) explain in future expenditure plans how it plans to manage the risk of performing detailed design or development work if the architecture is not completed or the ELC is not implemented. Subsequently, IRS reassessed its modernization program plans and decided to restructure the program by scaling back its systems development efforts until it addressed our concerns and put in place the requisite modernization management capability. On March 7, 2000, IRS submitted to the appropriations subcommittees its second spending plan, which reflected these restructuring decisions and sought to obligate about $176 million from ITIA. On March 31, 2000, we briefed the subcommittee and subsequently reported that (1) the plan satisfied the legislative conditions and was generally consistent with recommendations contained in our earlier reports and (2) IRS met few of the commitments in its first spending plan, even though it later received an additional $33 million and nearly 5 months additional time to do so. We stated that the key to success would be whether IRS effectively implemented the second plan. We also stated that until IRS implemented the plan, it would continue to lack key modernization and technical controls. In April 2000, the appropriations subcommittees approved IRS' plan for $148.4 million of the funds requested and underscored the need for IRS to pursue its restructuring plans by, among other things, directing IRS to establish modernization management capabilities and controls--including completion and implementation of the blueprint and systems life cycle methodology--before it began building systems. Recognizing that it would fully obligate the $148.4 million before it was ready to submit another spending plan, on August 25, 2000, IRS submitted its second interim plan. This interim plan requested $32.8 million to carry the modernization program into early fiscal year 2001. IRS did not fully satisfy all of the conditions that its appropriations subcommittees established in their April 2000 letters releasing modernization investment account funding. However, IRS has made important progress on both modernization projects and initiatives to address long-standing modernization management weaknesses. Our assessment of IRS' success in satisfying each condition is detailed below. IRS did not adhere to the approved and funded March 7, 2000, spending plan and did not seek approval to deviate from the plan. IRS' position is that variances from the approved funding levels were small and inevitable given the scale and complexity of the program and that because IRS is closely managing the variances, they do not necessitate the subcommittees' actions. In sum, IRS varied from the approved spending levels for twelve program projects and initiatives. These variances ranged in amount from approximately $44,000 to $9,000,000. In our view, such variances are to be expected given the projects' and initiatives' complexity and early stage in the life cycle and the inherent uncertainty associated with software/systems cost estimating. IRS did not meet its cost and schedule commitments on most projects and initiatives in its March 7, 2000, spending plan. For example, Milestone III for the Customer Account Data Engine (CADE) project slipped by 3 months, and the estimated cost for meeting Milestone III rose by about $1.4 million (9.1 percent). Also, Milestone III for the STIR project slipped 2 months, and its cost increased about $1.4 million (17.0 percent). In our view, these cost and schedule variances are to be expected because, as we previously reported, the milestone commitments for most of the initiatives in the March 2000 plan were not based on detailed work breakdown structures and schedules. Rather, they were merely targets that IRS had represented in its plan as achievable commitments. As a result, these variances are not good measures of progress. Most modernization initiatives, including key projects, have made important progress since March 2000. For example, the CADE and e-Services projects each completed the system planning and definition phase of the ELC, including a preliminary business case showing a positive return on investment, and both projects have met ELC criteria for beginning preliminary design work. However, one project, CAP, has been approved to begin the detailed design and development phase (i.e., the system is being built) without sufficient definition and without a compelling business case to justify treating it as a near-term investment priority. Specifically, the CAP business case does not show a positive return on investment; the $150 million estimated total project cost does not include the cost to define and build interfaces with projects that are yet to be defined (i.e., Integrated Financial Systems' standard general ledger, CADE, and STIR); CAP business requirements are not yet sufficiently defined to verify CAP was approved to begin the development phase with two important conditions that could affect the system's baseline requirements and functions (and thus cost); and CAP integration with other modernization projects is unclear because CAP is not included in the integrated master schedule and monthly program management reviews. Another project, STIR, was granted approval to begin the preliminary design phase without a risk-based approach to defining and economically justifying its requirements (i.e. preliminary security risk assessment, preliminary security certification package, and preliminary business case). Such an approach, as discussed in our November 1999 guide on security risk assessment practices of leading organizations, is a federal requirement and a key element of successful organizations' security risk assessment programs. IRS has fully addressed two of the five modernization management weaknesses cited by the subcommittees. Namely, it has hired a BSMO executive, and it reports that 20 of its 22 task orders issued since June 2000 are performance based. We reviewed 3 key task orders and found that they reflected the tenets of performance-based contracting. IRS is also making progress in addressing other modernization management weaknesses. For example, since March 2000, as part of the ELC, it has defined processes, roles, responsibilities, etc. for implementing selected Level 2 key process areas of the Software Engineering Institute (SEI) Software Acquisition Capability Maturity Model . IRS plans to train staff and have all projects following these by the November 2000-to- January 2001 timeframe. In September 2001, IRS plans to conduct an evaluation (via an independent assessor) to ensure compliance with SEI's Level 2 requirements. In addition, IRS has taken other steps to increase its acquisition discipline, such as conducting monthly program management reviews with the modernization contractor and establishing an integrated master schedule to track the projects' schedules and interdependencies. In July 2000, IRS defined and baselined most parts of the ELC. Our review of selected projects (CAP, CADE, and STIR) indicates that the ELC is generally being followed. Further, IRS has defined IT investment management controls and is in the process of incorporating them into the ELC. The initial release of IRS' enterprise architecture was being drafted and was not yet available for our review, however, some parts of the ELC are still being developed and finalized (e.g., program control policies and procedures, 75 to 80 change requests, legacy system impact supplement, and enterprise architecture management supplement). While IRS did not satisfy the September 30, 2000, commitments it made in the March 2000 plan for issuing the enterprise architecture and fully implementing the ELC, it is making important progress and plans to complete both between November 2000 and January 2001. IRS has revised and validated its March 2000 spending plan cost and schedule estimates to increase their reliability. According to IRS, it tasked the contractor to develop, for each initiative, a detailed work breakdown schedule of the tasks, deliverables, staff, and time needed to complete the initiatives in accordance with the ELC. With the resulting staffing and time requirements, the contractor applied its cost rates to determine cost and schedule estimates and then submitted the estimates to IRS in task order proposals. IRS validated these costs during the task order "definitization" process by, among other things, comparing them to historical and current contractor rates charged to IRS and other federal agencies (e.g., General Services Administration). IRS then negotiated rates that could not be validated. For two projects--CADE and Customer Communications--we verified that IRS followed this process (although we did not specifically evaluate the reliability of IRS' cost validation). IRS has made important progress in establishing its BSMO, although this office is not yet fully operational. Specifically, the office is in place, staff have been assigned, and roles, responsibilities, policies, and procedures have been defined that include SEI Level 2 software acquisition processes. However, IRS has not yet developed charters for the eight units (e.g., Architecture Management, and Program Oversight) within BSMO, although IRS plans to do so by the end of October 2000; completed training on BSMO roles, responsibilities, policies, and procedures and the ELC, although IRS plans to do so between November 2000 and January 2001; and filled 21 vacant positions, or 18 percent of total BSMO staffing levels, including managers for program integration, program control, and program planning and four staff for configuration management. Until these tasks are completed, IRS will continue to report BSMO readiness as a major program risk. Although IRS has not fully met all the conditions specified by the subcommittees, it has moved aggressively and made important progress since March 2000 in advancing many projects and addressing modernization management weaknesses. Nevertheless, until these management weaknesses are fully addressed, key controls will be missing, thereby increasing the risks that projects will not perform as intended and/or cost more and take longer to complete. These risks are not as severe early in a project life cycle when it is being planned, but they escalate as a project moves into the detailed design and development phase of its life cycle. Given that IRS plans show that several projects are scheduled to begin detailed design and development within the next 6 months, expeditious implementation of these program management controls is of paramount importance. Further, IRS' CAP project has prematurely passed milestone III before being economically justified and before addressing modernization integration issues. Also, IRS' STIR project is missing the requisite analysis of security threats and vulnerabilities so that a meaningful risk profile can be developed and cost effective system requirements to counter these risks can be specified. Because we have open recommendations to address IRS modernization management weaknesses that remain applicable, we reemphasize to the Commissioner of Internal Revenue that these recommendations should be implemented before IRS begins building software-intensive systems. In addition, we recommend that the Commissioner limit further investment in CAP until IRS (1) demonstrates that CAP is of sufficient business value to treat it as a near-term investment priority and (2) reports to the Senate and House appropriations committees on how it is mitigating the risks associated with beginning to build CAP before important program-level management controls are fully implemented. Further, we recommend that the Commissioner (1) direct the STIR project to complete as soon as possible a security risk assessment and validate STIR requirements against the risk assessment results and then (2) ensure that STIR requirements and the proposed design solution are economically justified through business case analysis. In commenting on a draft of this report, IRS' Chief Information Officer generally agreed with our findings and described, at a very high level, actions since our September 22, 2000, briefing that are completed, underway, and planned to address our recommendations. IRS stated that in November 2000 it plans to report on these actions in more detail to its Senate and House appropriations subcommittees. In closing, IRS stated that it appreciated our overall assessment of the management of the systems modernization program and that IRS has benefited from GAO's work. We are sending copies of this report to Senator Max Baucus, Senator Robert C. Byrd, Senator Orrin G. Hatch, Senator Joseph I. Lieberman, Senator Daniel Patrick Moynihan, Senator William V. Roth, Jr., Senator Ted Stevens, and Senator Fred Thompson, and to Representative Bill Archer, Representative Dan Burton, Representative William J. Coyne, Representative Stephen Horn, Representative Amo Houghton, Representative David R. Obey, Representative Charles B. Rangel, Representative Jim Turner, Representative Henry A. Waxman, and Representative C.W. Bill Young, in their capacities as Chairmen or Ranking Minority Members of Senate and House Committees and Subcommittees. We are also sending copies to the Honorable Charles O. Rossotti, Commissioner of Internal Revenue; the Honorable Lawrence H. Summers, Secretary of the Treasury; and the Honorable Jacob J. Lew, Director of the Office of Management and Budget. Copies will also be made available to others upon request. Should you or your staff have any questions on matters discussed in this report, please contact me at (202) 512-6240. I can also be reached by e-mail at hiter@gao.gov. Key contributors to this report are listed in appendix IV. F F R D C (M IT R E ) P ro g ra m M a n a g e m e n t O ffic e ($ 3 2 4 ) E L C E n h a n c e m e n t, M a in te n a n c e & F u lly D e p lo ym e n t E n te rp ris e A rc h ite c tu re a n d B lu e p rin t 2 0 0 0 C o n fig u ra tio n M a n a g e m e n t P o lic ie s a n d P ro c e s s e s ($ 1 ,6 1 7 ) ($ 3 1 0 ) ($ 8 4 4 ) variances could not be avoided given the size and complexity of the business systems modernization program. B u s in e s s In te g ra tio n ($ 1 ,4 0 5 ) C R M E x a m (1 1 2 0 re p la c e m e n t) S o lu tio n s D e ve lo p m e n t L a b ($ 1 ,3 9 3 ) Amountsintablearelistedinthousands($000). approved funding levels, IRS gave the following reasons why extra funds were needed: Customer Communications - the costs for modifying legacy systems to interface with Customer Communications were omitted from IRS' March 2000 spending plan. Virtual Development Environment - changes to scope of projects (e.g., CADE) required additional capabilities. and Funding ($000) Date and Funding ($000) Change ($000) our briefing and report on IRS' March 2000 plan1--the milestone commitments for most of the initiatives in the March plan were not based on detailed work breakdown structures and schedules, but rather were merely targets that IRS had represented in its plan as achievable commitments. As a result, these variances are not good measures of progress. The milestone changes are consistent with the Commissioner's stated position that initiatives will not be "schedule-driven." Rather, they will be "content-driven" and will not be allowed to proceed past a major ELC milestone unless they are ready to do so. TaxSystemsModernization:ResultsofReviewofIRS'March7,2000,ExpenditurePlan (GAO/AIMD-00-175, May 24, 2000) Completed ELC Milestone II work products and received Executive Steering Committee approval to proceed to Milestone III on August 8, 2000. Prepared preliminary business case showing positive return-on-investment. Issued performance-based task order to PRIME to develop necessary ELC work products to get Milestone III approval. Per ELC, conducted interim systems requirements review and produced interim systems requirements report. Per ELC, produced interim project management plan. Issued performance-based task order to PRIME to develop necessary ELC work products to get Milestone III approval. Completed ELC Milestone II work products and received Executive Steering Committee approval to proceed to Milestone III on August 8, 2000. Prepared preliminary business case showing a positive return-on-investment. Security Certification Package received an unqualified approval. Exited ELC Milestone III and received Executive Steering Committee approval to proceed to Milestone IV. Pilot on track for timely completion by September 28, 2000. Issued performance-based task order to PRIME to develop ELC required workproducts to get Milestone IV/V approval. * AccordingtoTreasuryInspectorGeneralforTaxAdministration(TIGTA) CAP is one of a collection of systems comprising IRS' Integrated Financial System project. CAP is to provide tax receipt and receivable analysis and reporting. Standard general ledger and other financial/administrative reporting as required by federal management directives are to be provided by other future projects. determine impact of enterprise data warehouse architecture standards on CAP baseline. identify whether ongoing legacy enhancements to masterfile will create new interfaces for CAP. CAP integration with other modernization projects is unclear because CAP is not included in the integrated master schedule and monthly program management reviews. without a risk-based approach to defining and economically justifying its requirements. Such an approach, as discussed in our November 1999 guide4 on security risk assessment practices of leading organizations, is a federal requirement and a key of element of successful organizations' security risk assessment programs. InformationSecurityRiskAssessments:PracticesofLeadingOrganizations(GAO/AIMD-00-33, November 1999). IRS plans to address these issues before milestone III. In March 2000, BSMO planned to hire a full-time business systems modernization executive. In June 2000, IRS hired this executive. These are Acquisition Planning, Solicitation, Requirements Development and Management, Project Management, Contract Tracking and Oversight, Evaluation, and Transition to Support. these by the November 2000-January 2001 timeframe. The BSMO's plans also includes evaluation in September 2001 by independent assessor to ensure compliance with SEI's level 2 requirements. The BSMO has also taken other steps to increase its acquisition discipline. For example, it and the PRIME contractor started conducting Monthly Program Management Reviews established an integrated master schedule to track the projects' schedules and interdependencies. However, neither of these two management steps included the CAP project. In March 2000, IRS was in the process of defining IRS, PRIME, and other support contractor roles and responsibilities. As part of plan for establishing BSMO, IRS has defined IRS and PRIME contractor roles/responsibilities. In doing so, IRS has clearly differentiated between its role as the acquirer and the PRIME's role as the developer in such areas as program leadership, architecture management, acquisition management, and program oversight and assurance, quality assurance, testing, risk management, and requirements management. IRS plans to implement these roles/responsibilities by December 2000. IRS has not yet defined roles and responsibilities for other modernization support contractors (e.g., Integrated Support Contractor for CAP project development). Since then, IRS reports that 20 of its 22 task orders issued since June 2000, were performance-based. Our review of 3 of these task orders showed that IRS was implementing the concepts of performance-based contracting. ELC implementation stated in the March 2000 plan, IRS has developed an ELC deployment plan to provide detailed training on ELC processes and procedures by the November 2000 to January 2001 timeframe. Review of selected projects (CAP, CADE, and STIR) indicates that ELC is generally being followed. Exceptions reflect: ELC tailoring whereby projects are allowed to delete applicability of selected work products. CAP and STIR initially followed another life-cycle management methodology because the ELC was not yet available for use. CAP and STIR have since been subjected to ELC requirements. business case assessment, portfolio prioritization, investment approval, investment monitoring, in-process evaluation, and post-implementation review. As defined, IRS' investment management approach appears consistent with our IT investment management framework. While these processes have been defined and IRS has begun using the business case procedure, it has not yet fully implemented all investment management controls. In March 2000, IRS had defined a high-level framework (consisting of approximately 60 notional work products) for developing its enterprise architecture and had begun to define "key concepts" that transcend individual architectural work products. Since then, IRS has determined that it will develop and complete its enterprise architecture in 3 phases--referred to as Releases 1.0, 1.1, and 2.0. Release 1.0 is referred to as Blueprint 2000. Applications (e.g. collections, exam, etc.) (e.g. user authentication, data access, image access, etc.) (e.g. messaging services, networks, hardware, operating systems, database management systems, etc.) Applications Infrastructure (lower pyramid) defined the framework's 60 work products and the process/management controls for populating these work products with the content associated with each release. Management and Other Work Products 57-Current Production Environment Release Plan Description(1 - 3 years) 58-Near-Term Sequencing and Sequencing and Release Plan (4 - 6 years) 60-Long-Term Sequencing and Release Plan (7+ years) platforms, backup/recovery, development environment and languages, data warehouse framework, and data base management. Nine have been completed, and IRS officials told us they plan to complete the remaining 14 by mid-November 2000. been drafting the 60 enterprise architecture products comprising Release 1.0. They are to be submitted for review to IRS business and other stakeholders by October 1, 2000. IRS estimates this review and approval process will take a month. IRS then plans to obtain Core Business Systems Executive Steering Committee approval in early November 2000. established structures and processes for developing, implementing, and monitoring the enterprise architecture. For example, Program Architecture Integration Team - provides forum for identifying and resolving enterprise architecture issues raised by ongoing projects. Program Integration Board - reviews projects at ELC milestones to ensure conformance with draft architecture work products and key concepts. required the PRIME to review the quality of each product to establish an enterprise architecture quality assurance function, and tasked the Federally Funded Research and Development Center (FFRDC) contractor (MITRE) with assessing release 1.0 completeness and adequacy. It tasked the contractor to develop for each initiative a detailed work breakdown structure/schedule of the tasks, deliverables, staff, and time needed to complete initiatives in accordance with the ELC. With the staffing and time requirements, the contractor applied its cost rates to determine how much the effort would cost and then submitted the cost and schedule estimates to IRS in a task order proposal. IRS validated these costs during the task order "definitization" process by, among other things, comparing them to historical and current contractor rates charged to IRS and other federal agencies (e.g., GSA) to assess their reasonableness. we verified that IRS did follow this process. Proposed Modernization Initiatives Program Level Activities 4,567 (1,941) Program Management Office ELC Enhancements, Maintenance, and Full Deployment Federally Funded Research and Development Contractor (MITRE) Management Reserve (324) (1,617) (2,559) Enterprise Architecture and Blueprint 2000 Configuration Management Policies and Procedures Business Integration (310) (844) Vision and Strategy, Tax Admininistration Vision and Strategy, Internal Management (1,405) (30) Customer Communications Customer Relationship Management Exam (1120 Replacement) Amountsintablearelistedinthousands($000). Customer Account Data Engine (CADE) Integrated Financial System (IFS) Collections Subledger Tax Account Subledger Security And Technology Infrastructure Release (STIR) Enterprise Systems Management (ESM) Telecommunications Enterprise Strategic Program (TESP) Solutions Demonstration Lab (SDL) Virtual Development Environment (VDE) Enterprise Integration and Test Environment (EITE) Customer Communications Customer Relationship Management Exam (1120 Replacement) Amountsintablearelistedinthousands($000). Pursuant to the mandate in Treasury's fiscal year 1998 and 1999 appropriations acts and requests by the Senate and House appropriations subcommittees, our objectives were to (1) determine to what extent IRS has satisfied the subcommittees' conditions regarding the ITIA funds and (2) provide any other observations about IRS' management of the program. The conditions specified by the subcommittees were: adhere to the March 7, 2000, spending plan as approved and funded or seek approval to do otherwise; meet the commitments made in the March 7, 2000, spending plan; address modernization management weaknesses and establish the capability to build systems by hiring a person to lead the business systems modernization office developing mature software acquisition capabilities, clearly defining and applying IRS and contractor roles and creating performance-based task orders, and fully implementing IRS' system life cycle management methodology--the Enterprise Life Cycle--and completing and using its enterprise architecture--Blueprint 2000--to define, direct, and control future modernization activities; and establishing more reliable cost and schedule estimates for the initiatives in the March 7, 2000, plan. To accomplish our objectives we took several steps. To understand the scope and content of each, we reviewed IRS' August 2000 interim ITIA funding request and other IRS documentation associated with the initiatives identified in the request. Then, to identify variances, we analyzed the interim request in relation to the conditions specified in the Senate and House responses to the March 7, 2000, spending plan. We compared progress on selected program and project initiatives-as reported in the request and other modernization documentation and supplemented by interviews of IRS and contractor officials-against deliverables promised and commitments made in the March 7, 2000, spending plan. The program initiatives we compared were Blueprint 2000 and the ELC and the project initiatives were CAP, CADE, and STIR. We selected (1) the Blueprint 2000 and ELC program initiatives because both are intended to address open GAO recommendations and (2) the CAP, CADE, and STIR project initiatives because our reviews of these projects were already under way for the subcommittees. In addition, we analyzed relevant documentation concerning the operational status of the Business System Modernization Office (BSMO), such as plans, status reports, policies and procedures, and descriptions of IRS and contractor roles and responsibilities. We also analyzed progress in definitizing task orders and making them performance based. To do this, we selected three task orders and compared them to performance-based contracting criteria. The task orders selected were for CADE, STIR, and the PRIME's program management office. We selected (1) the CADE and STIR task orders because the subcommittees had asked us to review these two projects and (2) the program management office task order to include one program-level task order in our scope. We analyzed the process that IRS and the contractor undertook to revise and validate cost and schedule estimates in the March 2000 spending plan and verified for two projects-CADE and Customer Communications- whether IRS adhered to its process. We selected these projects because they had the highest funding requirements. We collaborated with the Treasury Inspector General for Tax Administration (TIGTA) to avoid duplication of effort in reviewing program and project initiatives and incorporated the TIGTA results in this briefing where appropriate. Initiatives addressed by TIGTA included the Customer Communications and e-Services projects, ELC, BSMO, and Blueprint 2000. In order to meet the 30-day reporting requirement, we did not independently validate planned initiatives' cost estimates or confirm through review of system and project management documentation the IRS information provided on the initiatives' content and progress. We performed our work from August through mid-September 2000 in accordance with generally accepted government auditing standards. In addition to those named above, other key contributors were Bernard Anderson, Lorne Dold, Nancy DeFrancesco, Timothy Hopkins, Larry Korb, Karlin Richardson, Jay Pelkofer, Sherrie Russ, Tuyet Quan Thai, Aaron Thorne, and Teresa Tucker. The first copy of each GAO report is free. Additional copies of reports are $2 each. A check or money order should be made out to the Superintendent of Documents. VISA and MasterCard credit cards are accepted, also. Orders for 100 or more copies to be mailed to a single address are discounted 25 percent. Orders by mail: U.S. General Accounting Office P.O. Box 37050 Washington, DC 20013 Orders by visiting: Room 1100 700 4th St. NW (corner of 4th and G Sts. NW) U.S. General Accounting Office Washington, DC Orders by phone: (202) 512-6000 fax: (202) 512-6061 TDD (202) 512-2537 Each day, GAO issues a list of newly available reports and testimony. To receive facsimile copies of the daily list or any list from the past 30 days, please call (202) 512-6000 using a touchtone phone. A recorded menu will provide information on how to obtain these lists. Web site: http://www.gao.gov/fraudnet/fraudnet.htm e-mail: fraudnet@gao.gov 1-800-424-5454 (automated answering system)

Summary:
The Internal Revenue Service (IRS) has tried unsuccessfully to modernize its information systems. To prevent a repeat of past mistakes, Congress legislated explicit controls over IRS spending on systems modernization. Congress did so by creating Senate and House appropriations subcommittees, which specified four conditions. IRS has satisfied some, but not all, of the conditions set out by its appropriations subcommittees; however, it has moved aggressively and made important progress in advancing many projects and addressing modernization management weaknesses. Until these weaknesses are fully addressed, key controls will be missing, thereby increasing the risk that projects will not perform as intended or cost more and take longer to complete.