Tax Systems Modernization: Actions Underway But IRS Has Not Yet Corrected Management and Technical Weaknesses
Published: 1996-06-07T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: Pursuant to a legislative requirement, GAO reviewed the Internal Revenue Service's (IRS) actions to correct GAO-identified management and technical weaknesses that jeopardize its tax systems modernization (TSM) efforts.
section_title: What GAO Found
paragraphs: GAO found that: (1) IRS does not have a comprehensive strategy to maximize electronic filing because the present strategy targets only a small portion of the taxpayers likely to file electronically; (2) IRS strategic information management practices remain ineffective because information systems are not managed as investments; (3) IRS reengineering efforts lag behind the development of TSM projects; (4) IRS is improving its software development activities, but these improvements are not complete or institutionalized; (5) the IRS technical infrastructure, including systems architecture, integration planning, and system testing and test planning, is incomplete; (6) IRS has not established an effective organizational structure to consistently manage and control TSM; and (7) IRS plans to increase its use of contractors to facilitate TSM, but it has not been successful in managing all of its contractors.

section_title: Letter
paragraphs: In April 1995 briefings to your Committees and the Internal Revenue Service (IRS), we reported that the government’s efforts to modernize tax processing by investing what could be more than $8 billion in tax systems modernization (TSM) were at serious risk due to pervasive management and technical weaknesses. In our July 1995 report, we made several recommendations to IRS to correct these weaknesses. In this regard, GAO recommended, inter alia, that the Commissioner:
Formulate a comprehensive business strategy for maximizing electronic filings by targeting those sectors of the taxpayer population that can file electronically most cost-beneficially.
Take immediate action to improve IRS’ strategic information management by implementing a process for selecting, prioritizing, controlling, and evaluating the progress and performance of all major information systems investments, using explicit decision criteria. Further, by June 30, 1995, IRS should review all planned and ongoing systems for fiscal year 1996 using these criteria.
By December 31, 1995, implement disciplined, consistent procedures for software requirements management, quality assurance, configuration management, and project planning and tracking.
By December 31, 1995, complete an integrated systems architecture and security and data architectures.
The time frames that we recommended coincided with congressional deliberations on IRS’ 1996 and 1997 budget cycles and were intended to ensure that the Congress could have confidence that IRS could effectively start managing its fiscal year 1996 TSM budget.
Reflecting continued congressional concern with TSM, the Treasury, Postal Service, and General Government Appropriations Act of 1996 required that the Department of the Treasury provide a report to the House and Senate Committees on Appropriations (1) identifying, evaluating, and prioritizing all IRS systems investments planned for fiscal year 1996, using explicit decision criteria, (2) providing a schedule for successfully correcting weaknesses that we identified in April 1995, (3) presenting a milestone schedule for developing and implementing all projects included in the tax systems modernization program, and (4) presenting a plan to expand the utilization of external expertise for systems development and total program integration. The conference report on the Act directed that we review the Department of the Treasury report on IRS and, within 30 days of receipt of the Treasury report, provide your Committees our independent assessment of the actions taken by IRS to correct the weaknesses identified in our July 1995 report.
In March 1996 testimony to the Subcommittee on the Treasury, Postal Service, and General Government of the Committee on Appropriations, House of Representatives, we provided a progress report on actions taken by IRS to address our recommendations. On May 9, 1996, we received copies of the final report on TSM from the Department of the Treasury. As directed in the conference report on the Treasury, Postal Service, and General Government Appropriations Act of 1996, we have assessed IRS actions to correct its management and technical weaknesses, as delineated in Treasury’s report on TSM, and are reporting our findings to you by this letter.
subsections: section_title: Scope and Methodology
paragraphs: We reviewed the Department of the Treasury’s report, interviewed senior IRS officials responsible for the actions being taken to correct the management and technical weaknesses, and reviewed documentation. On June 4, 1996, we briefed senior Treasury and IRS officials, including the Deputy Secretary of the Treasury and the Commissioner of the IRS, on the results of our review.
We performed our work at IRS headquarters in Washington, D.C., between May 9, 1996 and June 4, 1996 in accordance with generally accepted government auditing standards. The Department of the Treasury and IRS provided comments on a draft of this report, which are discussed in the “Agency Comments and Our Evaluation” section and are reprinted in appendix I.
section_title: Background
paragraphs: IRS envisions a modernized tax processing environment which is virtually paper free and in which taxpayer information is readily available to IRS employees to update taxpayer accounts and respond to taxpayer inquiries. In our July 1995 report, we emphasized the need for IRS to have in place sound management and technical practices to increase the likelihood that TSM’s objectives will be cost-effectively and expeditiously met. A 1996 National Research Council report on TSM had a similar message. Its recommendations parallel the over a dozen recommendations we made in July 1995 to improve IRS’ (1) business strategy to reduce reliance on paper, (2) strategic information management practices, (3) software development capabilities, (4) technical infrastructures, and (5) organizational controls.
In the July 1995 report, we described our methodology for analyzing IRS’ strategic information management practices, drawing heavily from our research on the best practices of private and public sector organizations that have been successful in improving their performance through strategic information management and technology. These fundamental best practices are discussed in our report Executive Guide: Improving Mission Performance Through Strategic Information Management and Technology (GAO/AIMD-94-115, May 1994), and our Strategic Information Management (SIM) Self-Assessment Toolkit (GAO/Version 1.0, October 28, 1994, exposure draft).
To evaluate IRS’ software development capability, we validated IRS’ September 1993 assessment of its software development maturity based on the Capability Maturity Model (CMM) developed by Carnegie Mellon University’s Software Engineering Institute, a nationally recognized authority in the area. This model establishes standards in key software development process areas (i.e., requirements management, project planning, project tracking and oversight, configuration management, quality assurance, and subcontractor management) and provides a framework to evaluate a software organization’s capability to consistently and predictably produce high-quality products.
When we briefed the IRS Commissioner in April 1995 and issued our report documenting its weaknesses in July 1995, IRS agreed with our recommendations to make corrections expeditiously. At that time, we considered IRS’ response to be a commitment to correct its management and technical weaknesses.
In September 1995, IRS submitted an action plan to the Congress explaining how it planned to address our recommendations. In our March 1996 testimony to the House Appropriation Committee’s Subcommittee on Treasury, Postal Service, and General Government, we noted that this plan, follow-up meetings with senior IRS officials, and other draft and “preliminary draft” documents received through early March 1996 provided little tangible evidence that actions being taken would correct the pervasive management and technical weaknesses that continued to place TSM, and the huge investment it represents, at risk. This interim status report on IRS’ efforts to respond to our July 1995 recommendations noted that IRS had initiated a number of activities and made some progress in addressing our recommendations to improve management of information systems; enhance its software development capability; and better define, perform, and manage TSM’s technical activities. However, we reported that none of these steps had fully satisfied any of our recommendations. Consequently, IRS was not in an appreciably better position in March 1996 than it was in April 1995 to assure the Congress that it would spend its fiscal year 1996 and future TSM appropriations judiciously and effectively.
In a subsequent testimony before the Senate Committee on Governmental Affairs, we reiterated our concerns that IRS’ effort to modernize tax processing was jeopardized by persistent and pervasive management and technical weaknesses, and that ongoing efforts did not include milestones or provide enough evidence to conclude that weaknesses will soon be corrected. We also addressed analogous technical weaknesses in an electronic filing system project called Cyberfile which substantiated our concerns that IRS was continuing to risk millions of dollars in undisciplined systems development in fiscal year 1996. In addition, we identified physical security risks at the planned Cyberfile data center.
section_title: Treasury Department’s TSM Report Acknowledges Weaknesses and Describes Redirection Efforts
paragraphs: The Department of the Treasury, in its May 1996 report to the Senate and House Appropriations Committees, provides a candid assessment of TSM progress and future redirection, and a description of ongoing and planned actions intended to respond to our recommendations to correct management and technical weaknesses. It finds that despite some qualified success, IRS has not made progress on TSM as planned because systems development efforts have taken longer than expected, cost more than originally estimated, and delivered less functionality than originally envisioned. It concludes that significant changes are needed in IRS’ management approach, and that it is beyond the scope of IRS’ current ability to develop and integrate TSM without expanded use of external expertise.
The report notes that work has been done to rethink, scale back, and change the direction of TSM. Additional changes are still in progress with actions underway to restructure the management of TSM and expand the use of contractors. Agreeing that our July 1995 recommendations are valid, the report notes that more work has to be done to respond to our recommendations. It states that progress in IRS’ management and technical areas can only be achieved by institutionalizing improved practices and monitoring projects for conformance to mandated standards and practices.
The report does not address the basic problem of continuing to invest hundreds of millions of dollars in TSM before the requisite management and technical disciplines are in place. Neither does it address the risk inherent in shifting hundreds of millions of dollars to additional contractual efforts when the evidence is clear that IRS does not have the disciplined processes in place to manage all of its current contractual efforts (e.g., Cyberfile) effectively.
section_title: Fundamental Management and Technical Weaknesses Are Being Addressed, but None Have Been Fully Corrected
paragraphs: IRS has initiated a number of actions to address management and technical weaknesses that continue to impede successful systems modernization. However, ongoing efforts do not correct the weaknesses and do not provide enough evidence to determine when they will be corrected and what steps if any are being taken in the interim to mitigate the risks associated with ongoing TSM spending.
subsections: section_title: IRS Does Not Yet Have a Comprehensive Strategy to Maximize Electronic Filings
paragraphs: IRS has identified increasing electronic filings as critical to achieving its modernization vision. We noted that IRS did not have a comprehensive business strategy to reach or exceed its electronic filing goal, which was 80 million electronic filings by 2001. IRS’ estimates and projections for individual and business returns suggested that, by 2001, as few as 39 million returns may be submitted electronically, less than half of IRS’ goal and only about 17 percent of all returns expected to be filed.
We reported that IRS’ business strategy would not maximize electronic filings because it primarily targeted taxpayers who use a third party to prepare and/or transmit simple returns, are willing to pay a fee to file their returns electronically, and are expecting refunds. Focusing on this limited taxpaying population overlooked most taxpayers, including those who prepare their own tax returns using personal computers, have more complicated returns, owe tax balances, and/or are unwilling to pay a fee to a third party to file a return electronically.
We concluded that, without a strategy that also targets these taxpayers, IRS would not meet its electronic filing goals. In addition, if, in the future, taxpayers file more paper returns than IRS expects, added stress will be placed on IRS’ paper-based systems. Accordingly, we recommended that IRS refocus its electronic filing business strategy to target, through aggressive marketing and education, those sectors of the taxpaying population that can file electronically most cost-beneficially.
IRS agreed with this recommendation and said that it had convened a working group to develop a detailed, comprehensive strategy to broaden public access to electronic filing, while also providing more incentives for practitioners and the public to file electronically. It said that the strategy would include approaches for taxpayers who are unwilling to pay for tax preparer and transmitter services, who owe IRS for balances due, and/or who file complex tax returns. IRS said further that the strategy would address that segment of the taxpaying population that would prefer to file from home, using personal computers.
To date, IRS has performed an electronic filing marketing analysis at local levels; developed a marketing plan to promote electronic filing; consolidated 21 electronic filing initiatives into its Electronic Filing Strategies portfolio; and initiated a reengineering project with a goal to reduce paper tax return filings to 20 percent or less of the total volume by the year 2000. It plans to complete its electronic filing strategy in August 1996. These initiatives could result in future progress toward increasing electronic filings. However, our review found that these initiatives are not far enough along to determine whether they will culminate in a comprehensive strategy that identifies how IRS plans to target those sectors of the taxpaying population that can file electronically most cost-beneficially. It also is not clear how the reengineering project will impact the strategy or how these initiatives will impact TSM systems that are being developed.
section_title: IRS’ Strategic Information Management Practices Remain Ineffective
paragraphs: We reported that IRS did not have strategic information management practices in place. We found, for example, that, despite the billions of dollars at stake, information systems were not managed as investments. To overcome this, and provide the Congress with insight needed to assess IRS’ priorities and rationalization for TSM projects, we recommended that the IRS Commissioner take immediate action to implement a complete process for selecting, prioritizing, controlling, and evaluating the progress and performance of all major information systems investments, both new and ongoing, including explicit decision criteria, and using these criteria, to review all planned and ongoing systems investments by June 30, 1995.
In agreeing with these recommendations, IRS said it would take a number of actions to provide the underpinning it needs for strategic information management. IRS said, for example, that it was developing and implementing a process to select, prioritize, control, and evaluate information technology investments to achieve reengineered program missions.
Our assessment found that IRS has taken steps towards putting into place a process for managing its extensive investments in information systems. Following are examples of these steps.
IRS created the executive-level Investment Review Board, chaired by the Associate Commissioner for Modernization, for selecting, controlling and evaluating all of IRS’ information technology investments. IRS developed initial and revised sets of decision criteria used last summer and again in November 1995 as part of its Resource Allocation and Investment Review to make additional changes in information technology resource allocations for remaining fiscal year 1996 funds and planned fiscal year 1997 spending. This review included only TSM projects under development. It did not address operational systems, infrastructure, or management and technical support activities.
The Treasury Department created a Modernization Management Board to review and validate high-risk, high-cost TSM investments and to set policy and strategy for IRS modernization effort. IRS is considering the use of a “project readiness review” as an additional Investment Review Board control mechanism for gauging project readiness to proceed with spending. IRS developed the Business Case Handbook that includes decision criteria on costs, benefits, and risks. It is reassessing the business cases, which were developed on the TSM projects, using the handbook. Eleven cases are scheduled for completion in June 1996, and IRS plans to have the remaining cases completed by September 1996. Results are planned to be presented to the Investment Review Board to assist in making funding decisions for fiscal year 1997. IRS has developed the Investment Evaluation Review Handbook designed to assess projected costs and benefits against actual results. The handbook has been used on four TSM projects and five additional reviews are scheduled to be completed within the next year. The completed reviews contain explicit descriptions of problems encountered in developing these systems. The reviews make specific recommendations for management and technical process changes to improve future results. Specific recommendations pertain to strengthening project direction and decision-making. Many reflect concerns that we have raised in past reviews. The investment evaluation reviews were presented to the Investment Review Board and disseminated to other IRS managers. IRS is defining roles, responsibilities, and processes for incorporating Investment Evaluation Review recommendations at the project and process levels.
These are positive steps and indicate a willingness to address many of the weaknesses raised in our past reports and testimonies. But, as noted in Treasury’s report on TSM, the investment process is not yet complete. According to Treasury, it is missing (1) specific operating procedures, (2) defined reporting relationships between different management boards and committees, and (3) updated business cases for major TSM technology investments.
These concerns coincide with two central criticisms we have repeatedly made about TSM. Because of the sheer size, scope, and complexity of TSM, it is imperative that IRS institutionalize a repeatable process for selecting, controlling, and evaluating its technology investments, and that it make informed investment decisions based on reliable qualitative and quantitative assessments of costs, benefits, and risks. Although IRS is planning and in the initial stages of implementing parts of such a process, a complete, fully-integrated process does not yet exist. Specifically, IRS has not provided us evidence to justify its claims that its decisions were supported by acceptable data on project costs, benefits, and risks. For example:
Our review found no evidence to suggest that IRS established minimal data requirements for the decisions made as part of the TSM Resource Allocation and Investment Review or the rescope process in December 1995. For example, because IRS lacks the basic capabilities for disciplined software development, it cannot convincingly estimate systems development costs, schedule, or performance. Subsequent to its rescope analysis, IRS developed minimal data quality requirements for cost-benefit and risk studies, proposed return on investment calculations, and return on investment thresholds, or comparisons of expected performance improvements with results to date. However, to date, few, if any projects have met these criteria. In deciding whether to accelerate, delay, or cancel specific TSM projects, IRS did not use validated data on actual versus projected costs, benefits, or risks as set forth by the Office of Management and Budget (OMB). Instead, IRS continues to make its decisions based on spending whatever budgeted funding ceiling amounts can be obtained through its annual budget and appropriations cycles. As a result, IRS cannot convincingly justify its TSM spending decisions.
All projects (i.e., proposed projects, projects under development, operational systems, infrastructure, and management and technical support activities) were not included in a single systems investment portfolio. Instead, only TSM projects under development were ranked. As a result, there is no compelling rationale for determining how much to invest in these projects compared to other projects, such as operational systems, infrastructure, etc.
There is no defined process with prescribed roles and responsibilities to ensure that the results of investment evaluation reviews are being used to (1) modify project direction and funding when appropriate and (2) assess and improve existing investment selection and control processes and procedures. As a result, there is no evidence that changes are occurring based on the valuable lessons learned as in the recently completed post implementation review of the Service Center Recognition/Image Processing System. For example, IRS found that because system requirements were not adequately defined or documented, the system could not be quantifiably tested properly which adversely affected the implementation of the system. Moreover, with only four investment evaluation reviews completed to date and five planned for the upcoming year, this represents only a small fraction of the total IRS annual investment in TSM. More must be done to confirm actual results achieved from TSM expenditures.
section_title: Reengineering Efforts Not Linked to Modernization
paragraphs: We noted in our July 1995 report that IRS’ reengineering efforts were not linked to its systems development efforts. As shown in our work with leading organizations, information system development projects that are not driven by a critical reexamination and redesign of business processes achieve only a fraction of their potential to improve performance, reduce costs, and enhance quality.
Since our July report, IRS’ reengineering efforts have undergone a redirection. Three reengineering projects—processing returns, responding to taxpayers, and enforcement actions—were halted because IRS decided to focus instead on an enterprise-level view of reengineering.
Its new effort, entitled Tax Settlement Reengineering, was begun in March 1996 and involves a comprehensive review of all the major processes and activities that enable taxpayers to settle their tax obligations, from educational activities through final settlement of accounts. The reengineering project team, working with IRS’ Executive Committee, has identified 16 major processes involved in tax settlement and is about to begin reengineering four of them. High-level designs of the new processes are scheduled to be defined by September 30, 1996, with work on detailed designs to start early in fiscal year 1997, if approved by the Executive Committee. Reengineering efforts on as many as eight other tax settlement processes could be underway by the end of fiscal year 1997.
Although this effort could have substantial impact, IRS still faces the same problem we reported on a year ago. Reengineering lags well behind the development of TSM projects, whereas it should be ahead of it—defining and directing the technology investments needed to support new, more efficient business processes. Until the reengineering effort is mature enough to drive TSM projects, there is no assurance that ongoing systems development efforts will support IRS’ future business needs and objectives.
The reengineering team believes that by September 1996 they will have a general idea of how the first four tax settlement reengineering projects may impact current system development efforts. If additional reengineering projects are started as planned in 1997, it could be another year or more before most of the information and systems requirements stemming from these projects are defined. Meanwhile, investment continues in many TSM projects that may not support the requirements resulting from these reengineering efforts.
IRS acknowledges that integration of reengineering and TSM must occur, and has assigned responsibility for it to the Associate Commissioner for Modernization, but has not yet specified how or when the requisite integration will occur.
section_title: Software Development Activities Are Inconsistent and Poorly Controlled
paragraphs: We reported that unless IRS improves its software development capability, it is unlikely to build TSM timely or economically, and systems are unlikely to perform as intended. To assess its software capability, in September 1993, IRS rated itself using the Software Engineering Institute’s CMM. IRS placed its software development capability at the lowest level, described as ad hoc and sometimes chaotic and indicating significant weaknesses in its software development capability. Our review confirmed that IRS’ software development capability was immature and was weak in key process areas. For instance, a disciplined process to manage system requirements was not being applied to TSM systems, a software tool for planning and tracking development projects was not software quality assurance functions were not well defined or consistently systems and acceptance testing were neither well defined nor required, software configuration management was incomplete.
To address IRS’ software development weaknesses and upgrade IRS’ software development capabilities, we recommended that the IRS Commissioner immediately require that all future contractors who develop software for the agency have a software development capability rating of at least CMM Level 2, and before December 31, 1995, define, implement, and enforce a consistent set of requirements management procedures for all TSM projects that goes beyond IRS’ current request for information services process, and for software quality assurance, software configuration management, and project planning and tracking; and define and implement a set of software development metrics to measure software attributes related to business goals.
IRS agreed with these recommendations and said that it was committed to developing consistent procedures addressing requirements management, software quality assurance, software configuration management, and project planning and tracking. It also said that it was developing a comprehensive measurement plan to link process outputs to external requirements, corporate goals, and recognized industry standards.
Specifically regarding the first recommendation, IRS has (1) developed standard wording for use in new and existing contracts that have a significant software development component, requiring that all software development be done by an organization that is at CMM Level 2, (2) developed a plan for achieving CMM Level 2 capability on all of its contracts, and (3) started to implement a plan to monitor contractors’ capabilities, which may include the use of CMM-based software capability evaluations. The Department of the Treasury report also noted that a schedule for conducting software capability evaluations was developed. However, we found that IRS does not yet have the disciplined processes in place to ensure that all contractors are performing at CMM Level 2. For example, contractors developing the Cyberfile electronic filing system were not using CMM Level 2 processes, subsequent to our July 1995 recommendation. Further, no schedule for conducting software capability evaluations has yet been developed.
With respect to the second recommendation, IRS is updating its systems life cycle (SLC) methodology. The SLC is planned to have details for systems engineering and software development processes, including all CMM key process areas. IRS has updated its systems engineering process to include guidance for defining and analyzing systems requirements and for preparing work packages. Furthermore, IRS has drafted handbooks providing guidance to audit and verify developmental processes. In addition, IRS has developed a configuration management plan template, updated its requirements management request for information servicesdocument, and developed and implemented a requirements management course. The Department of the Treasury also reported that IRS is testing the SLC on two TSM efforts, Integrated Case Processing (ICP) and Corporate Accounts Processing System (CAPS). IRS also has a CMM process improvement plan and work is being done across various IRS organizations to define processes to meet CMM Level 2. Finally, IRS is assessing its capabilities to manage contractors using the CMM goals.
However, the procedures for requirements management, software quality assurance, software configuration management, and project planning and tracking are still not complete. A software development life cycle implementation project, which is to include these procedures, is not scheduled for completion until September 30, 1996. In addition, software quality assurance and configuration management plans for two ICP projects were not being used, and the groups developing software for CAPS do not have a software configuration management plan or a schedule for its development. Furthermore, ICP and CAPS development is continuing without the guidelines and procedures for other process areas (e.g., requirements management, project planning, and project tracking and oversight) required by CMM Level 2.
Regarding the third recommendation, IRS has a three-phase process to (1) identify data sources for metrics, (2) define metrics to be used, and (3) implement the metrics. A partial set of metrics is currently being identified. Initial use of these metrics—populated with real data and in a preliminary format—is scheduled for use on a set of identified projects beginning on June 30, 1996. Data sources for these metrics have been identified and weaknesses (such as difficulties in retrieving the data and inconsistencies in the data) are being documented to provide feedback to various systems’ owners.
However, this initial set of metrics is incomplete. It focuses on areas such as time reporting, project sizing, and defect tracing and analysis, but does not include measures for determining customer satisfaction and cost estimation. Such measures are needed to adequately track the needed functionality with associated costs throughout systems development. Further, there is no schedule for completing the definition of metrics or for institutionalizing the processes needed to ensure their use. Finally, there is no mechanism in place to correct identified data and data collection weaknesses.
In summary, although IRS has begun to act on our recommendations, these actions are not yet complete or institutionalized, and, as a result, systems are still being developed without the disciplined practices and metrics needed to give management assurance that they will perform as intended.
section_title: Systems Architectures, Integration, and Testing Are Incomplete
paragraphs: We reported that IRS’ systems architectures, integration planning, and system testing and test planning were incomplete.
To address IRS’ technical infrastructure weaknesses, we recommended that the IRS Commissioner before December 31, 1995, complete an integrated systems architecture, including security, telecommunications, network management, and data management; institutionalize formal configuration management for all newly approved projects and upgrades and develop a plan to bring ongoing projects under formal configuration management; develop security concept of operations, disaster recovery, and contingency plans for the modernization vision and ensure that these requirements are addressed when developing information system projects; develop a testing and evaluation master plan for the modernization; establish an integration testing and control facility; and complete the modernization integration plan and ensure that projects are monitored for compliance with modernization architectures.
IRS agreed with these recommendations and said that it was identifying the necessary actions to define and enforce systems development standards and architectures agencywide. IRS’ current efforts in this area follow.
In April 1996, IRS completed a descriptive overview of its integrated three-tier, distributed systems architecture to provide management with a high-level view of TSM’s infrastructure and supporting systems. IRS has tasked the integration support contractor to develop the data and security architectures. IRS has adopted an accepted industry standard for configuration management. It developed and distributed its Configuration Management Plan template, which identifies the elements needed when constructing a configuration management plan. In April 1996, enterprisewide configuration management policies and procedures were established. IRS also plans to obtain contractor support to develop, implement, and maintain a vigorous configuration management program. IRS has prepared a security concept of operations and a disaster recovery and contingency plan. IRS has developed a test and evaluation master plan for TSM. IRS plans to develop implementation and enforcement policies for the plan. IRS has established an interim integration testing and control facility, which is currently being used to test new software releases. It is also planning a permanent integration testing and control facility, scheduled to be completed by December 1996. IRS has completed drafts of its TSM Release Definition Document, which is planned to provide definitions for new versions of TSM software from 1997 to 1999, and Modernization Integration Plan, which is planned to define IRS’ process for integrating current and future TSM initiatives. centers need to take to absorb the workload of a center that suffers a disaster.
The test and evaluation master plan provides the guidance needed to ensure sufficient developmental and operational testing of TSM. However, it does not describe what security testing should be performed, or how these tests should be conducted. Further, it does not specify the responsibilities and processes for documenting, monitoring, and correcting testing and integration errors. IRS is still working on plans for its integration testing and control facility. In the interim, it has established a temporary facility which is being used for limited testing. The permanent facility is not currently being planned to simulate the complete production environment, and will not, for example, include mainframe computers. Instead, IRS plans to continue to test mainframe computer software and systems which interface with the mainframes in its production environment. To ensure that IRS does not put operations and service to taxpayers at risk, IRS should prepare a thorough assessment of its solution, including an analysis of alternative testing approaches and their costs, benefits, and risks. IRS’ draft TSM Release Definition Document and draft Modernization Integration Plan (1) do not reflect TSM rescoping and the information systems reorganization under the Associate Commissioner, (2) do not provide clear and concise links to other key documents (e.g., its integrated systems architecture, business master plan, concept of operations, and budget), and (3) assume that IRS has critical processes in place that are not implemented (e.g., effective quality assurance and disciplined configuration management).
In summary, although IRS has taken actions to prepare a systems architecture and improve its integration and system testing and test planning, these efforts are not yet complete or institutionalized, and, as a result, TSM systems continue to be developed without the detailed architectures and discipline needed to ensure success.
section_title: No Single IRS Entity Controls All Information Systems Efforts
paragraphs: We reported that IRS had not established an effective organizational structure to consistently manage and control systems modernization organizationwide. The accountability and responsibility for IRS’ systems development was spread among IRS’ Modernization Executive, Chief Information Officer, and research and development division. To help address this concern, in May 1995, the Modernization Executive was named Associate Commissioner. The Associate Commissioner was to manage and control systems development efforts previously conducted by the Modernization Executive and the Chief Information Officer.
In September 1995, the Associate Commissioner for Modernization assumed responsibility for the formulation, allocation, and management of all information systems resources for both TSM and non-TSM expenditures. In February 1996, IRS issued a Memorandum of Understanding providing guidance for initiating and conducting technology research and for transitioning technology research initiatives into system development projects.
It is important that IRS maintain an organizationwide focus to manage and control all new modernization systems and all upgrades and replacements of operational systems throughout IRS. To do so, we recommended that the IRS Commissioner give the Associate Commissioner management and control responsibility for all systems development activities, including those of IRS’ research and development division.
Steps are being taken by the Associate Commissioner to establish effective management and control of systems development activities throughout IRS. For example, its SLC methodology is required for information systems development, and information technology entities throughout the agency have been directed to submit documentation on all information technology projects for review. However, there is no defined and effective mechanism for enforcing the standards or ensuring that organizational entities cannot conduct systems development activities outside the control of the Associate Commissioner. Further, no timeframes have been established for defining and implementing such control mechanisms. As a result, systems development conducted by the research and development division has now been redefined as technology research, keeping it from the control of the Associate Commissioner.
In summary, although improvements have been made in consolidating management control over systems development, the Associate Commissioner still does not yet have control over all IRS’ systems development activities.
section_title: Plans Must Be Defined and Capabilities Strengthened Before Obtaining Additional Contractual Support
paragraphs: IRS plans to increase its reliance on the private sector by (1) preparing an acquisition plan and statement of work to conduct an expedited competitive selection for a prime development and integration contractor; (2) transferring responsibility for systems engineering, design, prototyping, and integration for core elements of TSM to its integration support contractor; and (3) making greater use of software development contractors, including those available under the Treasury Information Processing Support Services (TIPSS), to develop and deliver major elements of production TSM systems. By increasing its reliance on contractors, IRS expects to improve the accountability for and probability of TSM success.
IRS plans to increase the use of private-sector integration and development expertise by expanding the use of contractors to support TSM. It outlined a three-track approach for transitioning over a period of 2 years to the use of a prime contractor that would have, according to IRS, overall authority and responsibility for the development, delivery, and deployment of modernized information systems.
To facilitate this strategy, IRS reported it would consolidate the management of all TSM resources, including key TSM contractors, in its Government Program Management Office (GPMO). Under the direct control of the Chief Information Officer, GPMO will be delegated authority for the management and control of the IRS staff and contractors that plan, design, develop, test, and implement TSM components. IRS plans to have GPMO fully staffed and operational by October 1, 1996. IRS representatives told us the agency was currently developing a detailed contract management plan and a statement of work for acquiring its prime contractor, and believed it could award a contract in about 2 years.
IRS’ approach to expanding the use of contractors to build TSM is still in the early planning stages. Because of this, IRS was unable to provide us with formal plans, charters, schedules, or the definitions of shared responsibilities between GPMO and the existing program and project management staff.
At this point, it is unclear what these IRS planned actions entail, or how they will work. For example, IRS has not specified how and when it plans to transfer its development activities to contractors, and to what extent contractors could be held responsible for existing problems in these government-initiated systems. This is particularly important because if IRS continues as planned, the principal TSM systems will be in development and/or deployed before IRS plans to select a prime contractor in about 2 years. Moreover, it is not clear how the prime contractor would direct potential competitors that are already under contract with IRS. Without further explanation of and a schedule for transitioning specific responsibilities from IRS to contractors, we cannot fully understand or assess IRS’ plans.
Further, plans to use additional contractors will succeed if, and only if, IRS has the in-house capabilities to manage these contractors effectively. In this regard, there is clear evidence that IRS’ capability to manage contractors has weaknesses. In August 1995, IRS acquired the services of the Department of Commerce’s National Technical Information Service (NTIS) to act as IRS’ prime contractor in developing Cyberfile. However, Cyberfile was not developed using disciplined management and technical practices. As a result, this project exhibited many of the same problems we have repeatedly identified in other TSM systems, and, after providing $17 million to NTIS, it was not ready for planned testing during the 1996 tax filing season. Similarly, IRS contracted in 1994 to build the Document Processing System. After expending over a quarter of a billion dollars on the project, IRS has now suspended the effort and is reexamining some of its basic requirements, including which and how many forms should be processed, and which and how much data should be read from the documents.
We recently initiated an assignment to evaluate in detail IRS’ software acquisition capabilities using the Software Engineering Institute’s Software Acquisition CMM. This assignment is scheduled to be completed later this year. It is clear that unless IRS has mature, disciplined processes for acquiring software systems through contractors, it will be no more successful in buying software than it has been in building software.
section_title: Conclusions
paragraphs: IRS has initiated a number of actions and is making some progress in addressing our recommendations to correct its pervasive management and technical weaknesses. However, none of these actions, either individually or in the aggregate, fully satisfy any of our July 1995 recommendations and it is not clear when these actions will result in disciplined systems development. As a result, IRS continues to spend hundreds of millions of dollars on TSM through fiscal year 1997, while fundamental weaknesses jeopardize the investment.
Recognizing its internal weaknesses, IRS plans to use a prime contractor and increase use of software development contractors to develop TSM.
However, in this area, its plans and schedules are not well defined, and, therefore, cannot be completely understood or assessed. Further, as the experience with Cyberfile and the Document Processing System projects makes clear, IRS does not have the mature processes needed to acquire software and manage contractors effectively.
section_title: Matters for Congressional Consideration
paragraphs: Because IRS still does not have (1) effective strategic information management practices needed to manage TSM as an investment, (2) mature and disciplined software development processes needed to assure that systems built will perform as intended, (3) a completed systems architecture that is detailed enough to guide and control systems development, and (4) a schedule for accomplishing any of the above, the Congress could consider limiting TSM spending to only cost-effective modernization efforts that (1) support ongoing operations and maintenance, (2) correct IRS’ pervasive management and technical weaknesses, (3) are small, represent low technical risk, and can be delivered in a relatively short time frame, and (4) involve deploying already developed systems, only if these systems have been fully tested, are not premature given the lack of a completed architecture, and produce a proven, verifiable business value. As the Congress gains confidence in IRS’ ability to successfully develop these smaller, cheaper, quicker projects, it could consider approving larger, more complex, more expensive projects in future years.
Because IRS does not manage all of its current contractual efforts effectively, and because its plans to use a “prime” contractor and transition much of its systems development to additional contractors are not well defined, the Congress could consider requiring that IRS institute disciplined systems acquisitions processes and develop detailed plans and schedules before permitting IRS to increase its reliance on contractors.
section_title: Agency Comments and Our Evaluation
paragraphs: On June 6, 1996, we met with Treasury and IRS officials to discuss a draft of this report and we incorporated their comments as appropriate in finalizing it. In addition, on June 6, 1996, we received written comments from Treasury.
In his letter, the Deputy Secretary of the Treasury reiterates Treasury’s commitment to significantly increased oversight of TSM and to making a sharp turn in the way TSM is managed. He also makes clear Treasury’s and IRS’ understanding that additional improvements are necessary to fully correct the management and technical weaknesses delineated in our report. The Deputy Secretary of the Treasury also says that he is reducing the fiscal year 1997 budget request for TSM from $850 million to $664 million and will need to ensure, at all times, solid stewardship for the dollars appropriated and clear accountability for the investments undertaken.
Achieving sound management for the TSM program will require that IRS (1) institutionalize effective strategic information management practices, (2) institutionalize mature and disciplined software development processes, and (3) complete systems, data, and security architectures and use them to guide and control systems development, before making major investments in TSM systems development. Until these disciplined processes are in place and the requisite architectures completed, the Congress could consider limiting IRS TSM spending to only cost-effective modernization efforts that meet the criteria outlined in our Matters for Congressional Consideration.
We are sending copies of this report to the Chairmen and the Ranking Minority Members of (1) the Senate and House Committees on the Budget, (2) the Subcommittee on Taxation and IRS Oversight, Senate Committee on Finance, (3) the Senate Committee on Governmental Affairs, (4) the Subcommittee on Oversight, House Committee on Ways and Means, and (5) the House Committee on Government Reform and Oversight. We are also sending copies to the Secretary of the Treasury, Commissioner of the Internal Revenue Service, and Director of the Office of Management and Budget. Copies will be available to others upon request.
This work was performed under the direction of Dr. Rona B. Stillman, Chief Scientist for Computers and Telecommunications, who can be reached at (202) 512-6412. Other major contributors are listed in appendix II.
section_title: Comments From the Department of the Treasury
section_title: Major Contributors to This Report
subsections: section_title: Accounting and Information Management Division, Washington, D.C.
section_title: General Government Division, Washington, D.C.
paragraphs: Sherrie Russ, Senior Evaluator Christopher E. Hess, Evaluator The first copy of each GAO report and testimony is free. Additional copies are $2 each. Orders should be sent to the following address, accompanied by a check or money order made out to the Superintendent of Documents, when necessary. VISA and MasterCard credit cards are accepted, also. Orders for 100 or more copies to be mailed to a single address are discounted 25 percent.
U.S. General Accounting Office P.O. Box 6015 Gaithersburg, MD 20884-6015 Room 1100 700 4th St. NW (corner of 4th and G Sts. NW) U.S. General Accounting Office Washington, DC Orders may also be placed by calling (202) 512-6000  or by using fax number (301) 258-4066, or TDD (301) 413-0006.
Each day, GAO issues a list of newly available reports and testimony.  To receive facsimile copies of the daily list or any list from the past 30 days, please call (202) 512-6000 using a touchtone phone.  A recorded menu will provide information on how to obtain these lists.