Tax Systems Modernization: Management and Technical Weaknesses Must Be Corrected If Modernization Is to Succeed
Released: 1995-07-26T00:00:00

Letter

Introduction

The receipt, processing, and retrieval of vast quantities of paper forms and documents is one of IRS’s most critical problems. IRS annually receives over 200 million tax returns with multiple attachments, about 1 billion information documents (for example, W2s and 1099s), and several hundred million pieces of taxpayer correspondence.

To process this enormous volume of paperwork, IRS uses labor-intensive processes and systems to (1) convert data from tax returns into machine usable form, (2) maintain taxpayer accounts, including current and historical data, (3) ensure refunds are prompt, and (4) prepare bills for tax payments due. Retrieving paper forms and documents involves over 1.2 billion tax returns stored in over 1 million square feet of space.

Also, IRS collects most of the government’s revenue, currently over $1.25 trillion annually, and it employs over 113,000 people, more than any other civilian agency. IRS is headquartered in Washington, D.C., and has 7 regional offices, 63 district offices, 10 service centers, and 2 computer centers.

Background

Processing Tax Return Information

Upon receipt at IRS’s service centers, paper-based tax returns and related supporting and information documents are manually extracted from envelopes, sorted, batched, coded, and transcribed into electronic format. The service centers send electronically formatted data to IRS’s main computer center in Martinsburg, West Virginia. IRS stores nearly all the paper supplied by taxpayers as part of, or in support of, their tax filings.

Tax return processing at IRS service centers was designed in the late 1950s. Today, nearly 4 decades later, IRS still processes tax return data using the processes instituted when automated systems were first installed in the service centers.

In today’s technological climate, taxpayers have come to expect faster, better, more convenient service in virtually every facet of their lives. To meet these expectations, IRS’s outdated tax processes and systems are being used to electronically capture and provide more and more information. At the same time, the number of tax-related documents is greatly expanding.

Tax Systems Modernization

Between the late 1960s and the early 1980s, IRS began several efforts to modernize its operations. These efforts did not succeed, and on numerous occasions the Congress expressed concern about the cost of the redesign efforts, the inadequacy of security controls over taxpayer information, the lack of clear management responsibility for the programs, and the paucity of technical and managerial expertise.

In late 1986, IRS produced plans for a new modernization effort, known today as Tax Systems Modernization (TSM). IRS estimates that TSM could cost between $8 billion and $10 billion through 2001. Through fiscal year 1995, IRS will have spent or obligated $2.5 billion for TSM, which comprises 36 systems development projects. About $1.1 billion more has been requested for fiscal year 1996.

IRS has developed a business vision to guide its modernization efforts. This vision calls for a work environment that is virtually paper-free, where taxpayer account updates are rapid and taxpayer information is readily available to IRS employees for purposes such as customer service and compliance activities.

IRS’s overall redesign of its tax processing system is key to achieving this vision. An important component of the redesign is maximizing the receipt of electronic information to reduce the receipt of paper documents. IRS plans, for example, to expand the electronic receipt of tax returns.

However, IRS believes the requirement to process large volumes of paper documents will exist for the foreseeable future. As a result, IRS is designing the Document Processing System to scan paper documents and electronically capture data for subsequent processing and retrieval at workstations. This system will require staff using personal computers to correct and add data that the system cannot accurately capture from paper documents. Like its electronic filing system counterparts, the Document Processing System is to capture 100 percent of the numeric data submitted on tax returns, compared to about 40 percent captured from paper returns today.

Previously Reported TSM Problems

Throughout the modernization, we have reported on critical issues related to the need to build an effective organization structure for managing technology; problems in developing specific TSM systems and the reliability of reported weak internal, computer security, and fraud controls; and antiquated systems that were not designed to provide the meaningful and reliable financial information needed to effectively manage and report on IRS’s operations.

Because of problems such as these, in February 1995, we designated TSM a high-risk systems modernization effort. In general, these major efforts experience cost over-runs, are prone to delays, and often fail to meet intended mission objectives. Appendix II is a list of our prior reports and testimonies pertinent to TSM.

Objective, Scope, and Methodology

Our objective was to review the business and technical practices IRS has established to develop, manage, and operate its information systems and, in particular, the TSM initiative. We examined IRS’s business strategy for reducing paper tax return submissions, strategic information management processes, software development capability, systems development accountability and responsibility.

To review IRS’s business strategy for reducing paper tax return submissions, we interviewed IRS officials who have responsibility for submission processing and electronic filing. We analyzed various task force studies on electronic filing and summaries of issues compiled by an IRS task team charged with promoting electronic filing.

In addition, we examined IRS internal audit reports on the performance and development of systems designed to handle paper returns, reports of problems from the service centers responsible for processing tax returns, and a risk assessment and critical design review of operational and developmental systems. Further, we reviewed project plans and technical charters for paper processing systems, and we discussed systems requirements and performance test results with the contractor developing the Document Processing System.

To assess IRS’s strategic information management processes, we interviewed IRS officials who have responsibility for systems development. We also analyzed IRS planning documents, including IRS’s Business Master Plan, Future Concept of Operations, and Integrated Transition Plan and Schedule. We obtained and analyzed IRS documentation and task force studies related to (1) planning and managing information technology, (2) analyzing systems development costs and benefits, (3) reengineering business processes, and (4) training staff in the use of new information technology.

In analyzing IRS’s strategic information management practices, we drew heavily from our research on the best practices of private and public sector organizations that have been successful in improving their performance through strategic information management and technology. These fundamental best practices are discussed in our report, Executive Guide: Improving Mission Performance Through Strategic Information Management and Technology (GAO/AIMD-94-115, May 1994), and our Strategic Information Management (SIM) Self-Assessment Toolkit (GAO/Version 1.0, October 28, 1994, exposure draft).

To evaluate IRS’s software development capability, we validated IRS’s August 1993 assessment of its software development maturity based on the Capability Maturity Model (CMM) developed in 1984 by the Software Engineering Institute at Carnegie Mellon University. CMM establishes standards in key software development processing areas and provides a framework to evaluate a software organization’s capability to consistently and predictably produce high-quality products. We discussed with IRS software development officials IRS’s CMM rating and actions initiated to improve it.

We also identified and assessed IRS’s initiatives to improve software development capability in key process areas, including (1) requirements management, (2) project planning, tracking, and oversight, and (3) configuration management. In another key process area, software quality assurance, we examined, in particular, IRS’s use of metrics to control software development projects.

To assess IRS’s technical infrastructures, we discussed security and data standards with systems architects and technical specialists. In addition, we obtained and analyzed integrated systems architecture documents; systems development documents for security and data standards; and project plans, quality measurement plans, and technical charters for all TSM projects.

To assess accountability and responsibility for developing systems, we identified the IRS organizational components involved in developing and operating information systems. We discussed with IRS’s Modernization Executive, Chief Information Officer, and research and development division officials their respective systems development roles, responsibilities, and accountability.

We performed our work at IRS headquarters in Washington, D.C., and at facilities in Cincinnati, Ohio, and Nashville, Tennessee. On April 28, 1995, we briefed the IRS Commissioner and other senior IRS executives on the results of our review and made recommendations to them for overcoming the management and technical problems impeding successful systems modernization efforts.

Our work was performed between February 1995 and June 1995, in accordance with generally accepted government auditing standards. IRS provided written comments on a draft of this report, which are included as appendix I.

A Strategy to Maximize Electronic Filing Is Key to Modernization

IRS is currently drowning in paper—a serious problem IRS can mitigate only through electronic tax filings. But IRS will not achieve the full benefits that electronic filing can provide because it does not have a comprehensive business strategy to reach or exceed its electronic filing goal, which is 80 million electronic filings by 2001. Today, IRS’s estimates and projections for individual and business returns suggest that, by 2001, as few as 39 million returns may be submitted electronically, less than half of IRS’s goal.

Maximizing electronic filings is important because tax returns filed electronically do not have to move through IRS’s labor-intensive operations. Paper filings have to be opened, sorted, reviewed, transcribed, shipped and stored, and then physically retrieved if IRS employees later need data on the returns that are not transcribed. IRS recognizes that increasing the number of electronic filings is essential to both improve its tax return processing and advance toward the virtually paperless environment envisioned by IRS under TSM.

Creating a paperless environment, though, will involve making significant changes to improve IRS’s information management and will require new processes and new ways of doing business. Private and public sector organizations that have successfully improved their performance have found that to move away from the status quo, an organization must recognize opportunities to change and improve its fundamental business processes. Without well-conceived business strategies to capitalize on opportunities, meaningful change may be slow, the quality of service may not improve, and modernization may be impossible.

Consequently, one of IRS’s most pressing modernization issues is the efficient processing of vast quantities of information received on tax returns, which in 1994, amounted to about 205 million returns. In 1995, IRS expects total tax returns from individuals and businesses to increase by 2 million, and by 2001, to reach 224 million filings.

To help process its avalanche of paperwork more efficiently, in 1990, IRS introduced nationwide electronic filing to selected groups of taxpayers as a means of using modern technology to streamline its business processes. Looking to the future, IRS set a goal to receive 80 million tax filings electronically by 2001. IRS based this goal, which accounts for about 35.7 percent of all tax filings expected in 2001, on a projection of electronic filing of 70 million individual returns and 10 million business returns.

In working toward this goal, in 1994, about 16 million tax returns, or 7.8 percent of all returns, were filed electronically, with about 50 percent of these being 1040A forms. In 1995, IRS expects that electronic filings will decrease to about 15 million, or 7.2 percent of all tax returns.

On the basis of the current rate of electronic filings for individuals, IRS now estimates that in 2001 only about 29 million electronic returns will by filed by individuals. Combined with the projected 10 million electronic filings from businesses, IRS may receive only 39 million electronic returns in 2001. This is only about 17.4 percent of the 224 million tax returns anticipated in 2001, less than half of IRS’s goal. Table 2.1 summarizes IRS’s electronic filing activity for 1994 and projections for the future.

IRS’s current business strategy focuses primarily on promoting faster refunds to clients of businesses that prepare and electronically transmit tax returns. Tax return preparers and transmitters do not pay a fee to IRS for electronic filings, but they charge a fee to taxpayers. Consequently, IRS’s business strategy for promoting electronic filing is directed primarily at taxpayers who file using third parties, are willing to pay to file electronically, file simple tax returns, and are due refunds.

IRS has no comprehensive business strategy for promoting the benefits of electronic filing to other taxpayers. In doing this, IRS should consider all segments of the taxpaying population, including those who (1) are unwilling to pay for tax preparer and transmitter services, (2) owe IRS for balances due, and (3) file complex tax returns. These taxpayers represent considerable potential for making substantially greater use of electronic filing.

Moreover, IRS is not taking advantage of opportunities afforded by personal computers to increase electronic filings. In recent years, these computers have become a common fixture in many households. In this regard, when personal computers are used to prepare tax returns, taxpayers who are not willing to pay commercial transmitting fees must print their electronically produced returns on paper and mail them to IRS to be manually processed. This results in the redundant, counterproductive conversion of the same data by both taxpayers and IRS: taxpayers convert electronic data to paper returns, IRS then laboriously converts information on the paper returns back to electronic data.

Unless IRS attracts all potential electronic filers, it will never achieve its vision of virtually paperless processing and will be forced to process increasingly large workloads of paper tax returns. Further, IRS’s paper processing systems are not planned to accommodate the large volume of paper returns that will result if taxpayers file fewer returns electronically. For example, IRS is designing the Document Processing System for use at five service centers based on the assumption that, by 2001, at least 61 million of the 224 million returns will be filed electronically, that is, 163 million paper returns will be processed through the Document Processing System.

As table 2.1 shows, by 2001, since only 39 million tax returns may be filed electronically, 185 million taxpayers could submit paper returns, or about 22 million more returns than IRS is designing the Document Processing System to process. Thus, IRS’s most recent estimates on individual filings for 2001 indicate that IRS may fall far short of its electronic filing goal, which will result in an increasing struggle to process paper filings.

Recommendation

To better achieve its virtually paperless processing environment, we recommend that IRS refocus its electronic filing business strategy to target, through aggressive marketing and education, those sectors of the taxpaying population that can file electronically most cost beneficially.

Agency Comments and Our Evaluation

IRS agreed with our recommendation regarding its electronic filing strategy. IRS said it has convened a working group, chaired by the electronic filing executive, to develop a detailed, comprehensive strategy to broaden public access to electronic filing, while also providing more incentives for practitioners and the public to file electronically.

IRS said the strategy will include approaches for taxpayers who are unwilling to pay for tax preparer and transmitter services, who owe IRS for balances due, and/or who file complex tax returns. IRS said further that the strategy will also address that segment of the taxpaying population that would prefer to file from home by personal computers.

We believe that, by developing a more comprehensive electronic filing strategy, IRS will help to maximize the benefits possible through greater use by taxpayers of electronic filing. These benefits are central to more efficiently processing the vast quantities of information IRS receives on tax returns and, thus, to achieve the virtually paperless tax processing environment IRS hopes to attain through modernization.

IRS Is Working to Overcome Ineffective Strategic Information Management Practices

IRS is not yet effectively using a strategic information management process to plan, build, and operate its information systems. TSM has been underway for almost a decade and will require years of further development effort and substantial human and financial resources. IRS, however, does not yet have in place an effective process for selecting, prioritizing, controlling, and evaluating the progress and performance of major information systems investments.

A sound strategic information management process involves several fundamental practices: (1) applying strategic planning, (2) managing information technology as investments, (3) analyzing costs and benefits and measuring performance, (4) using business process analysis, and (5) upgrading skills and training. This process focuses on results and emphasizes simplifying and redesigning complex mission processes, which is essential to meeting mission goals and satisfying customers’ needs.

IRS recognizes the importance to TSM’s success of implementing a sound strategic information management process and has assessed its strategic information management using GAO’s strategic information management self-assessment toolkit. IRS’s self-assessment identified improvements for better managing information systems. We too found serious shortcomings that underscore the urgency of IRS bolstering the strategic information management process it has begun. We also identified IRS efforts to upgrade skills and training.

Strategic Planning Is Incomplete and Inconsistent

Although IRS has developed several types of plans for carrying out its current and future operations, these plans are neither complete nor consistent. Moreover, IRS’s various planning documents are not linked to each other or to TSM budget requests. Even though TSM has been underway for 10 years, complete, clear, and concise planning for TSM and its multibillion dollar investment is not evident. As a result, it is difficult for IRS to identify and effectively focus on completing priority aspects of TSM.

Public and private sector organizations that have been successful in developing major systems have found that, to be successful, once the organization has made a serious commitment to change its management of information and technology, it is paramount to adopt a strategic planning approach. Their experience is that strategic business and information system plans must have a tight link to mission goals and must be predicated on satisfying explicit, high-priority customer needs. This orientation helps to ensure that information technology projects are delivered on time and within budget and that they produce meaningful improvements in cost, quality, or timeliness of service.

We identified several different efforts by IRS to prepare plans to delineate a vision for the future and actions required to realize that vision. These planning documents include the Business Master Plan, which reflects the business priorities set by IRS’s top executives and links IRS’s strategic objectives and business vision with the tactical actions needed to implement them; the IRS Future Concept of Operations, which articulates IRS’s future business vision so that the Congress, IRS employees, and the public can see and better understand IRS’s plans for serving the public; and the Integrated Transition Plan and Schedule, which provides a top-level view of the modernization program’s tasks, activities, and schedules and is the primary tool used for accountability for delivering the products and services necessary to implement modernization.

We found, however, that these documents are incomplete and inconsistent. For example, as of May 1995, 4 volumes of the 10 volume IRS Future Concept of Operations had not been completed. These volumes covered (1) national and regional offices, (2) workload distribution management, (3) area distribution centers, and (4) process flows. While the six completed volumes include critical areas, the incomplete documents are necessary for a comprehensive vision of IRS’s future operations.

Also, of the 27 action items identified in the Business Master Plan that relate to information systems, 15 could not be identified in the Integration Transition Plan and Schedule. Further, the Business Master Plan’s actions and performance measures have not been changed to reflect recent electronic filing trends, which indicate that IRS will fall far short of its electronic filing goal.

We found other indications of weak planning processes as well. Specifically, IRS did not have a fully integrated planning and budgeting process for TSM, although the Office of Economic Analysis is moving in that direction. For example, this office is developing a new TSM cost model for IRS. Steps such as this are positive because a strong tie between TSM plans and IRS budgets will be especially important to ensure that information is available to IRS managers and the Congress to show TSM’s future funding needs and the results of past investments.

While IRS has undertaken fundamental TSM planning, stronger overall strategic planning for TSM is still needed. This would involve (1) defining the information technology capabilities required to support reengineered business processes, (2) identifying, assessing, and mitigating the risks involved in developing both TSM as a whole, as well as individual component projects, (3) formulating schedules and milestones for development, and (4) allocating needed resources.

Information Technology Is Not Managed as an Investment

Currently, IRS does not have a process to manage TSM information systems projects as investments, even though IRS expects the government’s past and future investment in TSM to exceed $8 billion. Foremost, at the time of our review, IRS lacked comprehensive decision criteria for controlling and evaluating TSM projects throughout their life cycles.

When organizations use strategic information management best practices, they manage information systems projects as investments, rather than expenses. These organizations view projects as efforts to improve mission performance, not as efforts to implement information technology. For public and private sector organizations that have been successful in developing major systems, the basis for making decisions on information technology investments has been an explicit set of criteria that are used to evaluate the expected mission benefits, potential risks, and estimated cost of each project. This investment focus systematically reduces inherent risks while maximizing benefits of complex projects.

IRS maintains that all TSM projects have equal priority and must be completed or the modernization will fail. An “all-or-nothing” approach to large information technology projects is usually unrealistic and generally unattainable. Instead, a reasoned and an explicit framework for managing information technology investments is essential.

IRS currently holds program control meetings to assess and control information technology. However, these meetings have generally focused on the costs and implementation schedules of individual projects, rather than on comprehensively evaluating and prioritizing risks and returns expected from these investments. Instead of using explicit criteria to measure risks and returns, IRS evaluates each project’s progress using a time-line.

At the completion of our review, IRS had developed draft criteria for TSM projects. These criteria included risk and return factors (e.g., cost, project size, and mission benefit), which it plans to use for the first time during top management’s review of the fiscal year 1997 budget. However, these factors were not defined so they could be used consistently to assess projects. For instance, IRS characterized project size as small, medium, large, and very large, but did not quantify these terms. Similarly, IRS has not yet defined decision criteria and quantifiable measures to assess mission benefits, risk, and cost, all of which are important to enable IRS managers to adequately select, control, and evaluate information systems projects. IRS is currently developing better decision criteria.

Managing TSM as an investment would require IRS to assess, prioritize, control, and evaluate its investment in current and planned TSM information technology projects based on explicit and consistently applied decision criteria. By adopting this approach, top management’s attention would be drawn to assessing and managing risk and making the tradeoffs between continued funding of existing operations and developing new capabilities. Most important, with a disciplined process, IRS could promptly identify, and thus avoid investing in, higher-risk projects that have little potential to provide significant mission benefits. Moreover, this would reenforce accountability for improved performance.

Analyzing Costs and Benefits Is Inadequate

Contrary to best practices used by leading private and public organizations, IRS’s TSM costs and benefits analysis is inadequate. As a result, IRS and the Congress do not know whether TSM information systems projects will really make a difference. Until an adequate analysis is performed and measures are defined, IRS will not know whether investments in TSM are worthwhile.

In January 1995, IRS advised the House Budget Committee that, including operating costs for the next 10 years, TSM will cost about $13 billion and will provide over $32 billion in benefits. However, IRS’s overall cost projection is unreliable for several reasons. For example, IRS based the projection on an October 1992 TSM cost model, which IRS did not adequately update to reflect systems that have since been added to TSM, IRS’s more recent business visions, and changes in TSM systems development methods.

The benefits estimate also had shortcomings. For instance, in some cases, IRS attributed to TSM the savings associated with reducing staff resources; in other cases, IRS computed benefits based on additional revenues expected if staff were reassigned to tax collection. Although a decision to use these staff for collections may increase revenue, the additional staff—not the system—will provide this benefit. This point becomes clear when the following scenarios are considered: (1) IRS could assign additional staff to collections independent of the information system and (2) if IRS reassigns to other nonrevenue producing activities the staff years saved, the revenue benefits would evaporate even though the information system would not change. A convincing benefits analysis for a system must compare operational costs with and without the system, other variables being held constant.

IRS recognizes that it has not adequately assessed TSM costs and benefits and is currently working with a contractor on an economic analysis to better reflect the cost and benefits of TSM. IRS expects another cost and benefit analysis to be completed by September 1995. We will continue to monitor IRS’s progress in analyzing TSM cost and benefits.

Reengineering Efforts Are Not Tied to Systems Development Projects

After many automated systems design and development efforts were already underway, IRS started business process reengineering, which involves critically reexamining core business processes and redesigning them to achieve significantly better performance. Compounding this problem is IRS’s lack of a comprehensive plan and schedule defining how and when to integrate these business reengineering efforts with on-going TSM projects.

Organizations that successfully develop systems do so only after analyzing and redesigning critical business processes. Information systems projects that do not consider business process redesign typically fail or reach only a fraction of their potential. Accomplishing significant improvement in performance nearly always requires streamlining or redesigning critical work processes.

IRS has identified six core business areas and defined 11 business processes that support these areas. Of these 11, 3 were selected to begin reengineering efforts. Those selected for initial redesign are (1) processing returns, (2) responding to taxpayers, and (3) enforcement actions.

Overall, we found IRS’s reengineering methods to be consistent with generally recognized business process reengineering principles. IRS had, for example, assessed some existing data on customer values, analyzed current processes, and designed target processes and plans to validate the target designs. Further, IRS has a project management structure consisting of process owners, an executive steering committee, project managers, cross-functional teams, and contractor support to ensure that all stakeholders can participate. However, these efforts are not yet complete, and IRS did not assess the actual steps needed to implement these efforts.

IRS officials acknowledge that reengineering efforts began after the start of many TSM systems projects. Until reengineering is sufficiently completed to drive TSM projects, there is no assurance that the projects will achieve the desired business objectives and result in improved operations.

IRS Is Working to Upgrade Skills and Training

IRS is currently reassessing its skill and competency base to ensure that its personnel and training programs will meet future needs. Operating and maintaining progressively sophisticated systems, such as those comprising TSM, requires continuously higher skill levels and updated knowledge—an additional critical factor for success, according to best-practice organizations. Antiquated skill bases can inhibit an organization’s ability to change.

IRS has several initiatives planned and underway to upgrade the skills of its personnel. For example, IRS has defined positions needing competency assessments; plans to assess staff skills using competency assessment instruments, which are currently being developed; and is reorganizing and strengthening its training program by establishing a Corporate Education unit.

We are currently assessing IRS’s human resource planning for modernization and will continue to monitor progress in this area.

Recommendations

To address IRS’s strategic information management weaknesses, we recommend that the IRS Commissioner take immediate action to implement a complete process for selecting, prioritizing, controlling, and evaluating the progress and performance of all major information systems investments, both new and ongoing, including explicit decision criteria.

We also recommend that IRS use these criteria to review all planned and ongoing systems investments by June 30, 1995. Meeting this time frame is important so that the Congress has a sound basis for determining IRS’s fiscal year 1996 appropriations.

Agency Comments and Our Evaluation

IRS agreed with our recommendations to improve its strategic information management. In addition, IRS said that it had recently completed a self-assessment of its practices compared to GAO’s best practices for strategic information management. According to IRS, its self-assessment confirmed GAO’s findings and will help strengthen IRS’s overall response to GAO’s concerns.

In response to our recommendations, IRS said that it will continue to work on simplifying and ensuring the consistency of all its has initiated a priority setting process for meeting business needs through has developed an initial set of investment evaluation criteria for use as part of an ongoing process to evaluate spending plans for information systems; has completed a comprehensive review of the proposed fiscal year 1996 budget for TSM, which will enable IRS to rescope its program objectives, set priorities, and adjust funding levels for TSM; will continue to refine the investment evaluation criteria and also institutionalize a formal process based on the use of this criteria; and is developing and implementing the use of an information technology investments alternative to select, prioritize, control, and evaluate information technology investments to achieve reengineered program missions.

Actions such as these could provide IRS the underpinnings it needs for strategic information management. IRS indicated that progress toward implementing these improvements will be monitored by the IRS’s Associate Commissioner. We believe that this is essential to ensure prompt and effective implementation.

Regarding a cost and benefits analysis, IRS said that the September 1995 analysis will address the costs and benefits of TSM and allow IRS to identify and focus on competing priorities. In particular, IRS expects the new analysis to reflect a much more extensive benefit estimate than IRS currently has available. We believe an adequate cost and benefits analysis will help IRS to know whether investments in TSM are worthwhile.

Regarding skills and training, IRS said that it is taking steps to ensure that personnel and training programs meet future needs, especially those relating to information systems. These steps include (1) establishing a training steering committee to consolidate all information systems training currently underway, with the goal of increasing the skill level of IRS employees and (2) identifying job requirements for information systems professionals, which IRS will use in developing training and education programs that are directly linked to mission needs and critical occupational performance goals.

Although IRS is in the process of identifying job requirements, we believe that, until reengineering is complete, IRS can only incorporate prototype job requirements into its training and development efforts. In addition, IRS’s current plans do not address how job requirements created as a result of reengineering efforts will be incorporated into its training environment.

Software Development Process Is Weak but Improvements Are Underway

IRS’s software development activities are inconsistent and poorly controlled because IRS has few detailed procedures for its engineers to follow in developing software. IRS’s software development deficiencies can greatly affect the quality, timeliness, and cost-effectiveness of TSM. Unless IRS improves its software development capability, it is unlikely to build TSM timely or economically and systems are unlikely to perform as intended.

To assess its software capability, in September 1993, IRS rated itself against a Capability Maturity Model (CMM) designed by the Software Engineering Institute, a nationally recognized authority in the area. IRS found that, even though TSM is a world-class undertaking, its software development capability is immature. IRS placed its software development capability at the lowest level, described as ad hoc and sometimes chaotic and indicating significant weaknesses in software development capability.

Realizing that its software development capability needed improvement, IRS initiated process action teams to address software development weaknesses in key process areas. These teams have made varying degrees of progress to improve IRS’s software development capability and define uniform procedures in the key process areas.

Their progress notwithstanding, substantial additional improvement is necessary before IRS’s software development capability can be upgraded to at least the next CMM level, where its activities would be more disciplined and considered to be repeatable. Whether software development is done by IRS, which has nearly 2,000 people working in the area, or by contractors, mature software development capabilities are key to quality, timely, and cost-effective TSM software development.

Closely associated with one key software development process area, software quality assurance, is the use of software metrics, which are numerical measures used to predict an aspect of software quality. In this regard, we found that IRS has not adequately defined a suite of metrics. Moreover, IRS is not consistently or effectively using even its limited metrics for assessing the quality of software development projects throughout their life cycles.

Software Development Capability Is Immature

The Software Engineering Institute was established at Carnegie Mellon University in 1984 primarily to address the Defense Department’s software development problems. In 1991, the Institute developed CMM for use by organizations to evaluate their capability to consistently and predictably produce high-quality software. Table 4.1 describes CMM’s five maturity levels.

IRS rated itself at CMM level 1 because its assessment showed significant weaknesses in all key process areas prescribed for an organization to attain a level 2 capability. The key process areas designated by the Institute as necessary to reach CMM level 2 include (1) requirements management, (2) software project planning, (3) software project tracking and oversight, (4) software quality assurance, and (5) software configuration management.

Further, the National Research Council also identified IRS’s software development weaknesses and, in its Fall 1994 report on TSM, stated that IRS needed to develop a mature software development organization. The Council reported that, compared to accepted modern standards, IRS’s internal development capability is largely out of date and rudimentary.

Software Development Improvement Initiatives Are Progressing

To improve its software development capability and attain a higher CMM rating, the IRS Information Systems Organization’s Applications Design and Development Management group initiated five process action teams to address the weaknesses identified by IRS’s assessment and the National Research Council’s review. Table 4.2 identifies the teams and describes the key process areas each was to address.

The following discussion highlights the work of these teams, which we found in various stages of completion. Although the teams have generally made progress, IRS’s software development capabilities remain weak in each of the key process areas they were to address.

The Requirements Management team (1) studied and flow charted the process for requesting information services and (2) generated and is delivering related training materials. However, the requirements management process developed by the team is currently being applied to only legacy systems (i.e., existing IRS systems). An equivalent requirements management process for TSM systems was still under development. Also, customer involvement with the team’s requirements management process has been limited.

The Software Quality Assurance team adopted the peer review portion of a planning, review, and inspection process developed by IRS’s Quality Assurance Group. The team is applying this process to selected projects and has developed training for using the process, which IRS is giving to its systems engineers. However, IRS has not yet decided whether to conduct the team’s peer review approach on all projects. Also, IRS has yet to define detailed procedures for performing other software quality assurance functions, such as (1) ensuring compliance of software products and processes with defined standards, (2) independent verification of product quality, (3) periodic audits and reviews by the Software Quality Assurance group, and (4) feedback of the software quality assurance activities and findings to facilitate improvement of the process.

The Project Planning and Tracking team selected a software tool for planning and tracking the progress of software development projects. Because the team did not prepare guidelines specifying the minimum planning and tracking elements to apply to projects, project managers who use the software must define the details to track. As a result, this tool is being inconsistently used and, thus, IRS has been unable to consistently track the progress of their projects.

The Testing team has issued guidance on unit testing. However, there are no procedures for systems and acceptance testing.

The Configuration Management team is waiting for configuration management of the corporate-level to be defined in order to define lower-level processes and procedures. The only configuration management in place is version control of software. As a result, important items are not yet under configuration management, including documentation, and software development folders.

Although the teams have made progress, their accomplishments have not significantly improved IRS’s software development capability. Foremost, IRS has not developed and implemented consistent guidelines and procedures in the key process areas essential at CMM level 2. Unless IRS’s weaknesses in software quality assurance and software configuration management are corrected, IRS faces a much greater risk of extensive rework, schedule slippage, and cost overruns in developing software.

This risk is present whether IRS or a contractor develops TSM software. In this regard, to effectively oversee a contractor’s work to develop software, and thereby help to ensure prompt and successful completion of the software, it is important for IRS’s software project managers to understand the practices needed to develop software at CMM level 2. To further mitigate the risk of potential problems in developing software under contracts, it is critical that IRS’s software development contractors not be at CMM level 1. IRS does not, however, require all of its software development contractors to be at least at CMM level 2.

Metrics Are Not Consistently Used

Although not a specific key process area for rating an organization’s software capabilities, it is nonetheless crucial that a set of quality indicators, and their associated measures, called metrics, be used to assess the quality of software development throughout a project. IRS has not yet effectively established such a measurement process.

Early detection and avoidance of problems and control of software development projects are possible through the collection, validation, and analysis of metrics, which are numerical measures presumed to predict an aspect of software quality. Useful metrics include numbers of defects found at various stages of development, costs to repair defects, and the extent of test coverage.

Basically, metrics, such as the number and frequency of errors associated with a specific section of software, are taken to analyze the quality of software. Such analyses can identify situations where quality is unacceptable or questionable. In this way, the metrics are validated against quality factors throughout a software development project.

According to IRS officials responsible for software development, IRS has not yet defined a complete suite of metrics to be used in the software development program to assess the on-going quality of TSM projects. IRS’s present use of metrics allows for only one type of metric, collectively called function points.

Even so, IRS’s use of function points for assessing all software development projects is inconsistent, and IRS does not have a firm schedule for full implementation throughout the agency. In addition to function points, the following metrics, would, at a minimum, also be necessary: (1) complexity, (2) personnel and effort, (3) problems/defects by development phase, and (4) cost per defect.

Further, IRS’s use of function points does not trace back to quality improvement goals derived from IRS’s business objectives. In this regard, IRS could use the following metrics to measure software attributes related to business goals:

Fewer product defects found by customers.

Earlier identification and correction of defects.

Fewer defects introduced during development.

Faster time to market.

Better predictability of project schedules and resources.

Without clearly establishing a suite of metrics that trace back to business objectives through quality improvement goals, and that are implemented organizationwide in a uniform and consistent manner, IRS will be hampered in assessing the progress and quality of its software projects. Moreover, the absence of a suite of metrics makes it difficult for IRS to identify the reasons certain software development practices perform well while others perform poorly. Metrics, therefore, when used organizationwide in developing software, would provide IRS a means to better ensure uniform software development, thus avoiding the potential for repeating problems that could be costly and time-consuming to correct.

Recommendations

To address IRS’s software development weaknesses, we recommend that the IRS Commissioner immediately require that all future contractors who develop software for the agency have a software development capability rating of at least CMM level 2.

To further upgrade IRS’s software development capability, we also recommend that the Commissioner take action before December 31, 1995, to define, implement, and enforce a consistent set of requirements management procedures for all TSM projects that goes beyond IRS’s current request for information services process, and for software quality assurance, software configuration management, and project planning and tracking and define and implement a set of software development metrics to measure software attributes related to business goals, such as those outlined in this chapter.

Completing these actions by the end of 1995 is essential so that the Congress, in monitoring TSM’s progress and acting on TSM budget requests, has assurance that IRS will be able to effectively develop, and/or oversee contractors’ development of, software associated with systems modernization projects.

Agency Comments and Our Evaluation

IRS agreed with our recommendations for improving its software development capability, and is taking steps to do so. IRS said that it is committed to developing consistent procedures addressing requirements management, software quality assurance, software configuration management, and project planning and tracking.

Regarding metrics, IRS said that it is developing a comprehensive measurement plan to link process outputs to external requirements, corporate goals, and recognized industry standards. IRS said also that it has “baselined” all legacy systems using an accepted Software Engineering Institute metric.

We believe these steps, if implemented and institutionalized effectively, would provide IRS the disciplined approach necessary to improve its software development capability. Mature software development capabilities are key to quality, timely, and cost-effective TSM software development.

IRS also stated its belief that most government agencies and private organizations are not far along in raising their software development maturity profiles. We have identified several government organizations that have adopted CMM and are moving toward higher CMM levels. For example, the Department of the Army’s Information Systems Software Development Center in Virginia and the Department of the Air Force’s Sacramento Air Logistics Center were both assessed by SEI authorized assessors as CMM level 3. The Air Force also has a deadline for all its software activities to reach CMM level 3 by 1998. The software development capabilities of other organizations notwithstanding, we believe that a complex and costly systems development project, such as TSM, at a minimum, would warrant a CCM level 2.

IRS Recognizes Importance of Better Systems Architectures, Integration, and Testing but These Are Not Yet Adequate

IRS is not adequately performing and managing key TSM technical activities critical to the success of a large and complex systems modernization effort. In particular, IRS has not (1) defined and completed a TSM architecture, (2) established effective processes for configuration management, (3) defined the interfaces and standards needed to ensure that TSM components successfully integrate and interoperate, and (4) defined and completed TSM testing plans and established a testing facility.

IRS recognizes that, for modernization to succeed, TSM’s technical activities must be better defined, performed, and managed. Until IRS improves these areas, it is at increased risk of developing systems that are unreliable, do not meet user needs, cannot work together effectively, and require significant and costly redesign and reprogramming to correct weaknesses.

TSM Integrated System Architecture Is Incomplete

IRS has adopted a systems development methodology, known as Information Engineering, which is a formal, structured system development methodology widely used in the public and private sectors to provide a disciplined approach to information systems development. The principal deliverable of Information Engineering’s first stage, Information Strategic Planning, is an integrated systems architecture.

An integrated systems architecture (1) guides and constrains system design and development by providing a balanced, top-down view of the system, which system designers need to build the system and (2) organizes system functionality and defines relationships among those functions. In establishing this guidance and functionality, it is key to define security and data architectures and standard application program interfaces.

In July 1993, IRS published an initial version of its integrated system architecture. According to this document, the TSM integrated systems architecture will be completed as other modernization work progresses. This approach defeats the purpose of an integrated systems architecture, which is to guide a system’s development, not to merely document its development without formal guidance. Further, TSM security and data architectures and standard application program interfaces are incomplete and, thus, designers and developers do not have sufficient guidance to build individual TSM systems.

Security Architecture Is Incomplete

Because TSM’s security architecture is incomplete, systems designers do not have sufficient guidance on how to incorporate restricted access to IRS systems and data. IRS has made progress in defining its security requirements, but it continues to develop and implement systems without first completing the necessary security architecture and security applications.

In February 1994, IRS issued a risk assessment that identified potential security risks, determined their severity, and identified areas needing safeguards, and in October 1994, issued an information security policy. Since then, IRS has completed security documents relating to high-level security requirements, including mission, management, and technical security requirements; functional security requirements, which specify user security needs; a preliminary data sensitivity analysis, which is used to determine data sensitivity (e.g., sensitive but unclassified, etc.); and a draft information system target security architecture, which specifies TSM information security goals.

In addition, an IRS infrastructure and engineering task group has defined a set of preliminary security applications program interfaces that will guide application developers in requesting systems security functions. IRS officials told us that once these interfaces have been completed and thoroughly tested, IRS will mandate their use.

This progress notwithstanding, the TSM security architecture and security applications interfaces remain incomplete and unavailable to systems designers and developers. Without this crucial systems security guidance, IRS has no assurance that taxpayer data will be adequately protected.

Key security guidance that has not yet been developed includes a disaster recovery and contingency plan, which would ensure that information systems can restore operations and data in the case of sabotage, natural disaster, or other operational disruption; a security concept of operations, which would define IRS plans for operating in TSM’s new security environment; a security test and evaluation plan, which would validate the operational effectiveness of system security controls; a security certification and accreditation plan, which would provide IRS managers and system security officers adequate assurance that the system will protect information as required by the security policy; a communications security plan, which would define how security controls will be implemented when sending and receiving sensitive information electronically between and among distributed TSM subsystems and external agencies that must provide tax-related information to IRS; and an identification and authentication plan, which would define processes to verify user identities when accessing sensitive tax data.

Security has been a serious problem with IRS’s current systems. Our audits of IRS’s financial statements under the Chief Financial Officers Act (Public Law 101-576) have shown that IRS’s controls do not yet ensure that taxpayer data are adequately protected from unauthorized access, change, disclosure, or disaster. Specifically, IRS has not adequately (1) restricted access to taxpayer data to only those employees who need it, (2) monitored the activities of thousands of employees who were authorized to read and change taxpayer files, and (3) limited the use of computer programs to only those that have been authorized.

We have reported that, as a result, IRS did not have reasonable assurance that the confidentiality and accuracy of these data were protected and that the data were not manipulated for purposes of personal gain. IRS’s own reviews have identified instances where IRS employees (1) manipulated taxpayer records to generate unauthorized refunds, (2) accessed taxpayer records to monitor the processing of fraudulent returns, and (3) browsed taxpayer accounts that were unrelated to their work, including those of friends, relatives, and neighbors.

Data Architecture Reflects Existing Processes Rather Than Reengineered Processes

IRS is perpetuating its current data weaknesses by continuing to build TSM systems without the guidance afforded by a data architecture that reflects reengineered processes. An IRS analysis of its current systems identified the following data weaknesses:

Updated data on one system are not immediately available to users of other systems.

Master data files are updated once a week, and it can take up to 2 weeks for data in a taxpayer account to be changed. Inconsistent and incomplete data on different systems can affect fundamental computations and can result, for example, in inconsistent calculations of interest and penalties.

Data are stored in unique formats on different systems and are accessed using various techniques.

In 1994, to address data weaknesses, IRS initiated the Corporate Accounts Processing System project. IRS is developing this project in phases over 7 years, with each phase adding new TSM functionality. Through the Corporate Accounts Processing System, IRS expects to provide more efficient access to data, reduce data redundancy, and improve data integrity.

Nonetheless, the success of the Corporate Accounts Processing System project depends on improving current business processes through reengineering. At the time of our review, however, the project was modeling IRS’s existing business processes because IRS had not completed its reengineering.

To effectively correct existing data weaknesses that IRS identified and that the Corporate Accounts Processing System project is to address, IRS must first define how its business processes will be reengineered. Only then will IRS be in a strong position to build new systems based on a data architecture that reflects reengineered business processes.

Standard Application Program Interfaces Are Not Defined

Standard application program interfaces are essential to guide systems development because they define how applications software can access and use standard functions and services (e.g., communications services). These interfaces provide many systems development benefits, including improved interoperability, consistent implementation, less complex applications, standardized software coding, and simplified maintenance.

Realizing the benefits of providing standard application program interfaces for system development, IRS has established an interface task group and initiated an effort to define, code, test, and document standard application program interfaces for TSM. IRS has drafted an infrastructure services manual to provide an explanation of infrastructure services that will be available to systems developers. IRS also expects to prepare a more comprehensive and detailed manual describing application processing interfaces.

However, many TSM standard application program interfaces are not yet defined, implemented, or documented. Nonetheless, IRS is continuing to build TSM projects. As a result, these projects are likely to require modification once standard application program interfaces are defined and required.

Effective Configuration Management Practices Are Not Established

Systems change throughout their life cycle to (1) improve systems designs and operations and facilitate maintenance, (2) reflect changing mission requirements, and (3) respond to changes in the budget and schedules. These changes must be controlled through configuration management to ensure that they are cost-effective and properly implemented, documented, and tested.

Configuration management ensures that the integrity and stability of a system are preserved as it is designed, built, operated, and changed. Configuration management is also important for making engineering and trade-off decisions, maintaining up-to-date systems descriptions, and tracking every system component.

In 1994, IRS established an Information System Configuration Control Board to manage and control all systems changes. However, the Board has focused on monitoring individual project costs and schedules and developing configuration management guidance. A process has not yet been established to manage systems changes. Further, IRS does not have a configuration management plan that precisely defines the processes to be implemented, how and when they will be implemented, and who will be responsible for performing specific configuration management functions.

Systems Integration Planning Is Incomplete

In 1992, IRS initiated an effort to design and develop both a comprehensive integration strategy and a programwide integration plan to help IRS successfully transition from its current environment to one that meets TSM-defined objectives and capabilities. A preliminary strategy described by IRS’s Executive for Systems Architecture was for (1) an integration approach that included a methodology to integrate current and future initiatives into the TSM systems architecture, (2) an associated problem detection and resolution process, and (3) the analysis processes (e.g., testing and quality assurance) required to ensure projects are being, and have been, successfully integrated. The preliminary strategy addressed both the integration of individual projects and the transition of all projects to an integrated processing environment.

Since then, little has been done to complete a comprehensive integration strategy or develop an integration plan that defines implementation guidance and processes. In 1994, IRS planned to perform further work on integration management, but did not fund this effort in either fiscal year 1994 or fiscal year 1995. Until there is an effective integration process and a completed integration plan in place, IRS will have little assurance that its systems modernization components will operate effectively together.

System Testing and Test Planning Are Inadequate

An organization performs system testing to detect system design and development errors and correct them before putting a system into operation. Inadequate testing increases the likelihood that errors will be undetected, reduces the extent to which a system can provide accurate and reliable processing services and information, and, because the discovery of errors is likely to be delayed, increases the cost of modifying the system.

A testing plan ensures that sufficient testing is done during system development and prior to deployment. The plan defines, for example, what is to be accomplished during testing, who is to do the testing, where it is to be performed, and what constitutes success.

IRS acknowledges the importance of testing in the development of TSM systems, but has not yet developed a complete and comprehensive testing plan for TSM. In addition, individual TSM system development projects are developing their own testing plans. IRS describes these individual testing plans as rudimentary and inadequate. As a result, IRS has no assurance that its individual systems will be thoroughly and consistently tested or that systems will perform correctly or effectively.

Currently, IRS performs system development testing in an operational environment using taxpayer data at its service centers or computer centers. Because tax processing production work at these facilities has a higher priority than testing, the time, computer, and human resources applied to testing, as well as the resulting depth of testing, are limited. This limitation seriously affects testing quality and completeness. This testing environment also introduces the possibility that testing can, under unforeseen circumstances, affect and disrupt production.

To help overcome this situation, IRS plans to establish an Integration Test and Control Facility to provide an environment that will more effectively support the testing and integration of legacy and TSM systems. By establishing this testing facility, IRS expects to (1) improve the quality of delivered software, (2) provide information resources needed for testing and integration, and (3) reduce risks in integrating and transitioning from current legacy systems to TSM.

In September 1994, IRS developed a concept of operations for the integrated testing facility, which describes its functions and responsibilities. IRS has been working with a contractor to define the facility’s functions and responsibilities. IRS is also working with the General Services Administration to select a facility site. However, until IRS completes its testing plans, implements effective testing processes, and establishes its Integration Test and Control Facility, it has little assurance that systems will be adequately and effectively tested.

Recommendations

To address IRS’s technical infrastructure weaknesses, we recommend that, before December 31, 1995, the IRS Commissioner complete an integrated systems architecture, including security, telecommunications, network management, and data management; institutionalize formal configuration management for all newly approved projects and upgrades and develop a plan to bring ongoing projects under formal configuration management; develop security concept of operations, disaster recovery, and contingency plans for the modernization vision and ensure that these requirements are addressed when developing information system projects; develop a testing and evaluation master plan for the modernization; establish an integration testing and control facility; and complete the modernization integration plan and ensure that projects are monitored for compliance with modernization architectures.

Completion of these actions in 1995 is essential so that the Congress, in carrying out its oversight role and making TSM funding decisions, has assurance that the government’s TSM investment is adequately protected through effective management of the technical aspects of tax processing modernization.

Agency Comments and Our Evaluation

IRS agreed with our recommendations to improve its systems architectures, testing, and integration. IRS commented that it is identifying the necessary actions to ensure that defined systems development standards and architectures are enforced agencywide. IRS said also that it is planning for its 1996 IRS Information System Architecture to reflect a total system view; is reviewing existing documentation to determine how best to incorporate our security architecture recommendation; is in the process of improving its configuration management process by implementing change control, as well as developing guidance; has initiated a series of assessments for major TSM systems to review and baseline existing requirements for each deliverable, including documented interfaces; will merge integration testing, systems testing, and other testing-related personnel in one facility, and is planning to establish an interim test and control capability; and has developed a release engineering approach to transition from its current environment to one meeting TSM-defined objectives and capabilities.

We believe that actions to improve TSM’s technical infrastructure, such as those IRS has outlined in its comments, are necessary prerequisites to adequately develop and implement new systems. In addition, while release engineering can facilitate the transition from IRS’s current environment to one meeting TSM-defined objectives and capabilities, to be successful, it must be closely coordinated with requirements and configuration management.

IRS Recently Acted to Strengthen Accountability and Responsibility for Systems Modernization

Effective overall systems modernization management is important because TSM is not a one-time, turnkey replacement of all current subsystems; rather, it is a target system that will be reached by incrementally upgrading or replacing operational subsystems. Consequently, to successfully implement IRS’s systems modernization, an organizational structure must be in place to consistently manage and control all systems development efforts. This organizational structure would provide accountability and responsibility for all systems investments, including prioritizing new modernization systems and upgrades and maintaining all operational systems.

However, below the Commissioner’s Office, the management authority and control needed to modernize tax processing has been fragmented. Until recently, IRS’s Modernization Executive was responsible for developing TSM information systems until they became operational. Under this executive, each TSM system was managed by a program control group that was tasked with reviewing the project, making milestone decisions, and mitigating project risks.

In addition, the Chief Information Officer was responsible for developing non-TSM systems and for the operation of all IRS systems. This included the TSM systems that were developed by the Modernization Executive and that had been in operation for about 1 year.

In addition to systems development and operations being managed and controlled by the Modernization Executive and the Chief Information Officer, several systems development projects were managed and controlled by IRS’s research and development division. For example, this division’s staff of 30 information specialists developed both Telefile and the Filing Fraud system, which are TSM systems. Neither the Modernization Executive nor the Chief Information Officer had decision-making responsibility for these systems or the authority to ensure compliance with IRS system development standards and practices.

During our April 28, 1995, meeting with the IRS Commissioner, we recommended that she establish consolidated, organizationwide control over all information systems investments, including all new systems in research and development and operational systems being upgraded and replaced. In May 1995, the Modernization Executive was named Associate Commissioner and given responsibility to manage and control all system development efforts that had previously been the responsibility of the Modernization Executive and the Chief Information Officer. However, the research and development division still does not report to the Associate Commissioner.

It is critical that the Associate Commissioner now establish organizationwide system modernization accountability and address the problems this report discusses. This entails ensuring strategic planning documents are complete and consistent; developing a comprehensive plan and schedule for linking reengineering efforts to systems development projects; exercising consolidated control over all information systems investments, including all new systems in research and development and operational systems being upgraded and replaced; and ensuring that defined systems development standards and architectures are enforced.

Recommendation

To fully strengthen systems development accountability and responsibility, we recommend that the IRS Commissioner give the Associate Commissioner management and control responsibility for all systems development activities, including those of IRS’s research and development division.

Agency Comments and Our Evaluation

In commenting on a draft of this report, IRS reiterated that the Associate Commissioner is responsible for all aspects of modernization program planning and management, budget formulation and execution, and information systems development and management. Further, IRS said that it was considering whether the Associate Commissioner’s systems development responsibilities are to include those of the research and development division.

We strongly urge IRS to also place with the Associate Commissioner accountability and responsibility for the research and development division’s systems development activities. By doing so, IRS will help to ensure that systems development efforts are consistently managed and controlled organizationwide.