Business Systems Modernization: IRS Needs to Complete Recent Efforts to Develop Policies and Procedures to Guide Requirements Development and Management
Published: 2006-03-20T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: The Internal Revenue Service's (IRS) effort to modernize its tax administrative and financial systems--Business Systems Modernization (BSM)--has suffered delays and cost overruns due to a number of factors, including inadequate development and management of requirements. Recognizing these deficiencies, IRS created a Requirements Management Office (RMO) to establish policies and procedures for managing requirements. GAO's objectives were to assess (1) whether the office has established adequate requirements development and management policies and procedures and (2) whether BSM has effectively used requirements development and management practices for key systems development efforts.
section_title: What GAO Found
paragraphs: BSM does not yet have adequate policies and procedures in place to guide its systems modernization projects in developing and managing requirements. In January 2006, the RMO developed a set of draft policies that address some key areas of requirements development and management; these policies are to serve as interim guidance while the final policies and processes are being developed. At the conclusion of GAO's review, the RMO also provided a high-level plan that includes milestones for completing these policies. Since critical BSM projects continue to be pursued and completion of the policies and procedures is not expected until March 2007, it is critical that BSM immediately implement the draft policies and continue to develop the final policies. As a result of the lack of policies and procedures, the one ongoing project--Modernized e-File (MeF)--and the two completed projects--Filing and Payment Compliance (F&PC) and Customer Account Data Engine (CADE)--GAO reviewed did not consistently follow disciplined practices for systems development and management. For example, all three projects had a key element of managing requirements--a change management process that requires approvals and impact assessments to be completed when there are changes to requirements--but none met all of the practices needed for effective requirements management. In addition, two projects did not have a clear, consistent way to elicit (gather) requirements, two did not have fully documented requirements, and two could not produce fully traceable requirements (i.e., the requirements could not be tracked through development and testing), which is another key element of managing requirements. Unless IRS takes the steps needed to develop and institutionalize disciplined requirements development and management processes and implements draft policies in the interim to cover key areas of requirements development and management, it will continue to face risks, including cost overruns, schedule delays, and performance shortfalls.

section_title: Letter
paragraphs: The Internal Revenue Service (IRS) has long relied on obsolete automated  systems for key operational and management functions; its attempts to  modernize these systems span several decades. IRS’s current  effort—Business Systems Modernization (BSM)—is a highly complex,  multibillion dollar effort to modernize its technology and related business  processes. Over the past 7 years, IRS has been appropriated approximately  $1.9 billion for BSM. These systems modernization projects have  experienced cost overruns and schedule delays due to, among other things,  inadequate development and management of requirements. Lack of  attention to these crucial processes has led to projects not meeting cost,  schedule, and performance goals.
IRS has recognized these deficiencies and established a Requirements  Management Office (RMO) to, among other things, develop the policies and  procedures that are to cover all aspects of requirements development and  management. This report responds to your request that we assess (1)  whether the RMO has established adequate requirements development and  management policies and procedures and (2) whether BSM has used  effective requirements development and management practices for key  systems development efforts.
To accomplish our objectives, we reviewed BSM requirements  development and management policies, guidance, procedures, and tools.  We also analyzed two completed and one ongoing BSM systems  development projects and interviewed appropriate IRS and contractor  officials. Further details on our objectives, scope, and methodology are  provided in appendix I. Our work was performed from June 2005 through  February 2006 in accordance with generally accepted government auditing  standards.
subsections: section_title: Background
paragraphs: IRS is currently replacing its antiquated tax administration and financial  systems. This effort, as we have reported numerous times, has suffered  delays and cost overruns due to a number of reasons, including inadequate  development and management of requirements.
subsections: section_title: History of IRS Modernization
paragraphs: The IRS tax administration system, which collects approximately $2 trillion  in annual revenues, is critically dependent on a collection of obsolete  computer systems. Congress and IRS designed the BSM program to bring  IRS tax administration systems to a level equivalent to private and public  sector best practices, while managing the risks inherent in one of the  largest, most visible, and sensitive modernization programs under way.  Over the past 7 years, IRS has been appropriated approximately $1.9 billion  for BSM (see fig. 1).
BSM is critical to supporting IRS’s taxpayer service and enforcement goals.  For example, BSM includes projects to allow taxpayers to file and retrieve  information electronically and to help reduce the backlog of collection  cases. It also provides IRS with the reliable and timely financial  management information it routinely needs to account for the nation’s  largest revenue stream.
BSM has had some recent successes with its modernization efforts. During  2004, IRS implemented initial versions of (1) Modernized e-File (MeF),  which provides electronic filing for large corporations and tax-exempt  organizations; (2) e-Services, which created a Web portal and other  electronic services to promote the goal of conducting most IRS  transactions with taxpayers and tax practitioners electronically; (3)  Customer Account Data Engine (CADE), which will replace the current  system that contains the agency’s repository of taxpayer information; and  (4) the Integrated Financial System, which replaced aspects of IRS’s core  financial systems and is ultimately intended to operate as its new  accounting system of record.
However, despite these successes, IRS has had difficulty developing and  managing requirements for its modernization efforts over the years. We  reported in 1995 that IRS did not have the requisite software development  capability to successfully complete a major modernization effort and that  the success of modernization would depend on whether IRS would  promptly address the weaknesses in several software development areas,  including requirements management. In 1998, we assessed IRS’s systems  life cycle document and reported a lack of sufficient information to  document how business requirements were to be specified. More recently,  in February and November of 2004, we reported in testimony and a report  that cost overruns in various BSM projects, including CADE, MeF, and  e-Services, were due in part to inadequate definition of requirements for  their new systems, leading to incorporation of additional requirements late  in the system’s life cycle and at a higher cost than if they had been included  in the initial requirements baseline. We continue to highlight management  control weaknesses such as these in our annual expenditure plan reviews.
Other organizations that have assessed BSM projects have also found that  IRS has not developed and managed requirements sufficiently on various  projects. In 2001, the Treasury Inspector General for Tax Administration  (TIGTA) reviewed key systems development practices of four BSM  projects and reported that weaknesses in several process areas, including  requirements management, were responsible for cost increases and  schedule delays. TIGTA noted that these weaknesses raised the risk that  systems would be developed that would not meet the needs of the  businesses they were intended to support and recommended that BSM  strengthen and/or implement aspects of these key systems development  practices. In 2003, an independent technical assessment of CADE noted  significant breakdowns in developing and managing requirements, which  resulted in the inability of CADE to meet its original schedule. The  assessment further stated that IRS focused primarily on the high-level  business requirements and paid less attention to the development of  specific, testable requirements developed later in the development life  cycle and that responsibility for developing and managing the requirements  was distributed among their various organizational component, instead of  being concentrated in a centralized authority.
BSM has acknowledged that it has weaknesses in developing and managing  requirements; since 2004, requirements management has been one of its  high-priority initiatives. To demonstrate their commitment to improving  the development and management of requirements, it created an RMO in  October 2004. This office is to address issues related to (1) lack of quality  and completeness of modernization requirements, (2) lack of alignment of  modernization requirements with business strategy and needs, (3) risks  incurred by projects transitioning to development without a sufficient  requirements baseline, and (4) lack of visibility into a fully traceable set of  modernization requirements. During 2005, the RMO created a Concept of  Operations that showed, at a high level, the RMO’s plans to address  requirements practices, and, in November 2005, it obtained contractor  support to develop new policies and procedures.
section_title: Requirements Development and Management
paragraphs: According to the Software Engineering Institute’s (SEI) Capability Maturity  Model Integration (CMMIsm), the requirements for a system describe the  functionality needed to meet user needs and perform as intended in the  operational environment. A disciplined process for developing and  managing requirements can help reduce the risks of developing or  acquiring a system. A well-defined and managed requirements baseline can,  in addition, improve understanding among stakeholders and increase  stakeholder buy-in and acceptance of the resulting system. The practices  underlying requirements development and management include eliciting,  documenting, verifying and validating, and managing the requirements  through the systems life cycle (see fig. 2). This set of activities translates  customer needs from statements of high-level business requirements into  validated, testable systems requirements.
section_title: Requirements Development and Management Processes
paragraphs: The requirements development process starts with project teams eliciting,  or gathering, requirements from stakeholders or participants involved in  the project (e.g., customers and users). Since the usefulness of the system  to its users and stakeholders is critically dependent on the accuracy and  completeness of the requirements, all user groups and stakeholders should  be identified and involved in defining requirements. In addition to gathering  requirements from users and other stakeholders, analysis and/or research  can be used to identify additional requirements that balance stakeholder  needs against constraints and ensure that the requirements can be met in  the proposed operational environment.
After requirements have been elicited, they are analyzed in detail;  documented as the business, or high-level, requirements; and agreed to by  all stakeholders. Stakeholder agreement is an important part of this activity  and is needed to demonstrate that the requirements accurately define  intended uses. The business requirements should then be decomposed into  detailed system requirements, which are analyzed to ensure that they can  be implemented in the expected operational environment and that they can  satisfy the objectives of higher-level requirements. The final requirements  are approved by all stakeholders and documented as the requirements  baseline. Once the baseline is established, it is placed under configuration  management (CM) control.
Once the requirements baseline has been developed, the requirements are  analyzed and broken down into more specific system-level requirements  and eventually into the code that makes up the system. The verification  process ensures that the system-level requirements and the resulting code  are an accurate representation of stakeholder needs. This process includes  checking selected work products, such as software code, against the initial  baseline requirements to ensure that the lower-level items fully satisfy the  higher-level requirements. It is an inherently incremental process,  occurring throughout the development of the product. This agreement  between work products, such as code and baseline requirements, is  verified by conducting peer reviews. Peer reviews can also be used to  identify action items that need to be addressed. Without such reviews, an  organization is taking a risk that substantial defects will not be detected  until late in the development and/or testing phases, or even after the system  is implemented.
While the system is being developed, each component must be tested to  ensure that its outputs are accurate. Testing (e.g., unit, system integration,  and user acceptance) is the process of executing a program with the intent  of finding errors. Clear, complete, and well-documented requirements are  needed in order to design and implement an effective testing program.  Linking the testing activities back to the requirements assures the  organization that, once testing activities are successfully completed, all  requirements have been addressed and will be met by the system. Without  such assurance, it is possible for a requirement to be missed in  development and the resulting lack of functionality not noticed until late in  testing, or even after deployment.
Requirements, once developed and approved, also need to be managed  throughout the system life cycle. Two key areas of requirements  management are addressing changes to requirements and establishing and  maintaining bidirectional traceability from high-level requirements through  detailed work products to test cases and scenarios. As mentioned earlier,  once a set of high-level requirements is documented, verified, and  approved, it is placed under configuration control. From this point,  changes to the requirements are evaluated and validated as part of the change control process. Change control includes reviewing and assessing  proposed changes to the requirements to determine the reasons for the  changes, determining if these changes are occurring due to flaws in the  requirements development process, and ensuring that any effects of the  change on other requirements as well as on the cost, schedule, and  performance goals of the project are determined and assessed.
Establishing and maintaining traceability from initial requirements to work  products and the resulting system is also important. A requirements  traceability matrix demonstrates forward and backward (bidirectional)  traceability from business requirements to detailed system requirements all  the way through to test cases.
section_title: BSM Lacks Policies and Procedures for Requirements Management
paragraphs: BSM does not yet have adequate policies and procedures in place to guide  its systems modernization projects in developing and managing  requirements. In January 2006, the RMO developed a set of draft policies  that address key areas of requirements development and management;  these policies are to serve as interim guidance while the final policies and  processes are being developed. At the conclusion of our review, the RMO  provided us the draft policies and a high-level plan that includes milestones  for completing these policies. Since critical BSM projects continue to be  pursued and completion of the policies and procedures is not expected  until March 2007, it is critical that BSM immediately implement the draft  policies and continue to develop the final policies.
subsections: section_title: BSM Lacks Comprehensive Policies and Procedures for Requirements Development and Management, but Has Initiated Their Development
paragraphs: BSM does not have comprehensive, detailed policies and procedures for  requirements management and development activities that include  requirements elicitation, documentation, verification and validation, and  management. During our review, BSM officials were unable to provide us  with detailed policies and procedures and agreed that they do not have  such documents. Project teams were not consistent in their understanding  of which guidance they should use for the development and management of  requirements; some project team members mentioned BSM’s Enterprise Life Cycle (ELC); others said they were waiting for guidance from the  RMO. Our review of the ELC showed that it did not provide the procedures  project managers would need to properly perform the steps in the  requirements development and management process. BSM program  officials agreed that the ELC did not provide the needed guidance.
In December 2005, the RMO completed an analysis of requirements  development and management areas that need improvement. The RMO  found, as we did, that BSM lacks detailed guidance; their recommendations  included developing process handbooks for aspects of requirements  elicitation, documentation, verification and validation, and management.  Subsequently, in January 2006, BSM officials developed draft guidance that  covers aspects of requirements development and management. However,  this guidance does not fully address requirements elicitation,  documentation, verification and validation, and management. At that time,  BSM also provided us with a high-level plan that contains interim  milestones and establishes a March 2007 completion date for the final set of  policies and procedures. BSM officials told us that these draft policies are  to serve as interim guidance while the remaining policies and procedures  are being developed. In addition, IRS also uses its governance processes,  particularly the milestone exit reviews, to find and mitigate issues and  problems with requirements development and management on existing  projects. Finally, the RMO is allocating resources to key projects—such as  F&PC version 1.2—to assist them in developing requirements.
Without a formal set of documents that detail organizational policies and  associated procedures, employees and contractors will rely on their  individual knowledge and expertise to complete requirements development  and management activities. This raises the risk of cost overruns, schedule  delays, and reduction of functionality. Since critical BSM projects are  already under way, and completion of the policies and procedures is not set  until March 2007, it is urgent that BSM immediately implement the draft  policies. Until BSM develops and implements policies and procedures that  fully address the key areas of requirements development and management,  including elicitation, documentation, tracking of cost and schedule impacts  associated with requirements changes, and establishing and maintaining  full bidirectional traceability, ongoing projects will continue to run greater  risk of cost and schedule overruns and poor system performance.
section_title: BSM Projects Have Not Consistently Followed Disciplined Requirements Development and Management Practices
paragraphs: As a result of the lack of policies and procedures, BSM projects varied in  the extent to which they followed disciplined requirements practices. All  three projects we reviewed—MeF release 3.2 (to be deployed March 2006),  and F&PC release 1.1 (deployed January 2006), and CADE release 1.1  (deployed July 2004)—performed some of the practices associated with  sound requirements development and management. For example, all three  projects had a change management process in place that requires approvals  and impact assessments to be completed when changes are made to  requirements. However, none of the three BSM project releases we  reviewed consistently performed all of the practices needed for effective  requirements management. Specifically:  Project teams did not have a clear, documented, and consistent method  of eliciting requirements.
Project teams did not adequately document all requirements.
Project teams did not effectively verify requirements.
Project teams did not demonstrate adequate management of  requirements.
subsections: section_title: Project Teams Have Inconsistent Requirements Elicitation Practices
paragraphs: Based on stakeholder information such as customer expectations,  constraints, and interfaces for a system, the requirements elicitation team  discovers, defines, refines, and documents business-level requirements.  Due to the importance of this activity, plans or strategies should be in place  to guide project officials in defining elicitation-related activities and in  outlining how the requirements will be gathered (e.g., interviewing the  users or analyzing the current or expected business processes).
BSM project teams did not have a clear, consistent, and documented  method of eliciting requirements for the projects. For example, although  the teams identified stakeholders in their project plans, only MeF 3.2  provided evidence that working group meetings were conducted with  stakeholders to understand their needs and to identify their problems and  expectations and that strategies or plans were developed for eliciting  requirements. CADE 1.1 project team members could not describe how  they elicited requirements or provide a requirements plan that documented  elicitation procedures or strategies. An F&PC project team member stated  that, for release 1.1, the project did not have a fully documented process for  elicitation; however, the team member stated that the team had held  workshops and obtained resources and assistance from the RMO to help  mitigate the lack of a process. The RMO used lessons learned from this  effort to develop a new requirements elicitation process, which they expect  will assist F&PC in elicitation for its next release.
BSM project and program officials agreed that requirements elicitation  processes could be improved and stated that they were planning to address  some of the problems we found. For example, when we asked project  officials about the policies and procedures underlying their current  requirements elicitation activities, some stated that they were waiting for  new policies to be issued by the RMO, and others cited the ELC as  guidance. As mentioned earlier, the ELC does not provide the information  needed for the requirements elicitation process. Furthermore, F&PC  officials could not state which sections in the ELC described the  requirements elicitation process. BSM program management and RMO  officials acknowledged the lack of policies and procedures and stated that  the RMO has since developed new guidance for eliciting requirements that  will be piloted on F&PC version 1.2, which is currently entering the  requirements development phase.
BSM project teams performed elicitation processes in a nonstandard  manner due to the lack of policies, procedures, and guidance. Without  standardized policies and procedures to guide this key part of requirements  development, BSM program officials cannot ensure that its systems  development projects have collected and documented all the necessary  requirements, which could result in systems being developed that do not  meet user needs.
section_title: Projects Do Not Fully Document Requirements
paragraphs: After collecting and documenting high-level requirements from customers,  users, and other stakeholders, the requirements team should analyze these  high-level requirements against the conceptual (or expected) operational  environment to balance user needs and constraints and to ensure that the  system developed will perform as intended. The resulting lower-level  requirements should also be analyzed to make sure they can be performed  in the expected environment and that they satisfy the objectives of the  higher-level requirements. The final requirements are documented in the  requirements baseline.
The BSM projects we reviewed did not complete all of the activities needed  to adequately document requirements. Although project teams provided  evidence that they created a set of high-level requirements and obtained  approvals from stakeholders on this set of requirements, two of the three  projects did not provide evidence that requirements were thoroughly  analyzed and decomposed to lower-level system requirements. For  example, part of this analysis would link all lower-level systems  requirements to the original higher-level business requirements. Only one  of the project teams—F&PC—provided documentation showing the  necessary linking or mapping of lower-level system requirements to the  business requirements. MeF and CADE provided documentation; however,  their documentation did not fully demonstrate the linking of system-level  requirements to the business requirements.
A MeF project official agreed that full linkage of system-level requirements  to business requirements should be implemented. The MeF official stated  that they planned to implement this in their next version—version 4.0. In  addition, a BSM program official indicated that additional project guidance  on requirements documentation would be part of the RMO’s deliverables  and would help to address this issue.
Without feasible and clearly defined requirements, projects run the risk of  cost overruns, schedule delays, and deployment of systems with limited  functionality. For example, incomplete definition of requirements has been  cited as one reason for schedule delays and cost overruns for both CADE  and MeF.
section_title: Projects Do Not Fully Verify Requirements; Validation Activities Completed, but Problems Remain
paragraphs: Once requirements are fully documented, software code and other work  products that will guide development and testing activities need to be  verified using peer review techniques against the original requirements. In  addition, these products should be validated through testing to ensure that  they will operate effectively in the intended environment.
subsections: section_title: Projects Do Not Fully Verify Requirements through Peer Reviews
paragraphs: Requirements verification ensures that the lower-level requirements,  software code, and other work products that will guide development and  testing activities are an accurate representation of stakeholder needs. Peer  reviews are an important part of the verification process and are a proven  mechanism for effective defect removal. During peer reviews, teams of  peers examine code and other work products to identify defects,  determine the causes of the defects, and make recommendations that  address changes needed to help ensure that the system will meet  stakeholder and developer needs. Peer reviews should follow a structured,  formalized process; peer review events should be planned in advance, with  items, such as code and other work products, selected in advance; the  results of the sessions should be incorporated into peer review reports that  project teams are expected to address before moving further into  development activities.
The BSM project teams did not provide evidence that work products had  been verified against requirements through the use of a formalized peer  review process and project officials did not follow recommended practices  for conducting peer reviews. BSM project team members stated that they  conducted customer technical reviews and milestone exit reviews that they  considered to be peer reviews; however, these kinds of reviews do not meet  the criteria for peer reviews. They were not structured, did not select code  and other items in advance to be evaluated, and did not produce formal  peer reports with action items that projects were required to address.
section_title: Project Teams Performing Validation, but Problems Remain
paragraphs: Requirements validation is the process of demonstrating that a product  fulfills its intended use in its environment. It differs from the verification  activities previously described, in that validation determines that the  product will fulfill its intended use, while verification ensures that work  products properly reflect the baseline requirements. Validation includes  tests conducted on the product during development to prove that the  product performs its intended functions correctly. In a disciplined software  development process, planning for validation activities begins as  requirements are developed; testing activities are critical to determining  that a system not only operates effectively but addresses all baseline  requirements. To complete validation activities, testing is conducted at  several levels, each of which validates that the system will operate  effectively at a different level. For example, unit testing validates individual  sections of code, and system integration testing ensures that the system as  a whole can operate effectively in its environment. User acceptance testing  allows the user community to determine whether the system, as developed,  can be used to effectively support their work. It also validates that the  system meets user expectations. An effective testing process confirms the  functionality and performance of the product prior to delivery. It is a  crucial process and needs to be well planned, well structured, well  documented, and carried out in a controlled and managed way.
The BSM projects provided evidence of validation activities, such as test  plans and test results. CADE 1.1 officials provided both test plans and test  reports. MeF release 3.2 and F&PC release 1.1 are still in the testing phase;  they provided available test plans but do not yet have test reports.
Despite the existence of test plans and reports, requirements are not fully  documented or fully traced. In addition, while the ELC provides guidance  on testing that discusses test planning, activities, and test responsibilities,  program officials say that this guidance is limited. BSM’s Enterprise  Services organization has initiated an effort to review, revise, and enhance  test procedures across BSM. Therefore, until BSM ensures full  documentation and traceability of requirements, questions about the  completeness of its testing will remain.
section_title: Project Teams Not Adequately Managing Requirements
paragraphs: Finally, requirements must be managed through the system development  life cycle. We found that the three projects did not fully demonstrate  adequate management of their requirements. Although the projects had a  formal change control process in place to analyze and manage changes to  requirements, associated costs and schedule changes resulting from  requirements changes were not always tracked or updated. In addition,  projects’ documentation did not demonstrate adequate traceability of  requirements from the business requirements (high-level requirements) to  system requirements (low-level requirements) to test cases.
subsections: section_title: Project Teams Not Updating Cost and Schedule to Reflect Requirements Changes
paragraphs: Managing changes to the original requirements is a formal process to  identify, evaluate, track, report, and approve these changes. As work  products are developed and more is learned about the system that is being  developed, information is occasionally found that requires a change to the  original requirements. Modifications to project scope or design can also  result in requirements changes. Therefore, projects need to manage these  changes to requirements in a structured way.
The BSM project teams used a change management process to manage  changes to requirements that included documenting the rationale for  changes, developing assessments of the impact of the change, and  obtaining approvals by the Configuration Change Board. However, only the  MeF and F&PC teams provided evidence that their cost and schedule  baselines were updated when changes to requirements impacted cost  and/or schedule. For example, CADE officials did not provide any evidence  to show how they updated and tracked cost changes resulting from  changes to requirements, nor did they provide evidence that the work  breakdown structure was updated to reflect schedule changes. F&PC  officials provided evidence for tracking changes to the cost and schedule.  MeF officials provided a document that tracked the cost implications of  changes to requirements and the work breakdown structure to reflect  schedule changes.
A BSM project official indicated that the BSM project was implementing  cost and schedule tracking on its current releases. However, it was not  clear whether BSM was doing this consistently or whether appropriate  guidance for tracking cost and schedule would be provided by BSM.
Project teams that do not effectively track cost and schedule changes as a  result of changes to requirements will not be able to effectively mitigate the  potential impact of these changes to overall cost, schedule, and  performance goals, thus raising the risk of cost overruns, schedule delays,  and deferral of functionality.
section_title: Project Teams Not Ensuring Full Traceability of Requirements
paragraphs: Another key element of requirements management is establishing and  maintaining the traceability of requirements. Traceability of requirements  is tracking the requirements from the inception of the project and  agreement on a specific set of business requirements to development of the  lower-level system requirements, detailed design, code implementation,  and test cases necessary for validating the requirements. Tracing a  requirement throughout the development cycle provides evidence that the  requirements are met in the developed system and ensures that the product  or system will work as intended. Requirements must be traceable forward  and backward through the life cycle. Each business requirement must be  traceable to associated system requirements and test cases. Without  adequate traceability, errors in functionality could occur and not be found  until the testing phase, when problems are more costly to fix and time  frames for fixing problems without causing a schedule slip for deployment  are limited.
Of the three projects, only the F&PC team showed evidence of full  traceability of the requirements from high-level requirements to low-level  requirements. MeF and CADE documentation did not demonstrate clear  traceability from the business requirements to lower-level system  requirements, coding, and test cases. MeF program officials acknowledged  weaknesses in this area and stated that they planned to develop full  bidirectional traceability to the business level requirements as part of MeF  release 4.0.
According to project officials, one reason they do not have full  bidirectional traceability is due to the lack of detailed procedures and  guidance for traceability of requirements. Until recently, BSM projects  were not required to develop and use a traceability matrix. While interim  guidance issued by BSM does require the use of traceability matrices and  use of its configuration management repository to manage requirements,  the guidance lacks the detail needed to ensure that projects meet criteria.  BSM program officials agreed that this was an area that needed additional  guidance. The RMO is currently reviewing new guidance on how to  improve requirements traceability.
Without adequate traceability of requirements, system requirements can be  missed during development and the agency cannot be assured that  validation activities fully demonstrate that all the agreed-upon  requirements have been developed, fully tested, and will work as intended.
section_title: Conclusions
paragraphs: BSM lacks policies and procedures to develop and manage requirements  for their systems modernization projects. BSM has acknowledged this  deficiency since late 2004 when it listed requirements management as one  of its high-priority initiatives and created an RMO. The office has now  developed draft policies that cover aspects of eliciting, documenting,  verifying and validating, and managing requirements. These draft policies  are to serve as guidance to projects teams as BSM projects are pursued. It  is critical that BSM implement these draft policies immediately and  continue to develop the remaining policies.
The three BSM development projects that we reviewed showed significant  differences in how they implemented practices for developing and  managing requirements. Until BSM and the RMO complete the  development of policies and procedures to ensure disciplined requirements  development and management practices, projects will not have sufficient  guidance to ensure implementation of these practices, which will impair  their ability to effectively manage the development and acquisition of  critical systems and increase the risk of cost overruns, schedule delays, and  deferral of functionality.
section_title: Recommendations for Executive Action
paragraphs: To improve the requirements development and management policies and  practices of the IRS’s BSM, we recommend that the Commissioner of  Internal Revenue direct the Associate Chief Information Officer for BSM to  take the following two actions: 1. Ensure that BSM completes the delivery of policies and procedures for  requirements development and management as planned. The policies  and procedures should fully describe the processes, include a minimum  set of activities required for each project, and provide detailed  procedures for each of the key areas of requirements elicitation,  documentation, verification and validation, and management. As part of  this effort, the policies and procedures should specifically include the  following:  A standardized process for the elicitation of requirements that ensures  that projects fully investigate the requirements needed for a specific  system, including gathering requirements from all relevant users and  stakeholders.
A standardized process for the documentation of requirements that  ensures full documentation of the baseline requirements.
A process for ensuring formal peer reviews are planned and completed  for key products.
Guidance on tracking cost and schedule impacts of changes to  requirements for all projects.
Guidance on establishing and maintaining full bidirectional  requirements traceability.
2.
In addition, since BSM has ongoing projects that are developing and  managing requirements and the development of new policies and  procedures is not scheduled to be complete until March 2007, the  Commissioner should direct the Associate CIO for BSM to immediately  implement its draft policies while the final policies and procedures are  being developed.
section_title: Agency Comments
paragraphs: In providing written comments on a draft to this report, the Commissioner  of Internal Revenue agreed with our findings and stated that the report  provided a sound and balanced representation of the progress IRS has  made to date as well as work that remains to be completed. The  Commissioner also described the actions that IRS is taking to implement  our recommendations, including establishing a schedule to complete the  development of policies that address the areas of requirements elicitation,  documentation, verification and validation, and management. The  Commissioner’s written comments are reprinted in appendix III.
As agreed with your office, unless you publicly announce the contents of  this report earlier, we plan no further distribution until 30 days from the  date of this letter. At that time, we will send copies of this report to the  Chairmen and Ranking Members of other Senate and House committees  and subcommittees that have appropriation, authorization, and oversight  responsibilities for IRS. We are also sending copies to the Commissioner of  Internal Revenue, the Secretary of the Treasury, the Chairman of the BSM  Oversight Board, and the Director of the Office of Management and Budget.  Copies are also available at no charge on the GAO Web site at  http://www.gao.gov.
Should you or your offices have questions on matters discussed in this  report, please contact David A. Powner at (202) 512-9286 or  pownerd@gao.gov or Keith A. Rhodes at (202) 512-6412 or  rhodesk@gao.gov. Contact points for our Offices of Congressional  Relations and Public Affairs may be found on the last page of this report.  GAO staff who made major contributions to this report are listed in  appendix III.
section_title: Scope and Methodology
paragraphs: The objectives of our review were to assess (1) whether the Requirements  Management Office (RMO) has established adequate requirements  development and management policies and procedures and (2) whether the  Business Systems Modernization (BSM) has effectively used requirements  development and management practices for key systems development  efforts.
To assess the adequacy of BSM’s requirements development and  management policies and procedures, including IRS’s Enterprise Life Cycle  (ELC), we compared it against criteria based on industry standards and  best practices, including the Software Engineering Institute’s (SEI)  Capability Maturity Model Integration (CMMIsm) version 1.1. We also  reviewed draft policies and procedures provided by the RMO in February  2006 and compared them against this criteria. In addition, we interviewed  appropriate BSM officials to discuss the creation and goals of the RMO and  whether there were BSM requirements development and management  policies and procedures in place.
To assess whether BSM project teams effectively used requirements  development and management practices on its systems acquisitions, we  selected three BSM projects to review: (1) Modernized e-File (MeF) release  3.2, which is to be deployed in March 2006; (2) Filing and Payment  Compliance (F&PC) release 1.1, which was deployed in January 2006; and  Customer Account Data Engine (CADE) Individual Master File release 1.1,  which was deployed July 2004. We selected these investments because they  were (1) important to the goals and mission of the agency, (2) large-scale  development efforts with significant costs, and (3) at different points in  their development life cycles.
To evaluate whether each of the three projects had effectively used  requirements development and management practices for key systems  development efforts, we compared the project’s documentation and  processes against criteria based on industry standards and best practices,  including SEI’s CMMIsm version 1.1. The documentation reviewed for each  of the projects included requirements management plans, traceability  matrices, testing plans, baseline requirements, and other items. We also  interviewed the program officials from each of these three projects to  further clarify issues on their requirements development and management  activities.
Our work was performed from June 2005 to February 2006 in Washington  D.C., in accordance with generally accepted government auditing  standards.
section_title: Project Descriptions
paragraphs: The following are descriptions of the three projects we selected to review:  Modernized e-File (MeF) release 3.2, Filing and Payment Compliance  (F&PC) release 1.1, and Customer Account Data Engine (CADE) release  1.1.
subsections: section_title: Modernized e-File
paragraphs: In fiscal year 2004, BSM introduced the Modernized e-File (MeF) system,  which allows e-filing for tax-exempt organizations and large corporations  and reduces the time to process their tax forms. The goal for MeF is to  replace the current e-filing technology with a modernized, Internet-based  electronic filing system. MeF is also expected to result in an increase in the  use of electronic filing because it is efficient and easy to access, use, and  maintain.
Projected benefits of the MeF program are as follows:  Reduction in the BSM’s effort associated with receiving, processing,  manually entering data, and resolving data entry errors from paper  returns;  Reduction in system maintenance costs;  Savings in time and money for taxpayers and tax practitioners due to  not copying, assembling, and mailing a return; and  Sharing of tax and information return data electronically throughout  state agencies.
The MeF project is projected to provide the capability for Internet-based  filing of 330 different BSM forms. The following is table 2, describing MeF  releases deployed and their functionalities, followed by table 3, which  describes MeF financial data.
section_title: Filing and Payment Compliance
paragraphs: The Filing and Payment Compliance (F&PC) project is intended to improve  technologies and processes that support BSM’s compliance activities.  According to BSM, their collection operations rely on 30-year-old  technology and processes that are no longer compatible with the realities  of today’s taxpayer environment. F&PC plans to provide support for  detecting, scoring, and working nonfiler and payment delinquency cases. It  is to use advanced software to analyze tax collection cases and divide them  into cases that require BSM involvement and those that can be handled by  private collection agencies. Case attributes are to be identified, segmented,  and prioritized to select the individual taxpayer cases that have a greater  probability of paying the tax liabilities in full or through installment  agreements.
The F&PC project is also to serve as an inventory management system to  assign, exchange, monitor, control, and update delinquent taxpayer  accounts between the BSM Authoritative Data Source and the private  collection agencies with whom BSM will contract.
The F&PC project is expected to increase the following:  collection case closures by 10 million annually by 2014,  voluntary taxpayer compliance, and  BSM’s capacity to resolve the buildup of delinquent taxpayer cases.
The BSM intends to deliver an initial limited private debt collection  capability in January 2006. Full implementation of this aspect of the F&PC  is projected to be completed by January 2008 with additional functionality  to follow in later years. Following is table 4, describing F&PC releases  deployed and their functionality, followed by table 5, which describes  F&PC financial data.
section_title: Customer Account Data Engine
paragraphs: The Customer Account Data Engine (CADE), intended to replace BSM’s  antiquated tax administration system, is BSM’s highest priority project and  is intended to house tax information for more than 200 million individual  and business taxpayers. The CADE databases and related applications are  also to enable the implementation of other systems that will improve  customer service and compliance and allow the online posting and  updating of taxpayer account and return data.
The CADE project is intended to  generate refund notices, detect potential fraudulent transactions, and   replace the group of BSM tax master files with a single database—the  Tax Account Data Store;  accept, validate, and store taxpayer return and account data, along with  financial account activity data, such as tax payments, liabilities, and  installment agreements; and  enable future business application systems.
In July 2004 and January 2005, BSM implemented the initial releases of  CADE, which have been used to process Form 1040EZ returns. CADE  posted more than 1.4 million returns for filing season 2005 and generated  more than $427 million in refunds. In 2006, CADE is expected to expand the  number and type of returns beyond the Form 1040EZ. BSM is also  projecting that CADE will process 33 million returns during 2007.  Following is table 6, describing CADE releases deployed and their  functionality, followed by table 7 describing CADE financial data.
section_title: Comments from the Department of the Treasury
section_title: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Staff Acknowledgments
paragraphs: In addition to those named above, Neil Doherty, Nancy Glover,   George Kovachick, Tonia Johnson, Tammi Nguyen, Madhav Panwar, and  Rona Stillman made key contributions to this report.