Information Technology: Management Needs to Address Reporting of IRS Investments' Cost, Schedule, and Scope Information
Released: 2015-02-25T00:00:00


Highlights: section_title: Why GAO Did This Study
paragraphs: IRS relies extensively on IT systems to annually collect more than $2 trillion in taxes, distribute more than $300 billion in refunds, and carry out its mission of providing service to America's taxpayers in meeting their tax obligations. For fiscal year 2014, IRS expected to spend about $2.4 billion on IT. Given the size and significance of IRS's IT investments and the challenges inherent in successfully delivering these complex IT systems, it is important that Congress be provided reliable cost, schedule, and scope information to assist with its oversight responsibilities.

Accordingly, GAO's objectives were to (1) evaluate IRS's efforts to address prior GAO recommendations for improving the reliability and reporting of cost, schedule, and scope information; (2) summarize the reported cost, schedule, and performance of IRS's major IT investments; and (3) assess the status and plans of selected investments.

To do so, GAO analyzed quarterly reports and reviewed information on cost and schedule from October 2013 to September 2014, interviewed program officials, and analyzed documentation for selected investments.

section_title: What GAO Found
paragraphs: The Internal Revenue Service (IRS) has made limited progress in improving the reliability and reporting of cost, schedule, and scope performance information–the agency has partially implemented two of GAO's five prior recommendations, but not yet addressed the remaining three (see table). IRS's implementation of these recommendations is critical in ensuring that Congress receives the reliable information it needs for effective oversight and decision making.

Key: ● Fully—the agency provided evidence that it fully addressed the recommendation. ◐ Partially—the agency provided evidence that it has addressed the recommendation to some extent. ◌ Not addressed—the agency did not provide any evidence that it addressed the recommendation.

Most of IRS's major information technology (IT) investments were reported as meeting cost and schedule goals. Specifically, 11 of 17 investments were reportedly within 10 percent of cost estimates, and 13 were within 10 percent of schedule estimates between October 2013 and September 2014. In addition, the agency reported “green” ratings for investments instead of their previous “yellow” ratings for Chief Technology Officer summary-level risk assessments. It is important to note that these ratings are not provided for 6 investments for which IRS provides detailed reporting to Congress. Providing summary-level risk ratings for all major investments would improve the visibility into changes in investment risk, and provide Congress with the information to more easily determine the investments requiring greater attention.

Selected investments experienced variances from initial cost, schedule, and scope plans that were not transparent in congressional reporting because IRS has yet to address GAO's prior recommendations. Specifically, the Return Review Program has so far exceeded planned costs by $86.5 million and has yet to deliver functionality that was scheduled for September 2012, and a key phase of Customer Account Data Engine 2 was developed 10 months late and at $183.6 million more than planned. However, none of these variances were clearly identified in congressional reporting. In addition, the consolidated reports on the status of testing for the Affordable Care Act Administration investment are not comprehensive, making it difficult to determine whether all required testing is being performed.

section_title: What GAO Recommends
paragraphs: GAO is making three recommendations to improve the reliability and reporting of investment performance information and management of selected investments. IRS agreed with two recommendations but disagreed with the third related to the reporting of test results. GAO continues to believe it is still warranted.

section_title: Letter
paragraphs: The Internal Revenue Service (IRS) relies extensively on information  technology (IT) systems to annually collect more than $2 trillion in taxes,  distribute more than $300 billion in refunds, and carry out its mission of  providing service to America’s taxpayers in meeting their tax obligations.  For fiscal year 2014, IRS expected to spend about $2.4 billion for IT  investments; this is about 21 percent of IRS’s total budget for the year.

Given the significant amount spent on IRS’s IT investments and the  challenges inherent in successfully delivering them, it is important that  Congress be provided ongoing, accurate, and objective information on  the progress toward completion and the risks facing IRS’s projects.  Accordingly, the Senate Appropriations Committee directed GAO to  review the cost and schedule performance of IRS’s major IT  investments. Specifically, our objectives were to (1) evaluate IRS’s  efforts to address our recommendations for improving the reliability and  reporting of cost, schedule, and scope information; (2) summarize the  reported cost, schedule, and performance of IRS’s major IT investments;  and (3) assess the status and plans of selected investments.

To address our first objective, we analyzed the four quarterly reports on  the performance of IT investments submitted by IRS to the appropriations  committees and us between December 2013 and September 2014. We  also reviewed documentation of training provided to IRS staff regarding  monthly reporting of investment performance information, and cost  estimation procedures that IRS stated addressed the calculation of  projected cost and schedule amounts. To address our second objective,  we analyzed documentation identifying the cost and schedule  performance of IRS’s major IT investments from October 2013 to  September 2014, and operational performance information as of  September 2014. We identified significant recurring cost, or schedule  variances and followed up with IRS to obtain the reasons for these  variances. Lastly, we reviewed the four quarterly reports on the  performance of IT investments submitted by IRS to the appropriations  committees and us between December 2013 and September 2014, to  identify the summary-level risk ratings assigned by the Chief Technology  Officer to major IT investments, and analyzed these ratings to identify  trends.

For our third objective, we selected the Return Review Program (RRP),  Customer Account Data Engine 2 (CADE 2), and Information Reporting  and Document Matching (IRDM) investments because the cost, schedule,  or scope of these investments had changed from initial plans. In addition,  we selected the Affordable Care Act Administration (ACA) investment due  to its criticality to the 2015 tax filing season and the significant amount of  resources expected to be expended. For RRP, CADE 2, and IRDM, we  interviewed program officials and analyzed documentation such as  performance work statements, business cases, and baseline change  requests, to determine the initial and revised cost, schedule, and scope  for these investments. For ACA, we interviewed program officials and  analyzed program and testing documentation to identify the deployment  plan for the investment, as well as the extent to which testing has been  planned, conducted, and reported on for two new releases that are  expected to be implemented for the 2015 tax filing season.

We conducted this performance audit from June 2014 to February 2015,  in accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives. Details on our objectives,  scope, and methodology can be found in appendix I.
subsections: section_title: Background
paragraphs: The mission of IRS, a component of the Department of the Treasury  (Treasury), is to provide America’s taxpayers top quality service by  helping them understand and meet their tax responsibilities and enforcing  the federal tax laws with integrity and fairness to all. In carrying out its  mission, IRS annually collects over $2 trillion in taxes from millions of  individual taxpayers and numerous other types of taxpayers and  manages the distribution of more than $300 billion in refunds. To guide its  future direction, the agency has two strategic goals: (1) deliver high  quality and timely service to reduce taxpayer burden and encourage  voluntary compliance and (2) effectively enforce the law to ensure  compliance with tax responsibilities and combat fraud.

IT plays a critical role in enabling IRS to carry out its mission and  responsibilities. For example, the agency relies on information systems to  process tax returns, account for tax revenues collected, send bills for  taxes owed, issue refunds, assist in the selection of tax returns for audit,  and provide telecommunications services for all business activities,  including the public’s toll-free access to tax information.

IRS’s fiscal year 2014 budget was $11.3 billion. Of this amount, IRS  expected to spend about $2.4 billion on IT investments. IRS expected to  fund 19 major investments at a cost of about $1.7 billion, or 71 percent, of  the total IT request, and 135 non-major investments at a cost of about  $700 million, or 29 percent, of the total IT request. For IRS, a major  investment is one that costs $10 million in either the current year or  budget year, or $50 million over the 5-year period extending from the prior  year through 2 years after the budget year. Table 1 provides high-level  descriptions of IRS’s 19 major IT investments and appendix II provides  detailed profiles of 7 investments critical to IRS’s mission for which we  performed in-depth reviews in recent audits (ACA, CADE 2, e-Services,  IRDM, IRS.Gov, Modernized e-File, RRP).
subsections: section_title: IRS Is Required to Report Quarterly to Congress on the Status of Its Major IT Investments
paragraphs: The conference report accompanying the Consolidated Appropriations  Act, 2012, directed IRS to submit quarterly reports on the cost and  schedule performance of its major IT investments to the Committees on  Appropriations and GAO no later than mid-April 2012. These quarterly  reports are to include detailed information on selected investments,  including their purpose and life-cycle stage, reasons for cost and  schedule variances, risks and mitigation strategies, expected  developmental milestones to be achieved, and costs to be incurred in the  next quarter. IRS’s current reporting provides detailed information on  eight investments, including six major investments that we have included  in our reviews: CADE 2, e-Services, IRDM, IRS.Gov, MeF, and RRP.

section_title: GAO and the Treasury Inspector General for Tax Administration Have Reported on IRS’s Major IT Investments
paragraphs: GAO and the Treasury Inspector General for Tax Administration (TIGTA)  have previously reported on IRS’s major IT investments.

We reported in June 2012 that while IRS reported on the cost and  schedule of its major IT investments and provided chief information  officer ratings for them, the agency did not have a quantitative   We  measure of scope–a measure that shows functionality delivered.reported that having such a measure is a good practice as it provides  information about whether an investment has delivered the  functionality that was paid for. We recommended that the  Commissioner of Internal Revenue develop a quantitative measure of  scope, at a minimum for its major IT investments, to have more  complete information on the performance of these investments. IRS  agreed with our recommendation at the time we made it. In March  2014, IRS reported that it had practices and processes in place that  addressed this recommendation, including quarterly reports to  Congress, and a baseline change request process. However, we did  not believe these practices addressed the recommendation, as  neither approach included a quantitative measure. For this reason, we  believed the recommendation was still warranted.

We noted in April 2013 that the majority of IRS’s major IT investments  were reportedly within 10 percent of cost and schedule estimates and  eight major IT investments reported significant cost and/or schedule  variances. We also reported that weaknesses existed, to varying  degrees, in the reliability of reported cost and schedule variances, and  key risks and mitigation strategies were identified. As result, we made recommendations for IRS to improve the reliability of reported  cost and schedule information by addressing the identified  weaknesses in future updates of estimates. We also recommended  that IRS ensure projects consistently follow guidance for updating  performance information 60 days after completion of an activity and  develop and implement guidance that specifies best practices to  consider when determining projected amounts. IRS agreed with three  of our four recommendations and partially disagreed with the fourth  recommendation related to guidance on projecting cost and schedule  amounts. The agency specifically disagreed with the use of earned  value management data as a best practice to determine projected  cost and schedule amounts, stating that the technique was not part of  IRS’s current program management processes and the cost and  burden to use it outweigh the value added. While we disagreed with  IRS’s view of earned value management because best practices have  found that the value generally outweighs the cost and burden of  implementing it, we provided it as one of several examples of  practices that could be used to determine projected amounts. We also  noted that implementing our recommendation would help improve the  reliability of reported cost and schedule variance information, and that  IRS had flexibility in determining which best practices to use to  calculate projected amounts. For those reasons, we believed our  recommendation was still warranted.  In September 2013, TIGTA reported on CADE 2 development  challenges and changes to the planned schedule for this investment.  TIGTA reported, among other things, that the CADE 2 database  cross-functional triage team had effectively managed and resolved  more than 1,000 data defects.that the downstream system interfaces had not been implemented  due to data quality issues and the implementation date of these  interfaces was revised to January 2014.

However, TIGTA’s review determined

We reported in April 2014, that 6 of IRS’s 19 major IT investments  were within 10 percent of cost and schedule estimates during fiscal  year 2013; however, the reported variances were for the fiscal year  only, and we therefore noted that IRS’s reporting would be more  meaningful if supplemented with cumulative cost and schedule  variances for the investments or investment segments. In addition, the reported variances for selected investments were not always  reliable because the projected and actual cost and schedule amounts  on which they depend had not been consistently updated in  accordance with OMB and Treasury reporting requirements. Further,  IRS was not working on developing a quantitative measure of scope  (i.e., functionality) as we recommended in 2012, and we noted that  reporting qualitatively in congressional reporting until a quantitative  measure is developed would help provide Congress with a complete  picture of the agency’s performance in managing its major  investments. Lastly, IRS continued to lack guidance that included best  practices for calculating projected cost and schedule amounts. We  made three recommendations for IRS to report more comprehensive  and reliable cost and schedule information and improve the  transparency of reported scope information for its major investments.  IRS agreed with our recommendations and stated it believed it had  addressed our recommendation to report cumulative investment and  investment segment cost and schedule information in the quarterly  reports to Congress, as well as our prior recommendation to develop  a quantitative measure of scope; we disagreed, however, and  maintained our recommendations.  In September 2014, TIGTA reported on challenges faced by IRS in  implementing the IRDM Case Management project. More specifically,  TIGTA noted that after a year of user acceptance testing, IRS officials  acknowledged that the IRDM Case Management project could not  effectively process business cases containing underreported income  and could not be deployed into the IRS production environment;  TIGTA identified insufficient project requirements as contributing to  these challenges. In addition, IRS officials stated that budget  constraints and difficulties encountered during user acceptance  testing resulted in IRS “strategically pausing” development of the  IRDM Case Management project. In response to TIGTA’s report,  IRS’s Chief Technology Officer stated that in January 2014, IRS  decided to strategically pause development of the IRDM Case  Management project due to budget constraints and the inability to  certify that the ongoing case management functionality deployment  would not have an adverse impact on taxpayers.

section_title: IRS Has Made Limited Progress in Implementing Prior Recommendations to Improve Reliability and Reporting of Cost, Schedule, and Scope Information
paragraphs: IRS has made limited progress in improving the reliability and reporting of  cost, schedule, and scope performance information: it has partially  implemented two of our five related recommendations and not yet  addressed the remaining three. IRS’s implementation of these  recommendations is critical in ensuring that Congress receives the  reliable information it needs for effective oversight and decision making.  Table 2 identifies the status of IRS’s efforts to address the  recommendations.
subsections: section_title: IRS Has Taken Action to Improve the Timeliness of Reported Performance Information for Completed Investment Activities
paragraphs: In April 2013, we reported that the cost and schedule performance  information for the completed activities for six selected investments was  updated within the 60-day time frame required by Treasury guidance in  77 percent of the cases. While the number of activities expected to be  completed was relatively low and IRS had updated the variance  calculations for these activities in the majority of the cases, we noted that  ensuring that updated actual information is consistently reported within  the required 60-day time frame would strengthen the reliability of their  variances and provide information that better reflects their performance.  Consequently, we recommended that IRS ensure its projects consistently  follow guidelines for updating performance information 60 days after  completion of an activity.

Treasury and IRS subsequently took actions to address our  recommendation. Specifically, starting in fiscal year 2014, Treasury  addressed the timeliness issue for schedule calculations by having the  monthly reporting system automatically calculate a variance based on the  current date for any activity where the planned completion date had  passed and investment staff have not provided an actual figure within 45  days. For cost, in June 2014, officials in IRS’s Strategy and Planning  group—which is responsible for overseeing monthly variance reporting— stated that they have been working closely with investment staff and  program managers to ensure that reporting is completed within the 60-  day requirement.

We reviewed the cost and schedule performance information for the six  selected investments for fiscal year 2014 and found that the actions taken  have resulted in actual cost and schedule amounts for completed  activities being updated within the 60-day time frame required in 86  percent of the cases. While this is an improvement from the 77 percent  we previously reported, IRS should continue its efforts to ensure full  compliance with Treasury’s guidance and thereby provide reliable  information on which to gauge its performance in meeting cost and  schedule goals.

section_title: IRS Has Begun to Take Steps to Address the Reporting of Performance Information for In-Process Investment Activities
paragraphs: In April 2014, we reported that IRS did not consistently report updated  variances for in-process investment activities for six investments in fiscal  year 2013 even though OMB and Treasury require cost and schedule  variances to be updated on a monthly basis. This was partly due to an  inconsistent understanding among investment staff of the information that  was to be included in the monthly reporting. As a result, we  recommended that IRS ensure that projected cost and schedule  variances for in-process activities are updated monthly consistent with  OMB and Treasury reporting requirements by ensuring investment staff  have a consistent understanding of the information to be included in  monthly reporting.

In response to our recommendation, IRS’s Investment Management and  Control office provided training in October 2014, which focused on,  among other things, the monthly update of investment performance  information. We believe this training will help to ensure investment staff  have a consistent understanding of the information to be included in  monthly reporting as the training outlines the specific information that is to  be reviewed or updated for in-process activities. However, since the  training was provided in October 2014, there have not yet been enough  monthly reports to determine the extent to which this training has  improved monthly reporting of variances for in-process activities.

Adherence to IRS’s training on monthly performance reporting should  help to ensure investments’ cost and schedule variances are updated in  accordance with OMB and Treasury guidance, and contribute to  producing reliable information on which to gauge IRS’s performance.

section_title: IRS Has Not Developed Guidance for Determining Projected Cost and Schedule Amounts for In- Process Investment Activities
paragraphs: In April 2013, we found that IRS had determined variances using  —which comprised  projected cost and schedule for in-process activities75 percent of all its activities. However, Treasury’s guidance, which IRS  follows, did not specify how projected amounts should be determined  when actual amounts are not available. We therefore recommended that  IRS develop guidance for determining projected amounts.

In response, IRS stated that the estimate variance reporting performed by  its Estimation Program Office applies the best practices we previously  recommended, and the practices used are documented in its July 2014  cost and schedule variance reporting procedure. We reviewed this  document and found that, while it described the methodology for revising  an estimate, it does not address the calculation of projected cost and  schedule amounts used for the monthly reporting of cost and schedule  variances for in-process activities, which was the subject of our  recommendation. At the conclusion of our review, officials sought  clarification on what was needed to address our recommendation and  agreed that the action taken did not address it. Developing and  implementing the recommended guidance should provide greater  assurance that projected amounts, when reported, are determined  consistent with best practices and therefore more reliable. This is  particularly important given the high percentage of reported investment  activities that we noted were in-process.

section_title: IRS Has Not Taken Steps to Report Cumulative Investment Performance Information
paragraphs: In April 2014, we reported that IRS’s reporting of cost and schedule  information in the quarterly reports to Congress would be more  meaningful for determining whether the agency is effectively managing its  investments if it included cumulative cost and schedule variances for the  investments or investment segments, consistent with OMB’s guidance for  measuring progress towards meeting investment goals. We noted that  cost and schedule variances were for the fiscal year only in that they  provide cost and schedule variance information for all projects and  activities underway in any portion of the fiscal year.year focus did not provide cumulative cost and schedule information at  the investment or investment segment level because it did not account for  activities that were completed in previous fiscal years. Accordingly, we  recommended that IRS report cumulative performance information at the   However, the fiscal  investment or investment segment level. At that time, the IRS  Commissioner stated that the agency agreed with our recommendation  but believed it had already been addressed in quarterly reports to  Congress. We noted that while the reports provide cumulative  information, it is for the fiscal year only, not for the investment as  recommended, and we therefore maintained our recommendation.

In June 2014, IRS officials stated they believed the investment  information reported in the Office of Management and Budget exhibit 300  addressed our recommendation and, therefore, they had not taken  additional steps. However, the reported cost and schedule variances in  the exhibit 300 are for the fiscal year only, and as a result, we believe our  recommendation is still warranted.

Providing Congress with cost and schedule information at the useful  segment level—in addition to the current fiscal year reporting—in the  quarterly reports would provide a more meaningful gauge of whether  investments are meeting cost and schedule performance goals.

section_title: IRS Has Not Taken Steps to Provide Scope Information for Selected Investments
paragraphs: In 2012, we reported that IRS did not have a quantitative measure of  scope (i.e. functionality delivered) that would provide a measure of  whether an investment delivered the functionality that was paid for and  recommended that the agency develop the measure, at a minimum, for its  major IT investments. At the time, IRS agreed with the recommendation  but stated that it had other methods in place to document delivered  functionality of a project throughout the life cycle. We agreed that the  methods identified addressed project functionality, but they did not  provide a quantitative measure of performance. In April 2014, seeing  that IRS had not made progress on developing a quantitative measure of  scope, we recommended the agency report qualitative scope information  in the interim.

IRS responded that it agreed with the recommendation and had practices  and processes in place to assess and report on the delivery of scope in  conjunction with cost and schedule management, and therefore, IRS had  not taken any additional steps to address our recommendation; however,  we did not believe that these practices and processes addressed our  recommendation. As of June 2014, IRS continued to assert that it had  addressed the recommendation and therefore did not take any additional  steps. Officials noted that the information reported in the Office of  Management and Budget exhibit 300 included information on changes in  investment scope. However, this reporting does not provide a quantitative  measure of scope or qualitative information showing how delivered scope  compares to what was planned.

Until IRS reports on progress in meeting scope in its quarterly reporting to  Congress, Congress may lack important information that it needs to  determine the extent to which the investments are delivering the  functionality that was paid for. This is particularly important given the  major changes in development highlighted in the latter portion of this  report.

section_title: IRS Reported Most Investments Meeting Cost, Schedule, and Operational Performance Goals, but Facing Increased Risks
paragraphs: Most of IRS’s major IT investments reportedly met cost and schedule  goals, with 11 of 17 investments within 10 percent of cost estimates, and  13 of 17 investments within 10 percent of schedule estimates. It is  important to note that the cost and schedule information was not updated  for two investments however, IRS did not consistently indicate so in its  reports to Congress. Consistently disclosing when reported information is  not updated would provide Congress and other decision makers with  improved information for oversight and decision-making purposes.

IRS also reported “green” ratings for investments instead of their previous  “yellow” ratings for Chief Technology Officer summary-level risk  assessments. However, IRS does not provide these ratings for the six  investments for which it provides detailed information in the quarterly  reports to Congress. Providing summary-level risk ratings for all major  investments would improve the visibility into changes in investment risk,  and provide Congress with the information to more easily determine the  investments requiring greater attention. Finally, of the 85 operational  performance metrics associated with the 17 major investments reporting  operational performance information, IRS reported meeting approximately  73 (86 percent) of these metrics.
subsections: section_title: Most Investments Were Reportedly Within Cost and Schedule Goals
paragraphs: According to IRS, 11 of 17 IT investments were within 10 percent of cost  estimates between October 2013 and September 2014, and 13 of 17  investments were within 10 percent of schedule estimates between  October 2013 and September 2014. While IRS reports on the cost and  schedule variance for its 19 major investments, the reports for two  investments (IRDM and RRP) were not updated to reflect actual  performance throughout the fiscal year. As illustrated in figure 1, of the six  investments that reported significant cost variances (equal to plus or  minus 10 percent variance from cost goals), four were significantly under  planned costs for at least 1 month during fiscal year 2014, one investment  reported being over cost, and one investment reported being, at different  times, both under and over cost during this period.

Three investments–ACA, e-Services, and IRS Telecommunications  Systems and Support–reported significant cost variances for a period of 3  or more consecutive months. IRS reported several reasons for these  variances, including refinement of processes for allocating costs, fewer  investment staff working on the investment during the 2013 government  shutdown, overestimation of required contractor support, and reduction of  planned funding.

In addition, as illustrated in figure 2, one investment reported being  significantly ahead of schedule for at least 1 month during fiscal year  2014, while three investments reported being significantly behind  schedule during this period.

As previously mentioned, Treasury and OMB guidance require cost and  schedule variances to be updated on a monthly basis. However, IRS did  not update information on cost and schedule variances to reflect actual  performance for their RRP and IRDM investments in its reports to  Congress. Officials said that updated cost and schedule performance  information for these investments was not included following pauses in  their development (which occurred in January 2014 for IRDM and  February 2014 for RRP) and during approval of baseline change  requests. IRS officials stated they did not yet know how to include the  pauses in development in their reports and that they had been instructed  by Treasury not to update monthly performance information until the  change requests had been approved. However, instances where  information was not updated were not disclosed in a consistent manner  for all investments. Specifically, while IRS identified such instances for  RRP, it did not provide similar disclosure for IRDM following its  development pause. Consistently disclosing reasons for why monthly  updates are not being made (such as during the baseline change request  approval process) would be helpful in providing decision makers with the  information they need for oversight purposes.

section_title: IRS Reported Increased Risks for Selected Investments
paragraphs: During the third quarter of fiscal year 2014, IRS reported increased risks  for the 13 investments for which it provides summary-level Chief  Technology Officer risk assessments to Congress. Specifically, while the  13 investments had a risk rating of “green” during the second quarter of  fiscal year 2014, 12 of these investments reported a risk rating of “yellow”  during the third quarter of fiscal year 2014, and 1 investment reported a  risk rating of “red.” According to the Deputy Chief Information Officer for  Strategy and Modernization, the Chief Technology Officer and Deputy  Chief Information Officers meet quarterly to make a broad assessment of  the major IT investments, and as a result, assign summary-level risk  ratings for 13 of the major IT investments. This assessment is based on  these officials’ knowledge of each of the major investments, as well as an  assessment of six key performance indicators (cost, schedule, scope,  risk, organizational readiness, and technical).

A reason IRS provided for the change in risk ratings for its major IT  investments was funding constraints as a result of additional legislative  mandates, such as the ACA and FATCA investments, which IRS noted it   In addition, IRS  does not receive funding from Congress to implement.noted that it has had to reallocate staffing to these investments, which  has created a skill set gap for other investments. To address this, IRS  stated that it is currently creating a skill set inventory to specifically  identify gaps between available and required skill sets.

It is important to note that, while IRS identified increased risks for the 13  major IT investments via its Chief Technology Officer risk ratings for the  first time in quarter three of fiscal year 2014, the assessments were not  indicative of new risks. Rather, they better reflected risks IRS had  previously shared with us during quarterly briefings. During the fourth  quarter of fiscal year 2014, the risk rating for 6 of the investments  improved from “yellow” to “green.” IRS’s Deputy Chief Information Officer  for Strategy and Modernization explained that this happened because the  agency was able to draw resources from infrastructure investments  deemed less critical for the upcoming filing season to address the risks  associated with most of the investments previously rated “yellow.” This  explains the “red” rating for the infrastructure investments in the fourth  quarter, as illustrated in figure 3 below.

We have previously reported on the importance of providing summary- level risk ratings for major IT investments. Specifically, we have noted  that such ratings improve the visibility into changes in the risk level of  investments over time. While IRS provides summary-level Chief  Technology Officer risk assessment ratings for 13 investments in  quarterly reporting to Congress, it does not provide such ratings for the 6  investments for which it reports detailed information–CADE 2; e-Services;  IRDM; IRS.Gov; MeF; and RRP. While the detailed information on the 6  investments is consistent with congressional reporting requirements,  supplementing it with Chief Technology Officer summary-level risk  assessment ratings would improve the visibility into risks faced by these  investments, and provide Congress with the information to more easily  determine the investments requiring greater attention. Figure 3 shows the  Chief Technology Officer risk assessment ratings for the four quarters of  fiscal year 2014.

section_title: Most Major IT Investments Reported Meeting Operational Performance Goals
paragraphs: According to OMB, operational performance metrics are used to examine  the performance of an investment in operation and demonstrate that the  investment is meeting the needs of the agency, delivering expected  value, or being modernized and replaced consistent with the agency’s  enterprise architecture. As of September 2014, IRS had reported on the  operational performance for 17 of its 19 major investments. IRS  establishes operational metrics and associated targets for its investments,  and on a quarterly, monthly, or annual basis reports on its performance in  meeting the targets. The operational metrics established for investments  include, for example, percentage of scheduled system availability,  percentage of individual tax returns processed electronically, and the  percentage of refunds processed daily.

As illustrated in figure 4, of the 85 operational performance metrics  reported with associated actuals, IRS reported meeting approximately 73  (86 percent) of these metrics. With respect to the 12 operational  performance metrics that were not met, the difference between the target  and actual performance was generally insignificant. For example, half of  the metrics were within 5 percent of the target.

section_title: Variances from Selected Investments’ Initial Cost, Schedule, and Scope Goals Have Not Been Transparent and Reporting of ACA Testing Status Is Not Comprehensive
paragraphs: Selected investments experienced variances from initial cost, schedule,  and scope goals that were not transparent in congressional reporting  because IRS has yet to address our prior recommendations for reporting  at the investment level and on progress in delivering scope. Specifically,  RRP has so far exceeded planned costs by $86.5 million and has yet to  deliver functionality that was scheduled for September 2012, in large part  due to the need to implement new technology and a lack of adequate  resources, including contracting expertise and staff; a key phase of CADE  2 was developed 10 months late and at $183.6 million more than  planned; and the IRDM Case Management project was cancelled.  However, these variances were not all included in congressional  reporting. In addition, the reports on the status of testing for the ACA  investment are not comprehensive, making it difficult to determine  whether all required testing is being performed.
subsections: section_title: IRS Delivered Less RRP Functionality at Higher Cost and Delayed Schedule
paragraphs: IRS delivered less functionality than planned for the RRP investment, and  did so at a higher than planned cost and behind schedule. Specifically,  IRS exceeded initial planned costs for this investment by approximately  $86.5 million and has yet to complete the first phase of the investment,  which was originally planned to be delivered in September 2012.

As early as May 2010, IRS issued several contracts to, among other  things, plan and develop four transition states to complete the RRP  investment; these contracts had a total planned cost of $57.5 million.  Figure 5 identifies the current and historical development plans for the  RRP investment.

The planned schedule and functionality for the four RRP transition states  are identified in table 3.

In March 2012, a baseline change request was approved for RRP that  included a revision to the planned completion dates for Transition States  1 and 2 to December 2013 and 2014, respectively. In addition, the  planned cost for the RRP investment was revised to $136.2 million, an  increase of approximately $79 million. According to IRS, these changes to  initial plans were a result of IRS’s decision to implement new technology  for delivering the RRP investment. More specifically, IRS began  implementation of the RRP investment using existing technologies;  however, IRS determined that new technology would be better suited to  meet the goals of the investment.

In February 2014, after developing most of the planned functionality for  Transition State 1–a senior RRP official estimates about 70 percent– IRS’s Executive Steering Committee made a decision to pause further  development of this investment. According to IRS officials, factors  contributing to this decision included budget constraints, as well as  uncertainty about next steps from a business and a technology  perspective, and the need to ensure alignment of RRP with the new  senior leadership’s strategic vision for identity theft and fraud detection.

In March 2014, IRS reported delivering the following Transition State 1  functionality:  Improvements in data analytics and linked return analysis above  current EFDS capabilities in order to detect more fraud.

Leveraged new Massive Parallel Processing technology, which IRS  noted has proven itself in data analysis, performance, and scoring  improvements in analyzing 3 years of taxpayer data.

Entity-based Data Model with a 3-year view of tax filer’s data.

Ability to add or modify rules and models in current processing year  based on current fraud patterns.

In addition, in April 2014, IRS launched a limited deployment of one of  RRP’s planned fraud detection capabilities–the capability to detect  identity theft in filed tax returns. IRS plans to use the RRP identity theft  functionality in conjunction with the Electronic Fraud Detection System  (the fraud detection system RRP is expected to eventually replace) for all  tax returns filed during the 2015 tax filing season. IRS also reported  beginning requirements development activities for RRP Transition State  2.

In September 2014, IRS proposed additional changes to the RRP  investment. More specifically, it revised the planned completion dates for  Transition States 1 and 2 to March 2015 and 2016, respectively. In  addition, the planned cost for the RRP investment was revised to $226.9  million, an increase of approximately $91 million. IRS identified several  reasons for these changes in plans to include, among other things: lack of  experience in integrating new technology required for RRP  implementation; the need for higher levels of contracting expertise; and  lack of staff to support the entire planned scope of RRP due to budgetary  constraints and increased costs.

As illustrated in figure 6, IRS reported spending approximately $144  million for the RRP investment through fiscal year 2014. Thus far, this  amount exceeds the initial planned cost for the investment by $86.5  million.

With respect to future development of the RRP investment, IRS stated  that it has begun work on a plan for re-starting development which is  heavily influenced by IRS’s Small Business/Self Employed and Wage and  Investment Concept of Operations (issued in July 2014), and an IT  technical roadmap that is currently being developed. IRS’s Small  Business/Self Employed and Wage and Investment Concept of  Operations identifies refund fraud and identity theft, as key drivers for  transforming the agency’s compliance efforts and services.

Although IRS has thus far exceeded the initial planned cost for the RRP  investment by $86.5 million, the agency reported a zero percent cost  variance for this investment in its fiscal year 2014 fourth quarter reporting  to Congress. Further, while IRS noted that it had delivered about 70  percent of the planned functionality for Transition State 1 of the RRP  investment that was planned for September 2012 in March 2014, this was  not identified in congressional reporting.

If IRS implemented our prior recommendations relative to cumulative  reporting of performance information, and reporting of quantitative scope  information, as previously mentioned, the variances from cost, schedule,  and scope plans identified for RRP would be more transparent in  congressional reporting.

section_title: IRS Has Delivered a Key Phase of CADE 2; however, Development of this System Has Been More Costly and Taken Longer than Planned
paragraphs: IRS has delivered a key phase of its modernized tax processing system;  however, in doing so, the agency exceeded planned costs by $183.6  million and fell behind schedule by 10 months; this included an unplanned  transition state with an associated cost of $101.1 million.

Figure 7 identifies the current and historical development plans for the  CADE 2 investment.

In 2008, IRS began defining a new strategy–CADE 2–that was intended  to deliver improved individual tax processing sooner, and move to a  single tax processing database. As shown in table 4, IRS planned to  deliver the CADE 2 investment through the completion of two transition  states and a target state.

In 2012, IRS completed a cost estimate for Transition State 1 of the  CADE 2 investment; this cost estimate was $315 million.

IRS reported completing functionality for the daily processing of individual  taxpayer returns in January 2012, and completing Transition State 1 in  November 2012, at a cost of $397.5 million; Transition State 1 was  completed 10 months behind planned schedule, and in excess of planned  costs by $82.5 million. Further, while IRS reported the completion of  Transition State 1, this transition state completed “conditionally” meaning  that the investment was approved to proceed to the next phase with  outstanding issues remaining to be addressed.

In June 2013, IRS submitted a baseline change request to create a new  transition state–Transition State 1.5–to address unfinished work from  Transition State 1. More specifically, this unfinished work included  ongoing data assurance, performance tuning, and downstream systems  efforts to prepare the CADE 2 database for filing season 2014 production;  IRS completed this transition state in July 2014. IRS officials stated that  the creation of this transition state did not affect the overall schedule for  the CADE 2 investment; however, it was accompanied by $101.1 million  in unplanned costs–$69.7 million in fiscal year 2013, and $31.4 million  planned for fiscal year 2014. IRS officials stated that investment funding  allocated for future work on Transition State 2 was used to fund the  unplanned Transition State 1.5 activities.

IRS began work on Transition State 2 in October 2010, and as of  September 2014, expected to complete this transition state by March 31,  2015. However, IRS noted that this planned completion date is likely to  change as soon as a revised schedule estimate is completed for this  transition state.

IRS’s delivery of CADE 2 Transition State 1 10 months behind its initial  planned completion date and in excess of initial planned costs by $183.6  million is not identified in congressional reporting. More specifically, IRS’s  congressional reporting identifies cost and schedule performance for a  12-month period of time, and does not compare current investment  performance to initial plans, as we have done in this report. Further, while  IRS’s fiscal year 2014 fourth quarter reporting to Congress identifies the  scope delivered for CADE 2 Transition State 1 during fiscal years 2009  through 2012, the reporting does not include a quantitative measure of  scope, or qualitatively show how the delivered scope compares to what  was planned for this transition state.

Similar to RRP, the CADE 2 schedule delays and challenges in meeting  planned costs would be more transparent in congressional reporting if it  contained cumulative reporting of performance information and reporting  of quantitative scope information.

section_title: IRS Has Cancelled the IRDM Case Management Project
paragraphs: IRS has cancelled its IRDM Case Management project—one of five  projects that make up the IRDM investment—due to budget constraints,  and is instead considering using an enterprisewide case management  solution. Table 5 identifies the initial planned cost, schedule, and scope  for the IRDM Case Management project.

According to IRS, the IRDM Case Management project began beta  testing in January 2013; however, further execution of the IRDM Case  Management project was cancelled in January 2014, and IRS noted that  this project would be shut down after the existing cases being worked  within the application were completed. According to officials, IRS made a  decision to investigate an off-the-shelf system for case management that  could be used as an enterprise-wide common service at IRS. IRS noted  that it has held three technical demonstrations to identify the extent to  which a vendor-provided, off-the-shelf solution would meet the enterprise- wide need, and future development of a case management tool will be  done using EntelliTrak technology. IRS officials stated they plan to  execute enterprise case management solutions as soon as budget  resources become available. As previously mentioned, TIGTA identified  challenges during user acceptance testing of the IRDM Case  Management project; however, IRS officials stated that these challenges  were not a contributing factor in the agency’s decision to pause  development of this project. As of October 2014, IRS reported spending  $16.2 million on the IRDM Case Management project—$8.8 million for  IRDMCM and $7.4 million for IRDMCM R2/Release Content Management  Plan.

section_title: IRS Is Performing Testing of ACA Releases; However, Reporting of Efforts Is Not Comprehensive
paragraphs: ACA encompasses the planning, development, and implementation of IT  systems needed to support IRS’s tax administration responsibilities  associated with certain provisions of the Patient Protection and Affordable  Care Act. IRS is developing this investment in 24 releases–12 which are  in production, 1 that is in production/in progress, 6 that are in progress,  and 5 that are in planning. IRS’s release plan for this investment is shown  in table 5.

Releases 5.0 and 6.0 (shaded in table 6) include development work that  is critical in implementing ACA requirements for the 2015 tax filing  season. The work associated with these releases impacts 66 IRS  systems via a system modification or by building a new system.

According to best practices, software testing should be guided by an  organizational test strategy that defines different levels of testing required  such as component, system, integration, and acceptance level testing. In  addition, the strategy should address how testing is to be managed and  results reported.that defines various levels of testing for ACA and has also assigned  responsibility for testing to various organizations within IRS.

Consistent with these practices, IRS has a test strategy  ACA systems testing is performed by each of the following organizations  within IRS, depending on the type of system work required. According to  IRS officials, these organizations coordinate testing activities during  systems integration testing.

The Enterprise Systems Testing group is responsible for performing  testing on systems that require modification to existing system  functionality. According to the Enterprise Systems Testing Director,  the group performs (1) systems acceptability testing, (2) integration  testing, and (3) final integration testing.

The Implementation and Testing group is responsible for performing  project and integration testing on new and modified ACA systems,  and coordinates integration tests with Enterprise Systems Testing for  ACA and existing tax return processing systems. In addition,  Implementation and Testing ensures testing for non-functional  requirements such as performance, security, and accessibility through  partnership with experts.

IRS has performed various levels of testing for the ACA releases that are  now in production. In addition, testing for systems currently in progress is  underway.

According to the Carnegie Mellon University Software Engineering  Institute (SEI), a consolidated report drawing information from many  sources is key to providing decision makers with the information they  need to make timely and informed decisions. This suggests that  consolidated reporting would be critical for a complex process such as  testing, where there are several organizations involved and a large  number of systems and requirements being tested at different levels. In  addition, SEI practices suggest that the status of all impacted systems  and requirements should be accounted for in overall status reporting— whether or not they are tested.

Although reports on the overall status of ACA testing activities are  provided to IRS senior management via ACA Testing Review Checkpoint  reports and filing season status reports, these reports are not  comprehensive because they do not identify the status of testing for all  systems impacted by ACA Releases 5.0 and 6.0. For example, IRS’s  October and December 2014 ACA Testing Review Checkpoint reports did  not identify the status of testing for 26 and 24 of the 66 impacted systems,  respectively. When asked about this, IRS officials stated that all systems  do not undergo the Enterprise Systems Testing and Implementation and  Testing group tests identified above. Specifically, the two organizations  responsible for testing collectively identify systems deemed critical for  testing and only those systems are included in the reports we reviewed.  Nevertheless, including all impacted systems in reporting, including those  that are not tested, as suggested by best practices, would ensure  accountability for all systems.

It is important to note that IRS’s Testing Review Checkpoint reports and  filing season status reports are not always aligned with the manner in  which ACA testing is being performed. For example, while IRS noted that  ACA testing is conducted on requirements, the reports did not provide a  status of requirements tested, making it difficult to determine whether all  requirements have been tested. Without status reports that account for all  impacted systems and are aligned with the manner in which IRS performs  testing, it will be difficult to determine whether all required testing is being  performed to ensure ACA is ready for the filing season.

section_title: Conclusions
paragraphs: IRS has made limited progress in improving the reliability and reporting of  cost, schedule, and scope performance information. Until the agency fully  implements the prior recommendations highlighted in our review, the  information Congress receives will not be reliable for effective decision  making and oversight.

While IRS is required to provide monthly updates on the cost and  schedule performance of its major investments, the information for two  investments (RRP and IRDM) was not always updated, and IRS did not  always disclose when this was the case in congressional reporting. In  addition, IRS reports summary-level risk assessment ratings for 13 of its  major investments in its reporting to Congress. Providing similar ratings  for its remaining 6 major investments would allow Congress to more  easily determine the ones requiring greater attention.

Three selected investments had exceeded initial planned costs, fallen  behind initial planned schedule, and had not produced all the expected  functionally; and two had been paused or cancelled. However, these  deviations were not transparent in congressional reporting because IRS  has yet to implement our prior recommendations regarding cumulative  performance and scope reporting. The magnitude of some of the changes  to plans we identified underscores the criticality of implementing our prior  recommendations in improving the transparency of congressional  reporting so Congress has the appropriate information needed to make  informed decisions.

Finally, the reporting of testing activities for the ACA investment segments  which are critical for the 2015 filing season showed that impacted  systems were not all captured in overall status reports. In addition, these  reports were not aligned with the manner in which ACA testing is being  performed. Addressing these two issues would improve IRS’s and key  decision makers’ ability to determine whether all required testing to  ensure readiness for the filing season is being performed.

section_title: Recommendations for Executive Action
paragraphs: To improve the reliability and reporting of investment performance  information and management of selected major investments, we  recommend that the Commissioner of the IRS direct the Chief  Technology Officer to take the following three new actions:

For major investments included in congressional reporting, disclose  instances where cost and schedule performance information reported  to Congress is not updated.

Provide summary-level Chief Technology Officer risk assessment  ratings for all major investments in the quarterly reporting to  Congress.

Modify reporting of ACA testing status to senior management to  include a comprehensive report on all impacted systems—including  an explanation for why impacted systems were not tested at a  particular level—and ensure this reporting is aligned with the manner  in which testing is being performed.

section_title: Agency Comments and Our Evaluation
paragraphs: We obtained written comments on a draft of this report from the  Commissioner of the IRS, which are reprinted in appendix III. In his  written comments, the Commissioner stated that IRS appreciated the  acknowledgment of progress it had made to address two prior year  recommendations to improve the consistency and timeliness in reporting  cost, schedule and scope information for its major information technology  (IT) investments, but disagreed with our assessment of its efforts to  address three prior recommendations for improving the reliability and  reporting of cost, schedule, and scope information. Finally, he stated that  IRS agreed with our two recommendations related to disclosing instances  where performance information is not updated in quarterly reporting to  Congress and expanding summary-level risk assessment ratings to all  major investments. Further, the Commissioner stated the agency would  provide a detailed corrective action plan addressing these  recommendations. The Commissioner also stated that IRS disagreed with  our third recommendation to modify the reporting of testing for the  Affordable Care Act Administration (ACA) investment to senior  management.

Regarding our prior recommendation to develop and implement guidance  that specifies best practices to consider when determining projected cost  and schedule amounts for in-process activities in the monthly reporting,  the Commissioner stated that this continues to be a work in progress for  IRS. Specifically, he stated that IRS’s Information Technology Strategy  and Planning organization and members of various investment teams are  currently collaborating on best practices and a centralized process for  determining project cost and schedules for in-process activities. As noted  in our report, we reviewed a July 2014 cost and schedule variance  reporting procedure that IRS stated addressed our recommendation.  However, while the document described the methodology for revising an  estimate, it did not address the calculation of projected cost and schedule  amounts used for the monthly reporting of cost and schedule variances  for in-process activities, which was the subject of our recommendation.  As a result, we believe the status of this recommendation stands as not  addressed.

Regarding our prior recommendation to report cumulative investment and  investment segment cost and schedule information in the quarterly  reports to Congress, the Commissioner stated that IRS believed the  recommendation was satisfied through its reporting of performance  information in the Department of the Treasury’s SharePoint Investment  Knowledge Exchange (SPIKE) tool, which is also included in IRS’s  quarterly reporting to Congress. However, as noted in our report, this  performance information is for the fiscal year only and is not cumulative  for the investment or investment segment, as recommended, and  therefore does not account for activities that were completed in previous  fiscal years. As a result, we believe the status of this recommendation  stands as not addressed.

Regarding our prior recommendation to develop a quantitative measure  of scope for IRS’s major investments, the Commissioner identified several  practices and processes that he stated are currently in place to assess  and report on the delivery of scope. He mentioned (1) IRS’s quarterly  reporting to Congress, and (2) the OMB exhibit 300 baseline change  request process as examples of such practices and processes. However,  as noted in this and prior reports, while these methods address project  functionality, they do not provide a quantitative measure of progress in  delivering this functionality. In addition, the Commissioner also mentioned  the post implementation review process; however, the post  implementation review process does not provide a measure of progress  in delivering scope as IRS has noted that this process is performed at the  close of each segment. For these reasons, we continue to believe the  status of this recommendation stands as not addressed.

Regarding our recommendation to modify reporting of ACA testing status  to senior management, the Commissioner stated that IRS followed a  rigorous risk-based process for planning the tests of ACA-impacted  systems, including the types and levels of testing. In addition, he stated  that IRS had comprehensive reporting for the filing season 2015 release,  which included ACA impacted systems. We acknowledge the various  levels and types of ACA testing that IRS has performed and have noted  this in our report. However, as also noted in our report, our review of ACA  Testing Review Checkpoint reports and filing season reports which  officials stated were used to provide comprehensive reports to senior  managers did not identify the status of testing for all systems impacted by  ACA Releases 5.0 and 6.0. For example, we found that IRS’s October  and December 2014 ACA Testing Review Checkpoint reports did not  identify the status of testing for 26 and 24 of the 66 impacted systems,  respectively. Including all impacted systems in reporting, including those  that are not tested, as best practices suggest, would ensure  accountability for all systems. Accordingly, we believe our  recommendation is still warranted. IRS also provided us with technical  comments that we have incorporated in the report as appropriate.

We are sending copies of this report to interested congressional  committees, the Commissioner of the IRS, and other interested parties. In  addition, the report will be available at no charge on GAO’s website at  http://www.gao.gov.

If you or your staffs have any questions on the matters discussed in this  report, please contact me at (202) 512-9286 or pownerd@gao.gov.  Contact points for our Offices of Congressional Relations and Public  Affairs may be found on the last page of this report. GAO staff who made  major contributions to this report are listed in appendix IV.

section_title: Appendix I: Objectives, Scope, and Methodology
paragraphs: Our objectives were to (1) evaluate IRS’s efforts to address our  recommendations for improving the reliability and reporting of cost,  schedule, and scope information; (2) summarize the reported cost,  schedule, and performance of IRS’s major IT investments; and (3) assess  the status and plans of selected investments.

For the first objective, we determined the status of actions taken to  address each of five prior recommendations to improve the reliability and  reporting of cost, schedule, and scope information we made in our 2013  and 2014 reviews of IRS’s major IT investments. They address (1) the  timely reporting of cost and schedule variance information for completed  activities; (2) consistently updating cost and schedule information for in- process activities; (3) developing guidance on best practices to consider  when determining cost and schedule variances for in-process activities;  (4) reporting cost and schedule information at the investment or  investment segment level (rather than by fiscal year only); and (5)  reporting qualitatively on how delivered scope compares to what was  planned for investments until a quantitative measure is developed.

For the first recommendation, we calculated the 60-day reporting time  frame required by Treasury for completed activities. We then analyzed  the four quarterly reports on the performance of IT investments  submitted by IRS to the appropriations committees and us between  December 2013 and September 2014 to determine whether  completed activities showed updated cost and schedule information  within those time frames.

For the second recommendation, we reviewed materials related to  training that IRS officials stated were provided to investment staff to  ensure a consistent understanding of the information to be included in  the monthly reports.

For the third recommendation, we reviewed the July 2014 cost and  schedule variance reporting procedure and other guidance IRS stated  it was using to determine projected cost and schedule amounts to  determine whether best practices were being included.

For the last two recommendations related to reporting cumulative  performance information and progress in meeting scope expectations,  we reviewed IRS’s reporting through the Office of Management and  Budget (OMB) exhibit 300 process that IRS stated addressed the  recommendations.

We assessed a recommendation as being fully addressed if IRS provided  evidence that it fully addressed our recommendation; partially addressed  if IRS provided evidence that it addressed our recommendation to some  extent; and not addressed if IRS did not provide any evidence that it  addressed our recommendation.

For our second objective, we obtained from IRS a list of the investments  classified as “major” during fiscal year 2014. We reviewed monthly cost  and schedule variance reports for these investments from October 2013  through September 2014, and followed up with IRS officials to identify the  reasons for investment-level variances that were significant (equal to plus  or minus 10 percent variance from cost or schedule goals) and recurring  (reported for 3 consecutive months or more). We assessed the reliability  of the reported information by confirming our understanding of IRS’s  process for reporting monthly cost and schedule variances, and by  determining the extent to which IRS had taken action to improve the  reliability and reporting of this information.

We reviewed operational performance information reported for IRS’s  major IT investments as of September 2014, to determine the extent to  which each investment met its operational performance goals; this  information included, where reported, the performance target and actual  results for each metric. We compared this information to information  reported for IRS’s major IT investments on OMB’s IT Dashboard website.  Lastly, we reviewed the four quarterly reports on the performance of IT  investments submitted by IRS to the appropriations committees and GAO  between December 2013 and September 2014, to identify the Chief  Technology Officer summary-level risk ratings assigned to major IT  investments. We analyzed these risk ratings to identify trends, and  interviewed IRS officials (including the Deputy Chief Information Officer  for Strategy and Modernization) to identify IRS’s methodology for deriving  these ratings.

For our third objective, we selected Return Review Program (RRP),  Customer Account Data Engine 2 (CADE 2), and Information Reporting  and Document Matching (IRDM) because the cost, schedule, or scope of  these investments had changed from initial plans; and the Affordable  Care Act Administration (ACA) investment due to the investment’s  criticality to the 2015 tax filing season and the significant amount of  resources expected to be expended. For RRP, CADE 2, and the IRDM  Case Management project, we interviewed program officials and  analyzed documentation such as performance work statements, business  cases, baseline change requests, and the four quarterly reports on the  performance of IT investments submitted by IRS to the appropriations  committees and us between December 2013 and September 2014. From  this documentation, we determined the initial cost, schedule, and scope  plans for these investments, as well as any revisions to these plans, and  the functionality delivered.

For ACA, we obtained documentation and interviewed key officials– including those from the ACA Program Management Office, and IRS’s  systems testing organizations–to determine the plan for deployment of  the investment. Further, we identified the plans and status of testing for  Releases 5.0 and 6.0, which are expected to be implemented for the  2015 tax filing season. Specifically, we analyzed the ACA system  architecture for Releases 5.0 and 6.0 to identify associated systems  impacted by the development of ACA. We then reviewed testing  documentation, such as testing status reports and test plans to determine  the extent to which these systems were tested. Lastly, we reviewed  various test reports to determine the extent to which IRS had a  mechanism in place to comprehensively report on the status of testing for  all systems related to ACA Releases 5.0 and 6.0. We compared the  information against best practices for software testing promulgated by the  International Organization for Standardization/International  Electrotechnical Commission/Institute of Electrical and Electronics  Engineers.

We conducted this performance audit from June 2014 to February 2015,  in accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.

section_title: Appendix II: Investment Profiles
paragraphs: This appendix contains the profiles for seven investments critical to IRS’s  mission which we examined in greater detail in our prior reviews of IRS’  major IT investments. Information contained within these profiles  includes, but is not limited to:

Current life-cycle phase: Life-cycle phases can be represented as  planning; development, modernization, and enhancement; operations  and maintenance; or mixed. Planning refers to preparing, or acquiring  the information used to design the asset; assess the benefits, risks,  and risk-adjusted costs of alternative solutions; and establish realistic  cost, schedule, and performance goals for the selected alternative,  before proceeding to full acquisition or termination of a project.  Development, modernization, and enhancement refers to projects and  activities that result in new assets/systems or projects and activities  that result in changes or modifications to existing assets that lead to  substantive improvements, implement legislative or regulatory  requirements, or meet an agency leadership request. Operations and  maintenance refers to those projects and activities that are operating  in a production environment. Finally, mixed refers to projects and  activities that are a combination of development, modernization, and  enhancement and operations and maintenance. Having detailed  information allows for clear tracking of a program’s costs as it moves  through its various life-cycle phases.

Development methodology: This is a framework that is used to  structure, plan, and control the process of developing an information  system. There are a number of approaches that can be utilized by an  investment. IRS’s Enterprise Lifecycle methodology includes the  following approaches: waterfall, planned maintenance, iterative, and  managed services. Waterfall is a sequential development of a solution  with planned reviews and formal approvals required before  continuation of work. The planned maintenance approach manages  change in an organized manner, minimizes the disruption caused by  frequent system changes, and increases the efficiency and  effectiveness of the system change process. Additionally, the iterative  approach is an adaptive development approach in which projects start  with a conceptual vision of the solution and end with deployment, with  repeated cycles of requirements discovery, development, and testing  in between. Finally, the managed services approach is designed to  capitalize on the benefits of managed services provided by either an  outside service, internal business processes, and/or existing  infrastructure service provider. This provides useful information on the  requirements of how a project is to progress through the life cycle.

Contract type: For purposes of this report, this can be broken down  into two categories. The first is firm, fixed price contracts in which the  price is not subject to any adjustments. The second is cost  reimbursement contracts which provide for the payment of allowable  incurred costs, to the extent prescribed in the contract. Types of cost  reimbursement contracts include, but are not limited to (1) a cost plus  fixed fee in which actual costs and a fixed fee can be charged;  however, costs are not allowed to exceed the agreed upon estimate  without approval; and (2) a cost plus incentive fee that provides for an  initially negotiated fee to be adjusted later by a formula based on the  relationship of total allowable costs to total target costs.

Number of rebaselines: Rebaselines are changes to projects’ cost,  schedule, and performance goals (i.e., baselines). According to  officials, scope changes must go through a baseline change request  process and be approved by Treasury and OMB.
subsections: section_title: Affordable Care Act Administration
paragraphs: According to IRS, the Affordable Care Act Administration (ACA)  investment encompasses the planning, development, and implementation  of IT systems needed to support IRS’ tax administration responsibilities  associated with certain provisions of the Affordable Care Act. Initiatives  that have already been deployed include the initial release of the Branded  Prescription Drug Industry Fee project; an effort intended to secure  connection between IRS and the Department of Health and Human  Services/Centers for Medicare and Medicaid Services (CMS) to support  health insurance exchange open enrollment for the Fall of 2013; and  2014 Non-Marketplace Provisions. Releases of the ACA investment that  are critical to the 2015 tax filing season include Release 5.0 for filing  season 2015, and Release 6.0 which includes compliance activities.

section_title: Customer Account Data Engine 2
paragraphs: The Customer Account Data Engine 2 (CADE 2) investment began in  2010 as a new strategy for accelerating completion of a modernized  database and converting to a single processing system sooner than was  expected under CADE (which was the predecessor investment to CADE  2, intended to provide a modernized system of taxpayer accounts, with  the ultimate goal of eventually replacing the Individual Master File). CADE  2 is expected to deliver its functionality incrementally through transition  states. Transition State 1 includes:  1.  Daily batch processing of individual taxpayer returns provided by  modifying the IMF to run on a daily, rather than weekly, basis.  2.  A comprehensive database for housing all individual taxpayer  accounts and loaded with data from CADE and IMF to provide more  timely updates of taxpayer information for use by IRS employees for  compliance and customer service.

IRS reported completing functionality for the daily processing of individual  taxpayer returns in January 2012, and completing Transition State 1 in  November 2012, at a cost of $397.5 million. In July 2014, IRS completed  Transition State 1.5, which included ongoing data assurance,  performance tuning, and downstream systems efforts to prepare the  CADE 2 database for filing season 2014 production.

IRS began work on Transition State 2 in October 2010, and expects to  complete this transition state by March 31, 2015; however, IRS noted that  this planned completion date is likely to change. Transition State 2  includes re-writing IRS’s legacy core tax processing applications in  modern programming language, and is intended to increase flexibility,  scalability, reliability, and security.

Full operational capability: Not applicable Life-cycle costs: 193.644 million  Actual spent to date: $182.837 million  Current life-cycle phase: Mixed (development,  modernization and enhancement, and  operations and maintenance)

The e-Services investment is a suite of web-based products that are  intended to allow tax professionals and payers to conduct business with  IRS electronically. These services are only available to tax practitioners,  registered agents, and other third parties and are not available to the  general public. The program is available via the Internet 24 hours a day, 7  days a week, and it contains products such as registration, an e-file  application, a Transcript Delivery System (a system which tax  professionals may use to request and receive account transcripts, wage  and income documents, tax return transcripts, and verification of non- filing letters), and Taxpayer Identification Number Matching (a pre-filing  service which allows authorized payers to match up to 25 payee taxpayer  identification number and name combinations against IRS records prior to  submitting an information return).

The Information Reporting and Document Matching (IRDM) investment is  aimed at helping close the tax gap—the difference between what  business taxpayers should have paid and actually did. It is intended to  improve voluntary compliance and accurate reporting of income by  establishing a new business tax return and information returns that focus  on merchant card payments and securities basis reporting. IRDM  supports IRS business using information systems that sort, match,  identify, manage, and report on returns that are likely sources of tax gap- reducing revenue. To accomplish this, IRS requires operational resources  and systems to be put in place to implement business and technology  changes that are intended to expand and improve its automated matching  of data on information returns to the data submitted on tax returns filed.  The investment consists of the following four projects. As detailed in this  report, this investment previously included a case management project  that was cancelled in January 2014.

Data Assimilation: Identifies the link between tax forms and  information returns filed for the same taxpayer to identify potential  under-reporter cases. The project then groups these into specific  categories to support IRS compliance programs associated with  merchant card payments, securities cost basis, and government  payments.

Data Correlation: Matches tax return and information return data and  applies business rules to identify potential under-reporter cases for  use in the IRDM case selection process. After case selection, data  correlation builds a complete case record for analysis by a tax  examiner to support IRS compliance programs.

Business Master File analytics: Provides IRS users the ability to  define and execute logic for the intelligent selection of business  taxpayer case inventory to ensure cases selected result in the largest  financial return.

Case Inventory Selection and Analytics: Provides IRS users the ability  to define and execute logic for the intelligent selection of individual  taxpayer case inventory and creates an analytical environment that  offers a greater ability to evaluate case data to improve the selection  of cases worked.

The IRS.Gov investment consists of a public user portal—IRS.Gov, a  registered user portal, and an employee user portal. The key goals of the  program include simplifying and transforming the user web experience,  consolidating and advancing IRS web technology to industry standards,  implementing a high-performing contract structure and terms, and  marketing competitive costs throughout the program’s life cycle.

Actual spent to date: $560.118 million Current life-cycle phase: Mixed (development,  modernization and enhancement, and  operations and maintenance) provide a cost effective and affordable program cost structure; and transition successfully from the old programs to the new program.

The Modernized e-File (MeF) investment is the primary system to receive  and process all tax returns submitted electronically. When MeF receives  an electronic tax return, the system determines if it satisfies the  acceptance rules required for further processing. MeF is intended to  benefit the tax preparation community and enables the IRS to answer  questions quickly and helps to resolve issues. MeF is also intended to  benefit corporations and tax-exempt organizations that must file tax  returns or annual information returns electronically and is intended to  reduce the handling/mailing of voluminous paper returns.

Actual spent to date: $417.871 million Current life-cycle phase: Mixed (development,  modernization and enhancement, and  operations and maintenance)

MeF stores all tax return data in Extensible Markup Language format in a  Modernized Tax Return Database, allowing authorized IRS viewers (IRS  Help Desk personnel and tax examiners) to see tax returns securely  online. According to IRS, as of August 2014, taxpayers used MeF to  submit over 228 million individual returns and over 14 million business  returns.

IRS deployed MeF Release 9.5 in May 2014, for filing season 2015.  According to IRS, Release 9.0 and 9.5 add the  employment/unemployment tax family of forms (forms 94x) and the U.S.  Income Tax Return for Estates and Trusts (Form 1041) to the MeF  environment, as well as a new RRP interface, Affordable Care Act and  other legislative changes.

The Return review Program (RRP) investment is a web-based automated  system that is intended to replace the legacy Electronic Fraud Detection  System (EFDS) built in the mid-1990s. It is intended to deliver  functionality incrementally through transition states. In September 2013,  IRS officials adopted a risk mitigation approach that split Transition State  1 into two releases. The first release —called Transition State 1 Release  1.0—occurred in March 2014 and contained functionality needed for  processing filing season returns. The second release—called Transition  State 1 Release 1.1—is planned to occur after filing season. RRP is to,  among other things: enable more effective routing of returns, detect noncompliant and fraudulent returns, ensure timely issuance of refunds and credits, prevent issuance of refunds and credits not legally due to filers, and streamline business processes used by the IRS criminal investigative  staff.

The new system is comprised of three major activities:

Detection. Intended to incorporate several existing models as well as  new models to enhance detection of probable noncompliance. Using  algorithms and business rule sets, the system is intended to detect  questionable information on each return as the return is processed.  The system is also intended to detect returns with potential fraud  characteristics, thereby allowing criminal investigators to link and  analyze groups of returns to identify schemes for potential criminal  prosecution.

Resolution. Intended to accommodate existing treatment streams and  new treatment streams. Returns will be routed systemically to the best  treatment stream, opened into the treatment stream’s inventory and, if  applicable, the system will send an initial contact letter to the  taxpayer.

Prevention. Intended to automatically integrate the results of each  return’s resolution into the detection models. The results can be used  to help target education and outreach efforts to taxpayers and  preparers on how to avoid unintentional noncompliance. The system  is also intended to allow analysis and identification of fraud and  noncompliance not identified by the predictive detection models.

section_title: Appendix III: Comments from the Internal Revenue Service

section_title: Appendix IV: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact

section_title: Staff Acknowledgments
paragraphs: In addition to the individual named above, the following staff made key  contributions to this report: Sabine Paul, assistant director; Chris  Businsky; Mary Evans; Rebecca Eyler, Nancy Glover, James MacAulay;  Paul Middleton; Bradley Roach; and Karl Seifert.


Question-summary annotations: questions: question: What were IRS investments reported as meeting?
answer: Most of IRS's major information technology (IT) investments were reported as meeting cost and schedule goals. Specifically, 11 of 17 investments were reportedly within 10 percent of cost estimates, and 13 were within 10 percent of schedule estimates between October 2013 and September 2014.

question: What did the agency report for Chief Technology Officer summary-level risk assessments?
answer: In addition, the agency reported “green” ratings for investments instead of their previous “yellow” ratings for Chief Technology Officer summary-level risk assessments.
child_questions: question: What are these ratings not provided for?
answer: It is important to note that these ratings are not provided for 6 investments for which IRS provides detailed reporting to Congress.

question: What would providing summary level risk rating for all major investments do?
answer: Providing summary-level risk ratings for all major investments would improve the visibility into changes in investment risk, and provide Congress with the information to more easily determine the investments requiring greater attention.

questions: question: Why did selected investments experience variances?
answer: Selected investments experienced variances from initial cost, schedule, and scope plans that were not transparent in congressional reporting because IRS has yet to address GAO's prior recommendations.
child_questions: question: What is one of these investments?
answer: Specifically, the Return Review Program has so far exceeded planned costs by $86.5 million and has yet to deliver functionality that was scheduled for September 2012, and a key phase of Customer Account Data Engine 2 was developed 10 months late and at $183.6 million more than planned.

question: Why is it difficult to determine whether all required testing is being performed?
answer: In addition, the consolidated reports on the status of testing for the Affordable Care Act Administration investment are not comprehensive, making it difficult to determine whether all required testing is being performed.

questions: question: What does the IRS rely on?
answer: IRS relies extensively on IT systems to annually collect more than $2 trillion in taxes, distribute more than $300 billion in refunds, and carry out its mission of providing service to America's taxpayers in meeting their tax obligations.
child_questions: question: What did IRS expect to spend on IT in FY 2014?
answer: For fiscal year 2014, IRS expected to spend about $2.4 billion on IT.

question: Why is it important that Congress be provided reliable information about IT?
answer: Given the size and significance of IRS's IT investments and the challenges inherent in successfully delivering these complex IT systems, it is important that Congress be provided reliable cost, schedule, and scope information to assist with its oversight responsibilities.