Information Technology: IRS Needs to Improve the Reliability and Transparency of Reported Investment Information
Published: 2014-04-02T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: IRS relies extensively on IT systems to annually collect more than $2 trillion in taxes, distribute more than $300 billion in refunds, and carry out its mission of providing service to America's taxpayers in meeting their tax obligations. For fiscal year 2014, the agency's budget request is $2.6 billion for IT. Given the size and significance of IRS's IT investments, and the challenges inherent in successfully delivering these complex IT systems, it is important that Congress be provided reliable cost, schedule, and scope information to assist with its oversight responsibilities.
Accordingly, GAO's objectives were to (1) summarize the reported cost and schedule performance for IRS's major IT investments and assess the reporting of the performance information (2) for selected investments, evaluate the reliability of reported cost and schedule variances; and (3) determine IRS's progress in implementing a quantitative measure of functionality delivered for projects. To address these objectives, GAO reviewed documentation, including monthly cost and schedule variance reports, and interviewed staff, including those from selected investments which are critical to IRS's mission.
section_title: What GAO Found
paragraphs: According to the Internal Revenue Service (IRS), 6 of its19 major information technology (IT) investments were within 10 percent of cost and schedule estimates during fiscal year 2013. The remaining 13 were reported as having significant cost and schedule variances for at least 1 month of the year. Specifically, IRS reported 8 investments as being below cost or ahead of schedule; 3 as behind schedule; and 2 as over cost (see figure). IRS reported a range of reasons for these variances, including cost savings from contract negotiations and scope deferrals for those below cost or ahead of schedule, and procurement delays for those behind schedule. It is important to note that the reported monthly cost and schedule variances are for the fiscal year only. IRS's reporting would be more meaningful if it were supplemented with cumulative cost and schedule variances for the investments or investment segments, consistent with the Office of Management and Budget's guidance for measuring progress towards meeting investment goals.
Summary of Investments with Significant Variances
The reported variances for six investments reviewed were not always reliable because the projected and actual cost and schedule amounts on which they depend were not consistently updated in accordance with OMB and Treasury reporting requirements. Specifically, projected amounts were not always updated on a monthly basis for in-process (i.e., ongoing) activities for a range of reasons; and actual information was not consistently updated for completed activities within the 60-day time frame required by Treasury guidance. Addressing these issues would help IRS improve the reliability of reported information and provide Congress with a more accurate report of the agency's performance in meeting cost and schedule goals.
IRS is currently not working on developing a quantitative measure of scope (i.e., functionality) as GAO recommended in June 2012 and does not plan on doing so, because it believes it is difficult to develop a single measure that would allow it to measure progress in delivering scope for the range of IRS's investments. However, the agency could develop measures using different methods as appropriate. In addition, IRS has started to include scope performance (in the functional performance section) in its quarterly reporting to Congress. However, this information does not include a quantitative measure of performance. In addition, it does not show how delivered scope compares to what was planned. Providing transparency into progress in delivering scope by reporting qualitatively in the congressional reports until a quantitative measure is developed would help IRS provide Congress a complete picture of the agency's performance in managing its major investments.
section_title: What GAO Recommends
paragraphs: GAO is making recommendations for IRS to report more comprehensive and reliable cost and schedule information and improve the transparency of reported scope information for its major investments. IRS agreed with GAO's recommendations and stated it believed it had addressed one of them. GAO continues to believe further actions are needed.

section_title: Letter
paragraphs: The Internal Revenue Service (IRS) relies extensively on information  technology (IT) systems to annually collect more than $2 trillion in taxes,  distribute more than $300 billion in refunds, and carry out its mission of  providing service to America’s taxpayers in meeting their tax obligations.  For fiscal year 2014, the agency’s budget request is $2.6 billion for IT,  about 20 percent of IRS’s total budget request for the year.
Given the size and significance of IRS’s IT investments, and the  challenges inherent in successfully delivery these complex IT systems, it  is important that Congress be provided ongoing, accurate, and objective  information on the progress toward completion and the risks facing these  projects. Accordingly, the Conference Report accompanying the  Consolidated Appropriations Act, 2012 and the Consolidated and Further  Continuing Appropriations Act, 2013 mandated GAO to review the cost  and schedule performance of IRS’s major IT investments. Specifically,  our objectives were to (1) summarize the reported cost and schedule  performance for IRS’s major IT investments and assess the reporting of  the performance information; (2) for selected investments, evaluate the  reliability of reported cost and schedule variances;IRS’s progress in implementing a quantitative measure of functionality  delivered for projects.   and (3) determine  To address our first objective, we obtained and analyzed monthly cost  and schedule variances for IRS’s major IT investments from October  2012 to September 2013, as reported by IRS. Additionally, we identified  significant cost and schedule variances—those variances that were 10  percent greater or less than planned cost and schedule goals.
For our second objective, we analyzed monthly cost and schedule  variance reports to determine whether they reflected monthly updates for  both in-process and completed activities. We also interviewed relevant  IRS officials to determine IRS’s progress in addressing our recent  recommendation to establish guidance for determining projected cost and  schedule amounts—which are critical to calculating variances—in  accordance with best practices.
To address our third objective, we followed up with IRS officials to  determine what progress had been made in putting in place a quantitative  measure of functionality delivered as we had previously recommended.
When we learned IRS had not addressed our recommendation and was  not planning on doing so, we followed up on the causes and, for each of  the investments in our selection, identified the qualitative methods IRS  stated it was using to manage scope.
We conducted this performance audit from June 2013 to April 2014 in  accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives. Details on our objectives,  scope, and methodology can be found in appendix I.
subsections: section_title: Background
paragraphs: The mission of IRS, a bureau within the Department of the Treasury  (Treasury), is to provide America’s taxpayers top quality service by  helping them understand and meet their tax responsibilities and by  applying the federal tax laws with integrity and fairness to all. In carrying  out its mission, IRS annually collects over $2 trillion in taxes from millions  of individual taxpayers and numerous other types of taxpayers and  manages the distribution of over $300 billion in refunds. To guide its  future direction, the agency has two strategic goals: (1) improve taxpayer  service to make voluntary compliance easier and (2) enforce the law to  ensure everyone meets their obligations to pay taxes.
IRS is organized into four primary operating divisions to meet the needs  of specific taxpayer segments.
The Wage and Investment Division services individual taxpayers and  provides the information, support, and assistance these taxpayers  need to fulfill their tax obligations.
The Small Business and Self-Employed Division services all fully or  partially self-employed individuals and corporations and partnerships  with assets of $10 million or less.
The Large Business and International Division services corporations  and partnerships with assets greater than $10 million.
The Tax Exempt and Government Entities Division services a large  and unique economic sector of organizations, which include pension  plans, exempt organizations, governmental entities, and tax-exempt  bond issuers.
It includes nine subordinate offices; among
The Associate CIO for Applications Development builds, tests,  delivers, and maintains integrated information application systems  that will support modernized systems and the production environment.  It is the development organization for systems that manage taxpayer  accounts from the initial filing of a tax return, interactions with the  taxpayers, and potential audit and collection activities. It also provides  enterprisewide administrative systems related to workforce support,  human capital, financial, and facilities. Application Development is  responsible for two of the seven programs we selected for detailed  analysis in our review.
The Associate CIO for Enterprise Operations is responsible for  supporting the mainframe and server environment by providing  efficient, cost-effective, secure and reliable server and mainframe  services for all IRS business entities and taxpayers. It also is  responsible for financial management, human resources, training  support and general business administration, developing an  enterprisewide development and test environment and establishing  security management. The Associate CIO is responsible for one of the  seven programs we selected for detailed analysis in our review.
The Associate CIO for Strategy and Planning is to facilitate the  alignment of IT and business. This is to be done through strategic  planning and financial management practices that offer transparency  of overall IT demand, supply, and the value of IT investments. This  office includes a suboffice that is responsible for optimizing the  efficiency and effectiveness of the IT portfolio through fiscally sound  portfolio management processes. The office also serves as the  primary interface with Treasury’s capital planning and investment  control organizations to coordinate actions including baseline change  requests, budget formulation documents, and IT Dashboard reporting.
The Associate CIO for Enterprise IT Program Management Office is  responsible for the delivery of integrated solutions for several of IRS’s  large-scale programs. It plays a key role in establishing change,  configuration, and release plans and implementing new information  system functional capabilities. This Associate CIO is responsible for  three of the seven programs we selected for detailed analysis in our  review.
The Associate Chief Information Officer (CIO) for the Affordable Care  Act Program Management Office is responsible for setting the overall  direction, providing day-to-day management and oversight and  delivery of IRS IT’s contribution for implementing provisions of the  Patient Protection and Affordable Care Act. This Associate CIO is  responsible for one of the seven programs we selected for detailed  analysis in our review.
Figure 1 shows the structure of IRS’s Information Technology  organization.
subsections: section_title: IRS Relies on IT to Carry Out Its Mission
paragraphs: IT plays a critical role in enabling IRS to carry out its mission and  responsibilities. For example, the agency relies on information systems to  process tax returns, account for tax revenues collected, send bills for  taxes owed, issue refunds, assist in the selection of tax returns for audit,  and provide telecommunications services for all business activities,  including the public’s toll-free access to tax information.
For fiscal year 2014, IRS requested about $2.6 billion for IT investments.  This included about $1.8 billion, about 69 percent, of the total amount for  19 major investments, and about $800 million, or 31 percent, of the total  amount for 130 non-major investments. For IRS, a major investment is  one that costs $10 million in either current year or budget year, or $50  million over the 5-year period extending from the prior year through the  budget year +2. The following table provides a list of the 19 investments  and their descriptions.
section_title: GAO and the Treasury Inspector General for Tax Administration Have Previously Reported on IRS’s Key Major IT Investments
paragraphs: GAO and the Treasury Inspector General for Tax Administration (TIGTA)  have previously reported on IRS’s ongoing effort to modernize its tax  administration and internal management systems, the agency’s  investment management process and key major investments—CADE 2,  IRDM, and ACA.
Regarding investment management, in July 2011 we found that IRS  had established most of the foundational practices needed to  effectively manage its IT investments by executing 30 of the 38 key  practices identified by the information technology investment  management framework, including all of those needed for effective  project oversight.
Further, we reported in October 2011 that CADE 2 was one of seven  investments considered to be successfully acquired in that it best  achieved its respective cost, schedule, scope, and performance goals.  Officials cited active engagement with program stakeholders as a  critical factor to the success of those investments and, overall nine  common factors that were critical to the success of major IT  investments were identified.
We reported in April 2013 that the majority of IRS’s major IT  investments were reportedly within 10 percent of cost and schedule  estimates and 8 major IT investments reported significant cost and/or  schedule variance. We also reported that weaknesses existed, to  varying degrees, in the reliability of reported cost and schedule  variances; and key risks and mitigation strategies were identified. As  result, we made recommendations for IRS to improve the reliability of  reported cost and schedule information by addressing the identified  weaknesses in future updates of estimates. We also recommended  that IRS ensure projects consistently follow guidance for updating  performance information 60 days after completion of an activity and  develop and implement guidance that specifies best practices to  consider when determining projected amounts. IRS agreed with three  of our four recommendations and partially disagreed with the fourth  recommendation related to guidance on projecting cost and schedule  amounts. The agency specifically disagreed with the use of earned  value management data as a best practice to determine projected  cost and schedule amounts, stating that the technique is not part of  IRS’s current program management processes and the cost and  burden to use it outweighs the value added. While we disagreed with  IRS’s view of earned value management because best practices have  found that the value generally outweighs the cost and burden of  implementing it, we provided it as one of several examples of  practices that could be used to determine projected amounts. We also  noted that implementing our recommendation would help improve the  reliability of report cost and schedule variance information, and that  IRS had flexibility in determining which best practices to use to  calculate projected amounts. For those reasons, we believed our  recommendation was still warranted.
In our September 2013 review, we reported that IRS had updated the  ACA cost estimate we had reviewed in our June 2012 budget report  and noted that it reflected best practices to a greater extent.example, the estimate reflected the full life-cycle cost of the program.  However, we noted a few areas remained where IRS could continue  to improve the reliability of the estimate. We will continue to monitor  IRS’s progress in revising this estimate.
In September 2013, TIGTA reported that the CADE 2 database cross- functional triage team effectively managed and resolved more than  1,000 data defects. However, TIGTA’s review determined that the  downstream system interfaces were not implemented due to data  quality issues and revised the implementation date of these interfaces  to January 2014, raising the estimated cost of transition state 1.  Finally, the lack of security systems integration prevents transaction- level tracking of employee access to the database. As a result of  these findings, TIGTA made recommendations aimed at improving the  management of the CADE 2 program, including not exiting transition  state 1 milestone 5 until interfaces with selected downstream systems  are implemented into production; ensuring the CADE 2 database is  accurate, complete, timely, and available; deploying the CADE 2  database as the transactional database architected for transition state  2 and the target state; and ensuring that user access to the CADE 2  database is traced at the transaction level by individual user’s  identification. IRS agreed to certify the database as accurate,  complete, timely, and available but disagreed with the remaining three  recommendations.
section_title: Most of IRS’s 19 Major Investments Reportedly Within or Below Cost or Schedule Goals, but Reported Information Is Only for the Current Fiscal Year
paragraphs: According to IRS, 6 of 19 investments were within 10 percent of cost and  schedule estimates during fiscal year 2013. The remaining 13 were  reported as having significant cost and schedule variances for at least 1  month of the year. Specifically, IRS reported 8 investments as being  below cost or ahead of schedule; 3 as behind schedule; and 2 as over  cost (see fig. 2). IRS reported a range of reasons for these significant  variances, including cost savings from contract negotiations and scope  deferrals for those below cost or ahead of schedule, and procurement  delays for those behind schedule. Specifically, IRS reported that
CADE 2, Integrated Financial System, Integrated Submission and  Remittance Processing System, IRS.gov, Main Frames and Servers  Services and Support, and RRP were within 10 percent of cost and  schedule estimates.
Account Management Services, Electronic Fraud Detection System,  e-Services, Foreign Account Tax Compliance Act, Integrated  Customer Communication Environment, IMF, IRDM, and Integrated  Data Retrieval System were significantly below cost or ahead of  schedule for reasons including early delivery of scope, redesign of  releases, prematurely closing of an activity, and actual costs being  lower than expected.
End User Systems and Services and SCRIPS were reported as being  behind schedule for reasons such as scope changes, and hardware  and software procurement issues.
MeF and ACA were reported as being significantly under cost and  over cost at different times of the year, and Telecommunications  Systems and Support was reported as being significantly under cost  and behind schedule at different times during the year.
According to IRS officials, the significant variances experienced by its  major investments had no impact on the delivery of expected benefits.
For a detailed list of reasons associated with the significant variances see  appendix II. Figure 2 shows a summary of investments with significant  cost and schedule variances.
It is important to note that the reported monthly cost and schedule  variances are for the fiscal year only. IRS’s reporting would be more  meaningful for determining whether IRS is effectively managing its  investments if it included cumulative cost and schedule variances for the  investments or investment segments, consistent with OMB’s guidance for  measuring progress towards meeting investment goals. Specifically,  OMB’s fiscal year 2015 guidance on the planning, budgeting, acquisition,  and management of IT capital assets instructs agencies to structure their  efforts into “useful segments” (such as projects which accomplish a  unique product or service with a defined start and end point and specific  objectives that, when attained, signify completion), and to establish  systems to measure progress towards these useful segments in terms of  cost and schedule, among other things.
IRS’s monthly reports—and thus the quarterly reports to Congress that  are derived from them—provide cost and schedule variance information  for all investment projects and activities underway in any portion of the  fiscal year. However, this fiscal year focus does not provide cumulative  cost and schedule information at the useful segment level because it  does not account for activities that were completed in previous fiscal  years. In addition, it does not provide a realistic representation of  performance because it includes activities which are not scheduled to  begin until later in the fiscal year and reports these activities as having  zero cost and schedule variances.monthly report, IRS reported a 5.07 percent cost underrun for RRP. The  cost variance calculation included a zero variance for an activity that was  not scheduled to start until the following month. The variance not  including this activity would have been 6.06 percent.
For example, in the June 2013  Officials told us that the way the agency reports is consistent with  Treasury guidance and OMB reporting requirements. Nonetheless, while  IRS’s reporting is aligned with the manner in which budgets are allocated,  it does not provide a measure of progress towards the useful segments which are associated with specific functionality as recommended in OMB  guidance, and therefore does not fully capture IRS’s performance in  meeting overall project and investment level cost or schedule goals.  Providing Congress with cost and schedule information at the useful  segment level—in addition to the current fiscal year reporting—in the  quarterly reports would provide a more meaningful gauge of whether the  agency is effectively managing its investments.
section_title: Reported Cost and Schedule Information Not Always Reliable
paragraphs: IRS’s investments are typically structured by milestones or releases which are  consistent with the definition of a useful segment. of reported information and provide Congress with a more accurate report  of the agency’s performance in meeting cost and schedule goals.
subsections: section_title: Cost and Schedule Variances for In-process Activities Are Not Consistently Reported
paragraphs: Cost and schedule variances, which OMB and Treasury require to be  updated on a monthly basis, are determined by subtracting actual cost  and schedule from planned amounts or using projected information if  actual information is not available. In addition, as we previously reported,  according to best practices, projected cost and schedule can be  determined in a number of ways, including evaluating critical path (for  projected schedule), using earned value management data, evaluating  the performance of completed work and comparing it to the remaining  budget, assessing commitment values for material needed to complete  remaining work, and estimating future conditions.
Although required, IRS’s monthly reports for fiscal year 2013 showed that  all six investments reviewed did not consistently include updated  variances for in-process activities. For two of the six investments, staff  told us they did not include variances for in-process activities for several  reasons, including that activities are usually 6 months or less. However,  OMB and Treasury’s guidance do not provide exceptions for shorter  periods of time such as 6 months or less. Staff for the remaining four  investments told us they update cost and schedule variances on a  monthly basis; however, our analysis of the reports we reviewed showed  that they did not. This is partly due to an inconsistent understanding  among investment staff of the information that was to be included in the  monthly reporting—which the Strategy and Planning group responsible  for overseeing monthly variance reporting also acknowledged.
Regarding the calculation of projected cost and schedule amounts, IRS  has not yet updated its guidance to include best practices to consider as  we recommended in April 2013. According to officials, the agency has  experienced delays in addressing the recommendation due to budget  issues, staffing shortages, and the government shutdown. However,  officials reported that they have begun evaluating practices and  processes currently in use as inputs for standard, repeatable, best  practices for determining cost and schedule projections but they did not  specify a time frame for fully addressing our recommendation. Developing  and implementing the guidance will provide greater assurance that  projected amounts, when reported, are determined consistent with best  practices and therefore reliable.
section_title: Cost and Schedule Variances for Completed Activities Were Not Always Updated with Actual Information in a Timely Manner
paragraphs: Treasury’s guidance states that investments have 60 days to report actual  cost and schedule from the time an activity has been completed.  According to the guidance, this requirement is one of several intended to  ensure the accuracy of the monthly reporting data.
The quarterly reports submitted to the appropriations committees  between December 2012 and September 2013 for performance between  January 2013 and September 2013 show that, for the six investments  featured in the reports, IRS updated investment cost and schedule  variance information with actual amounts within the 60-day time frame  required by Treasury in 78 percent of the cases, a slight improvement  from last year. Specifically, four investments—e-Services; IRS.Gov; MeF;  and RRP—updated all activities within the required time frame. However,  most of the variances for CADE 2 activities were not updated in a timely  manner. In addition variances for several IRDM activities were not  updated on time. Table 2 identifies the number of activities which were  updated within the 60-day window.
In April 2013, we reported on this issue and noted that, according to IRS  officials, actual information is sometimes not updated within the 60-day  time frame because of delays associated with approval through the  governance process. We stated that we believed 60 days to be a  reasonable amount of time to provide information on the status of an  activity that was expected to be completed and recommended that IRS  take action to ensure projects consistently follow guidelines for updating  performance information 60 days after completion of an activity. Staff  from IRS’s Strategy and Planning Office told us that, starting in fiscal year  2014, Treasury has addressed the timeliness issue for schedule  calculations by having the monthly reporting system automatically  calculate a variance based on the current date for any activity where the  planned completion date has passed and investment staff have not  provided an actual figure within 45 days of that projected date. We  believe this is a positive step towards ensuring actual information is more  consistently reported for completed activities within the required 60-day  time frame and improving the reliability of their variances so they better  reflect cost and schedule performance.
Ensuring the variances for completed activities are consistently updated  with actual information within the 60-day time frame required by Treasury  as we previously recommended would provide more reliable information  on which to gauge IRS’s performance in meeting cost and schedule  goals.
section_title: IRS Has Yet to Develop a Quantitative Measure of Scope
paragraphs: According to OMB guidance, in managing their IT investments,  agencies must use a performance-based acquisition management system  or earned value management system in which work progress is quantified  to measure both what was spent to perform the work and what was  scheduled to have been accomplished. Consistent with this concept, we  previously reported that having a quantitative measure of scope is a good  practice as it provides an objective measure of whether an investment  delivered the functionality that was paid for, and, in June 2012,recommended that IRS develop such a measure for all its major  investments. At that time, IRS agreed with our recommendation.
IRS is currently not working on developing a quantitative measure of  scope and does not plan on doing so because it believes it is difficult to  develop a single measure that would allow it to measure progress in  delivering scope for the range of IRS’s investments. In addition, IRS  stated that its current scope reporting is adequate for its project  management purposes. Finally, it has begun reporting on scope  performance in its quarterly reports to the appropriations committees.
First, we believe IRS could develop quantitative measures using different  methods as appropriate. For example, it could use earned value  management, a technique which integrates the technical scope of work  with schedule and cost elements for investment planning and control or it  could leverage its requirements management processes to assign value  to planned work. Second, while IRS has identified several processes for  managing scope, they are primarily used for internal management  purposes. Finally, while to its credit, IRS has started to include scope  performance (in the functional performance section) in its quarterly  reporting to Congress, there are no quantitative measures of progress  made. In addition, the descriptions provided do not show how delivered  scope compares to what was planned.
Staff for four of the seven investments we reviewed said their investments  have a fixed schedule (tied to the filing season) and they often revert to  modifying scope to meet this constraint. These changes in scope—which  staff stated are tracked internally—are often not apparent in reported cost  and schedule variances, and underline the importance of reporting on  progress in delivering scope. As an example, in April 2013, data  assurance, performance tuning, and downstream systems work that was  planned for CADE 2 Transition State 1 was put into a new release— Transition State 1 release 1.5—allowing IRS to “exit” Transition State 1 by  the end of fiscal year 2012, in time for the filing season.
As previously mentioned, a quantitative measure of scope would provide  an objective indicator of performance. Until such a measure is developed,  being more transparent about progress in delivering scope by reporting  qualitatively on how delivered scope compares to what was planned in  the congressional reports would help IRS provide Congress a complete  picture of its performance in managing its major investments—particularly  in those cases where functionality is deferred.
section_title: Conclusions
paragraphs: For fiscal year, 2013, the cost and schedule performance for IRS’ major  IT investments was mixed, with six investments reported as being within  10 percent of cost and schedule goals throughout the year, eight reported  as being significantly below cost or ahead of schedule—although in some  cases, these are due to scope deferrals or poor estimating practices— and 5 behind schedule or over cost. While the current fiscal year focus of  the reported information is aligned with the manner in which budgets are  allocated, reporting cumulative information at the investment or major  investment segment level—consistent with the delivery of functionality— would provide a better indicator of IRS’s performance in meeting cost and  schedule goals. However, the reported cost and schedule variance  information is not always reliable and Congress therefore may not have  the most accurate information to inform its decisions. Specifically, the  variances for in-process and completed activities are not consistently  updated due to, among other reasons, an inconsistent understanding of  the information to be included in the monthly reports. IRS’s unreliable  reporting of cost and schedule variance information is further exacerbated  by the fact that the agency has not yet developed a quantitative measure  of scope that would help provide a complete picture of how IRS is  managing its major investments.
section_title: Recommendations for Executive Action
paragraphs: To improve the reliability of reported cost and schedule variance  information for major investments, we recommend that the Commissioner  of IRS direct the Chief Technology Officer to take the following three  actions:  report cumulative investment and investment segment cost and  schedule information in the quarterly reports to Congress, consistent  with OMB requirements for measuring progress towards meeting  goals; ensure that projected cost and schedule variances for in-process  activities are updated monthly, for the six investments for which we  reviewed monthly updates, consistent with OMB and Treasury  reporting requirements, by ensuring investment staff have a  consistent understanding of the information to be included in monthly  reports; and until a quantitative measure of scope is developed, qualitatively report   on how delivered scope compares to what was planned in its quarterly  reports to Congress, for the seven investments for which we reviewed  scope reporting.
While we did not review the monthly update process and scope reporting  for all of IRS’s major IT investments, the agency should extend the  related recommendations we are making as appropriate to other  investments that are featured in the quarterly reports to Congress.
section_title: Agency Comments and Our Evaluation
paragraphs: We obtained written comments on a draft of this report from IRS’s  Commissioner, which are reprinted in appendix V.  In his written  comments, the Commissioner stated that IRS agreed with our  recommendations and recognizes the importance of consistent reporting  to Congress on cost, schedule, and scope and will continue its effort to  improve management of its investment process and values our support  and guidance. He further stated IRS believed it had addressed our first  recommendation to report cumulative investment and investment  segment cost and schedule information in the quarterly reports to  Congress as well as the recommendation we previously made to develop  a quantitative measure of scope relating to our third recommendation. He  added that the agency would provide a detailed corrective action plan  addressing our second recommendation with its response to the final  report.
Regarding the recommendation to report cumulative investment cost and  schedule information, the Commissioner stated that cumulative life-cycle  investment costs for each investment are included in the quarterly report  to Congress. While the report includes cumulative costs, they are  cumulative for the fiscal year, not for the investment or investment  segment as we recommended and they therefore do not account for cost  variances from prior fiscal years. As noted in this report, such reporting  would be consistent with OMB guidance for measuring progress towards  useful segments and provide a more meaningful gauge of whether the  agency is effectively managing its investments.
Relating to the recommendation we previously made to develop a  quantitative measure of progress in delivering scope, the Commissioner  noted that the agency has practices and process in place to assess and  report on the delivery of scope in conjunction with cost and schedule  management. He cited the quarterly report to Congress which draws on  the Exhibit 300 SharePoint Investment Knowledge Exchange data (i.e.,  the monthly variance information we reviewed) and the baseline change  request process, which specifically addresses the changes that are  occurring within investments, as examples illustrating the implementation  of the recommendation. However, we do not believe that these actions  address our recommendation. Specifically, our review of the monthly  variance reports show they report cost and schedule variances, not scope  information. In addition, while the baseline change request process may  document changes in scope, it does not provide a quantitative measure of  progress in delivering scope as recommended. Further, while IRS has  started to include scope performance (in the functional performance  section) in its quarterly reports to Congress, there are no quantitative  measures of progress in delivering scope. Finally, as noted in our report,  the qualitative information (i.e., descriptions) provided do not show how  delivered scope compares to what was planned.  For these reasons, we  believe our prior recommendation to develop a quantitative measure of  progress in delivering scope and our new recommendation to qualitatively  report on how delivered scope compares to what was planned in its  quarterly reports to Congress until such measure is developed are still  warranted. IRS also provided us with technical comments, which we  incorporated into this report as appropriate.
We are sending copies of this report to interested congressional  committees and the Commissioner of IRS. This report will also be  available at no charge on our website at http://www.gao.gov.
If you or your staffs have any questions on matters discussed in this  report, please contact me at (202) 512-9286 or pownerd@gao.gov.  Contact points for our Offices of Congressional Relations and Public  Affairs may be found on the last page of this report. GAO staff who made  major contributions to this report are listed in appendix VI.
section_title: Appendix I: Objectives, Scope, and Methodology
paragraphs: Our objectives were to (1) summarize the reported cost and schedule  performance for IRS’s major information technology (IT) investments and  assess the reporting of the performance information; (2) for selected  investments, evaluate the reliability of reported cost and schedule  variances; and (3) determine IRS’s progress in implementing a  quantitative measure of functionality delivered for projects.
To address the first objective, we obtained and analyzed monthly cost  and schedule variance data for IRS’s major IT investments for October  2012 through September 2013. The variance data provided were from the  Department of Treasury’s (Treasury) monthly variance reporting tool  which the department uses for internal oversight and external reporting  purposes. We identified the investments with reported variances of 10  percent or more from estimates as having significant variances and  determined the number of months they reported significant variances. For  these investments, we obtained the reason for the significant variances  from the information reported by Treasury on OMB’s IT Dashboard (in  Treasury’s Chief Information Officer comments) and followed up with IRS  for clarification as needed. Because the Affordable Care Act (ACA)  investment was not reporting monthly cost and schedule variances in  Treasury’s reporting tool, we requested and analyzed the quarterly project  management reports summarizing the investment’s cost and schedule  performance. We also identified OMB reporting requirements for  measuring progress towards useful segments and analyzed IRS’s  monthly cost and schedule variance reports and the quarterly reports to  Congress that are derived from them against these requirements.
For our second objective, we selected seven investments: Customer  Account Data Engine (CADE) 2; Modernized e-File (MeF); IRS.gov— which is comprised of the Public User Portal, Registered User Portal, and  Employee User Portal—Return Review Program (RRP); Information  Reporting and Document Matching (IRDM); e-services; and ACA. We  performed a detailed analysis of the cost and schedule performance for  these seven investments in our April 2013 review of IRS’s major  investments summarized in our prior report. We selected them again for  this review because (1) we identified reliability issues in the April 2013  review that we wanted to follow up on, and (2) these investments are still  critical to IRS’s mission. As noted earlier, while ACA is not classified as  major investment in fiscal year 2013, we included it in our selection  because of its criticality to IRS’s mission and expected costs. In addition,  IRS began reporting on it as a major investment in fiscal year 2014.
We identified how each of the six investments in our selection which  reported monthly cost and schedule variances during fiscal year 2013  calculated projected amounts. We interviewed IRS officials responsible  for performing these calculations. We also determined the extent to which  these variances were included in the monthly reports by reviewing  monthly variance reports from October 2012 through September 2013. In  addition, to determine, for these investments, the extent to which the IRS  included variance data for completed activities in its external reporting  within the 60-day time frame specified in Treasury guidance, we analyzed  the four quarterly reports on the performance of IT investments submitted  by IRS to the appropriations committees and GAO between December  2012 and September 2013. Further, we identified all the activities  included in the variance calculations by analyzing monthly variance  reports and supplementing our analyses with interviews of IRS officials  cognizant of the manner in which IRS reports these variance data to  Treasury, including the Acting Manager of the IRS’s Strategy and  Planning group which has oversight responsibility for this reporting. We  recently reported that IRS did not have guidance from Treasury for  calculating the variances associated with in-process activities. As such,  we did not perform a qualitative assessment of the investments’ process  for calculating cost and schedule variances.
To address our third objective, we interviewed IRS officials to determine  what progress the agency had made in developing a quantitative  measure of progress in delivering scope as we recommended in June  2012. When we learned IRS had not addressed our recommendation,  we followed up on the causes, and, for each of the investments in our  selection, identified the qualitative methods IRS said it was using instead.  We also analyzed the “functional performance” information IRS has  begun to include in its quarterly reports to the congressional committees  and GAO to determine the extent to which it completely and clearly  describes how delivered scope compares to what was planned.
We conducted this performance audit from June 2013 to April 2014 in  accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.
section_title: Appendix II: Reasons for Significant Variances
paragraphs: In the Treasury Chief Information Officer’s comments on the federal IT  dashboard, IRS provided the specific reasons why its investments  experienced significant cost and schedule variances. development work and (2) contractor over-estimates of development  costs.
IMF experienced a cost underrun in October 2012. The variance  noted was due to resource contention—staff were pulled off of IMF to  work on CADE 2 development and testing.
IRDM experienced cost underruns from April 2013 to September  2013. The variances noted were due to, among other things, (1)  delivery partner services costing less than planned because of a delay  in the contract award being utilized and (2) costs were less than  planned because resources were reduced on the Business Master  File Analytics project and utilized on the Case Inventory Selection and  Analytics project.
Integrated Data Retrieval System experienced a cost underrun in  September 2013. The variance noted was due to actual labor costs  being lower than planned due in part to retirements, attrition, and the  government-wide pay freeze.
For investments that were significantly behind schedule, the Treasury  Chief Information Officer’s comments on the federal IT dashboard  identified the following reasons:
End User Systems and Services experienced a schedule overrun in  October 2012. The variance noted was due to several factors: (1)  scope changes and new funding and resource requirements; (2) a  schedule change for site assessments; and (3) the need for an  additional site walk through.
SCRIPS experienced a schedule overrun from February 2013 to April  2013 because the Technical Refresh project was delayed due to  issues with the procurement of the required and approved hardware  and software. A baseline change request was submitted and  approved in April 2013.
For ACA, MeF, and TSS, IRS reported the following reasons:
ACA experienced a cost overrun for the last quarter of fiscal year. The  overrun noted was due to a timing difference between planned  obligations and actual contract award dates. ACA also experienced  cost underruns throughout the first three quarters of the fiscal year.  The underruns noted were due to (1) delays in contract awards, (2)  re-evaluation of the needs for technical contractor support, and (3)  underrun in IRS labor because of slower than expected hiring due to  sequestration.
MeF experienced a cost overrun in October 2012. The variance noted  was due to a delay in the roll out of new forms and added  performance tuning stabilization and monitoring into the production  system; and baseline change request submitted. Additionally, MeF  experienced cost underruns from May 2013 to June 2013 and from  August 2013 to September 2013. The variances noted were due to (1)  the under-budget cost as the result of significantly lower infrastructure  costs than originally anticipated and the decoupling of XML  enhancement work, (2) lower infrastructure costs, (3) removal of  scope, and (4) fewer than anticipated change requests received from  business partners.
Telecommunications Systems and Support experienced schedule  overruns from November 2012 through September 2013. The  variances noted were due to (1) delays associated with two projects— Video Conferencing and Enterprise Electronic Fax and Networks  Convergence, (2) construction delays associated with Hurricane  Sandy, and (3) delays associated with the phone carrier providing toll  free numbers. A baseline change request was submitted to address  the schedule variance. The investment also experienced cost  underruns from July 2013 to September 2013. The variances noted  were due to (1) changes in project scope, (2) projected costs being  incorrectly reported, and (3) a reduction in planned spending. A  baseline change request was submitted to correct projected cost  figures.
section_title: Appendix III: Investment Variances Reported by IRS from October 2012 to September 2013
paragraphs: The following table shows significant cost and schedule variances reported monthly for IRS’s major investments from  October 2012 to September 2013.
section_title: Appendix IV: Investment Profiles
paragraphs: This appendix contains the profiles for the seven key investments we  reviewed. Information contained within these profiles includes, but is not  limited to:  Current life-cycle phase: Life-cycle phases can be represented as  planning; development, modernization, and enhancement;  operations and maintenance; or mixed. Planning refers to preparing,  or acquiring the information used to design the asset; assess the  benefits, risks, and risk-adjusted costs of alternative solutions; and  establish realistic cost, schedule, and performance goals for the  selected alternative, before proceeding to full acquisition or  termination of a project. Development, modernization, and  enhancement refers to projects and activities that result in new  assets/systems or projects and activities that result in changes or  modifications to existing assets that lead to substantive  improvements, implement legislative or regulatory requirements, or  meet an agency leadership request. Operations and maintenance  refers to those projects and activities that are operating in a  production environment. Finally, mixed refers projects and activities  that are a combination of development, modernization, and  enhancement and operations and maintenance. Having detailed  information allows for clear tracking of a program’s costs as it moves  through its various life-cycle phases.
Development methodology: This is a framework that is used to  structure, plan, and control the process of developing an information  system. There are a number of approaches that can be utilized by an  investment. IRS’s Enterprise Lifecycle methodology includes the  following approaches: waterfall, planned maintenance, iterative, and  managed services. Waterfall is a sequential development of a  solution with planned reviews and formal approvals required before  continuation of work. The planned maintenance approach manages  change in an organized manner, minimizes the disruption caused by  frequent system changes, and increases the efficiency and  effectiveness of the system change process. Additionally, the  iterative approach is an adaptive development approach in which  projects start with a conceptual vision of the solution and end with  deployment, with repeated cycles of requirement discovery,  development, and testing in between. Finally, the managed services  approach is designed to capitalize on the benefits of managed  services provided by either an outside service, internal business  processes, and/or existing infrastructure service provider. This  provides useful information on the requirements of how a project is to  progress through the life cycle.
Contract type: For purposes of this report, this can be broken down  into two categories. The first is firm, fixed price contracts in which the  price is not subject to any adjustments. The second is cost  reimbursement contracts which provide for the payment of allowable  incurred costs, to the extent prescribed in the contract. Types of cost  reimbursement contracts include, but are not limited to (1) cost plus  fixed fee in which actual costs and a fixed fee can be charged;  however, costs are not allowed to exceed the agreed upon estimate  without approval; and (2) cost plus incentive fee that provides for an  initially negotiated fee to be adjusted later by a formula based on the  relationship of total allowable costs to total target costs.
Number of rebaselines: Rebaselines are changes to projects’ cost,  schedule, and performance goals (i.e., baselines). According to  officials, scope changes must go through a baseline change request  process and be approved by Treasury and OMB.
subsections: section_title: Affordable Care Act
paragraphs: Current life-cycle phase: Mixed  (development/operations and maintenance)
section_title: Customer Account Data Engine 2 (CADE 2)
paragraphs: CADE 2  Investment Details  Program start date: January 2010 Full operational capability: 2020 Life-cycle costs: $1.48 billion  Current life-cycle phase: Mixed  (development/operations and maintenance)
CADE 2 began in 2010 as a new strategy for accelerating completion of a  modernized database and converting to a single processing system  sooner than CADE (which was intended to provide a modernized system  of taxpayer accounts, with the ultimate goal of eventually replacing IMF)  would allow. CADE 2 is expected to deliver its functionality incrementally  through transition states. Transition state 1 includes: (1) daily batch  processing of individual taxpayer returns provided by modifying the IMF to  run on a daily, rather than weekly, basis; and (2) a comprehensive  database for housing all individual taxpayer accounts and loaded with  data from CADE and IMF to provide timelier updates of taxpayer  information for use by IRS employees for compliance and customer  service. Transition state 2, which IRS started in October 2010 and IRS  plans to complete in March 2015, includes re-writing IRS’s legacy core  tax processing applications in modern programming language, and is  intended to increase flexibility, scalability, reliability, and security.
According to IRS, the database component of transition state 1 has been  in production as of May 2012, and in August 2012, daily updates to the  database began. In addition, according to IRS, intense focus was placed  on optimizing performance of the CADE 2 database in 2013. For 2014,  the database will be kept in sync with the Individual Master File; data  quality will be validated through the 2014 filing season; and risk-based  decisions will be used to determine readiness to feed data downstream to  other IRS systems.  e-Services is a suite of web-based products that are intended to allow tax  professionals and payers to conduct business with IRS electronically.  These services are only available to tax practitioners, registered agents,  and other third parties and are not available to the general public. The  program is available via the Internet 24 hours a day, 7 days a week, and it  contains products such as registration, an e-file application, a Transcript  Delivery System (a system which tax professionals may use to request  and receive account transcripts, wage and income documents, tax return  transcripts, and verification of non-filing letters), and Taxpayer  Identification Number Matching (a pre-filing service which allows  authorized payers to match up to 25 payee taxpayer identification number  and name combinations against IRS records prior to submitting an  information return).
section_title: Information Reporting and Document Matching (IRDM)
paragraphs: IRDM is aimed at helping close the tax gap—the difference between what  business taxpayers should have paid and actually did. It is intended to  improve voluntary compliance and accurate reporting of income by  establishing a new business tax return and information returns that focus  on merchant card payments and securities basis reporting. IRDM  supports IRS business utilizing information systems that sort, match,  identify, manage, and report on returns that are likely sources of tax gap- reducing revenue. To accomplish this, IRS requires operational resources  and systems to be put in place to implement business and technology  changes that are intended to expand and improve its automated matching  of data on information returns to the data submitted on tax returns filed.  The program consists of the following five projects that the IRS  implemented in 2011 and 2012:
Data Assimilation: Identifies the link between tax forms and  information returns filed for the same taxpayer to identify potential  under-reporter cases. The project then groups these into specific  categories to support IRS compliance programs associated with  merchant card payments, securities cost basis, and government  payments.
Data Correlation: Matches tax return and information return data and  applies business rules to identify potential under-reporter cases for  use in the IRDM case selection process. After case selection, Data  Correlation builds a complete case record for analysis by a tax  examiner to support IRS compliance programs.
Business Master File Analytics: Provides IRS users the ability to  define and execute logic for the intelligent selection of business  taxpayer case inventory to ensure cases selected result in the largest  financial return.
Case Inventory Selection and Analytics: Provides IRS users the  ability to define and execute logic for the intelligent selection of  individual taxpayer case inventory and creates an analytical  environment that offers a greater ability to evaluate case data to  improve the selection of cases worked.
Case Management: Enables IRS users to manage and work  business cases selected through Business Master File Analytics with  identified discrepancies that could potentially affect tax liability.
section_title: IRS .Gov
subsections: section_title: IRS.Gov  Investment Details Program start date: July 2009
paragraphs: The IRS.Gov investment consists of a public user portal—IRS.Gov, a  registered user portal, and an employee user portal. The key goals of the  program include simplifying and transforming the user web experience,  consolidating and advancing IRS web technology to industry standards,  implementing a high-performing contract structure and terms, and  marketing competitive costs throughout the program’s life cycle.
Additionally, this effort supports the framework and functionality of IRS  portals services used by the American taxpayer, IRS employees, and  registered users. The overall objectives of this program are to: provide a high-performing partnership between IRS and its contractor; provide industry-leading web practices and innovations; provide a compelling program performance management framework; create a new IRS web environment that is the trusted taxation
Create a simple and manageable IRS web environment; provide single point, end-to-end operational accountability and provide a cost effective and affordable program cost structure; and  transition successfully from the old programs to the new program.
section_title: Modernized e-file (MeF)
paragraphs: MeF is the primary system to receive and process all tax returns  submitted electronically. When MeF receives an electronic tax return, the  system determines if it satisfies the acceptance rules required for further  processing. MeF is intended to benefit the tax preparation community and  enables the IRS to answer questions quickly and helps to resolve issues.  MeF is also intended to benefit corporations and tax-exempt  organizations that must file tax returns or annual information returns  electronically and is intended to reduce the handling/mailing of  voluminous paper returns.
MeF stores all tax return data in Extensible Markup Language format in a  Modernized Tax Return Database, allowing authorized IRS viewers (IRS  Help Desk personnel and tax examiners) to see tax returns securely  online. According to IRS, as of September 10, 2013, taxpayers used MeF  to submit over 223 million individual returns and nearly 15 million  business returns.
Release 9, which IRS planned to deploy for Filing Season 2014, is  intended to add the Employment/Unemployment Tax family of forms  (Forms 94x) and the U.S. Income Tax Return for Estates and Trusts  (Form 1041) to the MeF environment, as well as a new Return Review  Program interface.
section_title: Return Review Program (RRP)
paragraphs: RRP is a web-based automated system that is currently in development,  and is intended to replace the legacy Electronic Fraud Detection System  (EFDS) built in the mid-1990s. It is intended to deliver functionality  incrementally through transition states. Transition State 1 is scheduled to  be completed in March 2014. Transition State 2, the final transition state,  is scheduled to be completed in March 2015. RRP is to provide, among  other things, the following functionality:  enable more effective routing of returns;  detect noncompliant and fraudulent returns;  ensure timely issuance of refunds and credits;  prevent issuance of refunds and credits not legally due to filers; and  streamline business processes used by the IRS criminal investigative  analysts, aids, and civil employees.
The new system is comprised of three major activities:
Detection. The system will incorporate several existing models as  well as new models to enhance detection of probable  noncompliance. Using algorithms and business rule sets, the system  will detect questionable information on each return as the return is  processed. The system will also detect returns with potential fraud  characteristics, thereby allowing criminal investigators to link and  analyze groups of returns to identify schemes for potential criminal  prosecution.
Resolution. The system will accommodate existing treatment  streams and new treatment streams. Returns will be routed  systemically to the best treatment stream, opened into the treatment  stream’s inventory and, if applicable, the system will send an initial  contact letter to the taxpayer.
Prevention. The system will automatically integrate the results of  each return’s resolution into the detection models. The results can be  used to help target education and outreach efforts to taxpayers and  preparers on how to avoid unintentional noncompliance. The system  will also allow analysis and identification of fraud and noncompliance  not identified by the predictive detection models.
section_title: Appendix V: Comments from the Department of the Treasury
section_title: Appendix VI: GAO Contact and Staff Acknowledgements
subsections: section_title: GAO Contact
section_title: Staff Acknowledgements
paragraphs: In addition to the contact name above, individuals making contributions to  this report included Sabine Paul (Assistant Director), Mary Evans,  Rebecca Eyler, Paul Middleton, and Bradley Roach.