Information Technology: IRS Needs to Improve Its Processes for Prioritizing and Reporting Performance of Investments
Published: 2016-06-29T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: IRS relies extensively on IT systems to annually collect more than $2 trillion in taxes, distribute more than $300 billion in refunds, and carry out its mission of providing service to America's taxpayers in meeting their tax obligations. For fiscal year 2016, IRS planned to spend approximately $2.7 billion for IT investments. Given the size and significance of these expenditures, it is important that Congress be provided information on agency funding priorities, the process for determining these priorities, and progress in completing key IT investments.
Accordingly, GAO's objectives were to (1) describe IRS's current IT investment priorities and assess IRS's process for determining these priorities, and (2) determine IRS's progress in implementing key IT investments.
To do so, GAO analyzed IRS's process for determining its fiscal year 2016 funding priorities, interviewed program officials, and analyzed performance information for six selected investments for fiscal year 2015 and the first quarter of 2016.
section_title: What GAO Found
paragraphs: The Internal Revenue Service (IRS) has developed information technology (IT) investment priorities for fiscal year 2016, which support two types of activities—operations and modernization. For example, it has developed priority groups for operations such as: (1) critical business operations, infrastructure operations, and maintenance; and (2) delivery of essential tax administration/taxpayer services. It has identified priorities for modernization, such as web applications, to help reach IRS's future state vision. However, while IRS has developed a structured process for allocating funding to its operations activities consistent with best practices, it has not fully documented this process. IRS officials stated this is because the process is relatively new and not yet stabilized. In addition, IRS does not have a structured process for its modernization activities, because, according to officials, there are fewer competing activities than for operations activities. Fully documenting a process for both operations support and modernization activities that is consistent with best practices would provide transparency and greater assurance it is consistently applied.
Of the six investments GAO reviewed, two investments—Foreign Account Tax Compliance Act and Return Review Program—provided complete and timely performance information for GAO's analyses. These investments performed under cost, with varying schedule performance, and delivered most planned scope (see table). However, IRS did not always use best practices for determining scope delivered. Specifically, IRS used a method inconsistent with best practices for determining the amount of work completed by its own staff.
Two other investments reported completing portions of their work on time and $1.7 million under planned costs (for the Customer Account Data Engine 2), and on time and $10.3 million under planned costs (for Affordable Care Act Administration). However, neither investment reported information on planned versus actual delivery of scope in accordance with best practices. The remaining two investments—Mainframes and Servers Services and Support and Telecommunications Systems and Support—generally met performance goals.
section_title: What GAO Recommends
paragraphs: GAO is recommending that IRS develop and document its processes for prioritizing IT funding and improve the calculation and reporting of investment performance information. IRS agreed with two recommendations regarding its prioritization processes, disagreed with one related to the calculation of performance information, and did not comment on one recommendation. GAO maintains all of the recommendations are warranted.

section_title: Letter
paragraphs: The Internal Revenue Service (IRS) relies extensively on information  technology (IT) systems to annually collect more than $2 trillion in taxes,  distribute more than $300 billion in refunds, and carry out its mission of  providing service to America’s taxpayers in meeting their tax obligations.  For fiscal year 2016, IRS planned to spend approximately $2.7 billion for  IT investments.
Given the size and significance of these expenditures and the challenges  inherent in successfully delivering complex systems, it is important that  Congress be provided information on IRS’s investment priorities and  process for determining these priorities, as well as timely performance  information for key IT investments. Accordingly, you have asked that we  review IRS’s IT investment priorities and key IT efforts. Specifically, our  objectives were to (1) describe IRS’s current IT investment priorities and  assess IRS’s process for determining these priorities, and (2) determine  IRS’s progress in implementing key IT investments.
To address our first objective, we reviewed documentation, including  documents supporting IRS’s Portfolio Investment Planning process used  to allocate funding for operations support activities. Further, we  interviewed officials in IRS’s Office of Strategy and Planning who have  primary responsibility for the IT funding allocation process, as well as  officials from IRS business units who were involved in carrying the  process out. We analyzed the process against relevant best practices in  GAO’s IT Investment Management Framework, as well as the Office of  Management and Budget’s Capital Programming Guide.
For our second objective, we analyzed the performance of six key  investments—four development investments (Customer Account Data  Engine 2 (CADE 2), Return Review Program (RRP), Foreign Account Tax  Compliance Act (FATCA), and the Affordable Care Act Administration  (ACA)) and two key operational investments (Telecommunications  Systems and Support (TSS) and Mainframes and Servers Services and  Support (MSSS)). We chose these investments because they  represented a large portion of IRS’s development and operations  expenditures for fiscal year 2015. For the development investments, we  obtained and analyzed information relative to expected and actual cost,  schedule, and scope for the work IRS was performing for fiscal year 2015  and the first quarter of fiscal year 2016. For the operational investments,  we analyzed reported operational performance information from October  2014 to September 2015 to determine the extent to which the  investments met operational performance goals.
We conducted this performance audit from September 2015 to June 2016  in accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives. Details on our objectives,  scope, and methodology can be found in appendix I.
subsections: section_title: Background
paragraphs: The mission of IRS, a bureau within the Department of the Treasury, is to  provide America’s taxpayers top quality service by helping them  understand and meet their tax responsibilities and by applying the federal  tax laws with integrity and fairness to all. In carrying out its mission, IRS  annually collects over $2 trillion in taxes from millions of individual  taxpayers and numerous other types of taxpayers and manages the  distribution of over $300 billion in refunds. To guide its future direction,  the agency has two strategic goals: (1) deliver high quality and timely  service to reduce taxpayer burden and encourage voluntary compliance;  and (2) effectively enforce the law to ensure compliance with tax  responsibilities and combat fraud.
IRS has established seven overarching priorities to accomplish its  mission:  facilitate voluntary compliance by empowering taxpayers with secure  and innovative services, tools, and support; understand non-compliant taxpayer behavior, and develop  approaches to deter and change it;  leverage and collaborate with external stakeholders;  cultivate a well-equipped, diverse, skilled, and flexible workforce;  select highest value work using data analytics and a robust feedback  loop; drive more agility, efficiency, and effectiveness in IRS operations; and strengthen cyber defense and prevent identity theft.
The mission of IRS’s Information Technology organization is to deliver IT  services and solutions that drive effective tax administration to ensure  public confidence. It is led by the Chief Technology Officer, who reports to  the Deputy Commissioner for Operations Support of the IRS. Several  subordinate offices report to the Chief Technology Officer. Figure 1 shows  the structure of IRS’s Information Technology organization.
subsections: section_title: IRS Relies on IT to Carry Out Its Mission
paragraphs: IT plays a critical role in enabling IRS to carry out its mission and  responsibilities. For example, the agency relies on information systems to  process tax returns, account for tax revenues collected, send bills for  taxes owed, issue refunds, assist in the selection of tax returns for audit,  and provide telecommunications services for all business activities,  including the public’s toll-free access to tax information.
For fiscal year 2016, IRS is pursuing 23 major and 114 non-major IT  investments to carry out its mission. These investments generally support  (1) day-to-day operations (which include operations and maintenance, as  well as development, modernization, and enhancements to existing  systems), and (2) modernization efforts in support of IRS’s future goals.  The day-to-day operations are primarily funded via the operations support  appropriation account, user fees and other supplemental funding. The  modernization efforts are funded via the business systems modernization  appropriation account. IRS expects to spend about $2.7 billion for IT,  including $2.2 billion in appropriated funds, $391.9 million in user fees,  and $108.2 million in other supplemental funding.
Approximately $1.4 billion of IRS’s IT funding for fiscal year 2016  supports the two operational investments (TSS and MSSS), and four  development investments (FATCA, ACA, CADE 2, and RRP) that we  selected for review.
TSS supports IRS’s network infrastructure services such as network  equipment, video conference service, enterprise fax service, and  voice service for over 85,000 IRS employees at about 1,000 IRS  locations. According to IRS, the investment continues delivery of  services and products to employees which translates into service to  taxpayers. IRS allocated approximately $366.6 million to activities  supporting the TSS investment. Table 1 identifies the fiscal year 2016  funding allocation for the TSS investment, as well as the types of  activities being funded.
MSSS provides for the design, development, and deployment of  server, middleware, and large systems as well as enterprise storage  infrastructures, including systems software products, databases, and  operating systems for these platforms. For fiscal year 2016, IRS  allocated approximately $454.2 million for activities supporting the  MSSS investment. Table 2 identifies the fiscal year 2016 funding  allocation for the MSSS investment, as well as the types of activities  being funded.
FATCA is intended to improve tax compliance by identifying U.S.  taxpayers that attempt to shield or divert assets by depositing funds in  foreign accounts. A law enacted in 2010 requires foreign financial  institutions to report to the IRS information regarding financial  accounts held by U.S. taxpayers or foreign entities in which U.S.  taxpayers have a substantial ownership interest. IRS allocated $89.1  million to FATCA for fiscal year 2016.
ACA encompasses the planning, development, and implementation of  IT systems needed to support IRS’s tax administration responsibilities  associated with parts of the Patient Protection and Affordable Care  Act. IRS allocated $311.2 million to ACA for fiscal year 2016.
CADE 2 is intended to provide daily processing of taxpayer accounts.
A major component of the program is a modernized database for all  individual taxpayers that is intended to provide the foundation for  more efficient and effective tax administration. In Transition State 2 of  the initiative, the modernized database will become IRS’s authoritative  source for taxpayer account data, as it begins to address core  financial material weakness requirements for individual taxpayer  accounts. Existing financial reports will be modified to take into  account the increased level of detail and accuracy of data in the  database. CADE 2 data will also be made available for access by  downstream systems such as the Integrated Data Retrieval System  for online transaction processing by IRS customer service  representatives. IRS allocated $129.9 million to CADE 2 for fiscal year  2016.
RRP is intended to deliver an integrated and unified system that  enhances IRS’s capabilities to detect, resolve, and prevent criminal  and civil tax noncompliance. In addition, it is intended to allow  analysis and support of complex case processing requirements for  compliance and criminal investigation programs during prosecution,  revenue protection, accounts management, and taxpayer  communications processes. IRS allocated $91.7 million to RRP for  fiscal year 2016.
section_title: GAO Has Reported on IRS’s Major IT Investments and Identified Needed Improvements
paragraphs: Over the past several years, we have issued a series of reports which  have identified opportunities for IRS to improve the management of its  major IT investments.
We reported in June 2012 that while IRS reported on the cost and  schedule of its major IT investments and provided chief information  officer ratings for them, the agency did not have a quantitative  measure of scope–a measure that shows functionality delivered. We  noted that having such a measure is a good practice as it provides  information about whether an investment has delivered the  functionality that was paid for.
We recommended that IRS develop a quantitative measure of scope,  at a minimum for its major IT investments, to have more complete  information on the performance of these investments. In December  2015, IRS officials told us that they were exploring options to report  scope and proposed an option in a December 2015 quarterly report  on IT to Congress. We examined the suitability of proposed solutions  for a quantitative measure of scope as part of this review.
Further, in April 2013 we reported that the majority of IRS’s major IT  investments were reportedly within 10 percent of cost and schedule  estimates and eight major IT investments reported significant cost  and/or schedule variances. We also reported that weaknesses  existed, to varying degrees, in the reliability of reported cost and  schedule variances, and key risks and mitigation strategies were  identified. As a result, we made recommendations for IRS to improve  the reliability of reported cost and schedule information by addressing  the identified weaknesses in future updates of estimates. We also  recommended that IRS ensure projects consistently follow guidance  for updating performance information 60 days after completion of an  activity and develop and implement guidance that specifies best  practices to consider when determining projected amounts.
IRS agreed with three of our four recommendations and partially  disagreed with the fourth recommendation. The agency specifically  disagreed with the use of earned value management data as a best  practice to determine projected cost and schedule amounts, stating  that the technique was not part of IRS’s current program management  processes and the cost and burden to use it outweighed the value  added. While we disagreed with IRS’s view of earned value  management because best practices have found that the value  generally outweighs the cost and burden of implementing it, we  provided it as one of several examples of practices that could be used  to determine projected amounts. We also noted that implementing our  recommendation would help improve the reliability of reported cost  and schedule variance information, and that IRS had flexibility in  determining which best practices to use to calculate projected  amounts.
Finally, our February 2015 report found that most of IRS’s major IT  investments reported meeting cost and schedule goals; however,  selected investments experienced variances from initial cost,  schedule, and scope plans that were not transparent in reports to  Congress because IRS had yet to address our prior  recommendations. Specifically, IRS had not addressed our  recommendation to report on how delivered scope compares to what  was planned, and also did not address guidance for determining  projected cost and schedule amounts, or the reporting of cumulative  cost and schedule performance information.
section_title: IRS Has Identified IT Priorities for Fiscal Year 2016, but Does Not Have a Process for Prioritizing Modernization Efforts
paragraphs: IRS has identified priorities for operations and modernization but does not  have a structured process for prioritizing among modernization efforts.  Specifically, IRS has developed eight priority groups for operations, such  as the delivering essential tax administration and taxpayer services  group, and identified eight priority projects for modernization, including  CADE 2 and RRP, to help reach IRS’s future state vision. In addition, IRS  has developed a structured process for allocating funding to its operations  support activities which is consistent with best practices. However, IRS  has not fully documented this process. In addition, IRS does not have a  similar structured process for prioritizing funding among its modernization  activities, stating it does not have such a process because there are  fewer competing activities than for operations support. A documented  process for both operations support and modernization activities that is  consistent with best practices would provide transparency into the  process and provide greater assurance it is consistently applied.
subsections: section_title: IRS Has Identified Priorities for Operations Support and Modernization
paragraphs: IRS has identified eight priorities—referred to as repeatable priority  groupings—for its operations support activities. Officials told us that these  priorities evolved from lessons learned in using priorities established the  prior year and noted that they will continue to be refined over time. For  example, in fiscal year 2015, activities associated with the tax filing  season were identified as IRS’s top priority; however, in fiscal year 2016,  IRS decided that infrastructure (i.e., telephones and computer servers)  was essential in supporting tax processing and should thus be classified  as IRS’s top priority. Each of the priority groupings includes several  supporting business activities associated with major and non-major  investments that IRS allocates funding to. Examples of such business  activities include enterprise video conferencing service, and print support  for taxpayer notices. These priorities and information related to these  priorities are identified in order of importance, as determined by IRS, in  table 3.
IRS has also identified eight priority projects for modernization. These  projects, as well as their descriptions and associated funding allocations  are identified in table 4.
section_title: IRS Has a Structured Process for Prioritizing Funding for Its Operations Support Activities, but Lacks a Similar Process for Modernization Activities
paragraphs: According to GAO’s Information Technology Investment Management  Framework, an organization should document policies and procedures for  selecting new and reselecting ongoing IT investments. These policies  and procedures should include criteria for making selection and  prioritization decisions. A policy-driven, structured method for reselecting  ongoing projects provides the organization’s investment board with a  common understanding of how ongoing projects will be reselected for  continued funding. In addition, executives funding decisions should be  aligned with the selection decision. Specifically, the organization’s  executives have discretion in making the final funding decisions on IT  proposals. However, their decisions should be based upon the analysis  that has taken place in the previous activities.
Further, the Office of Management and Budget’s (OMB) Capital  Programming Guide requires, among other things, that agencies have a  disciplined capital programming process that addresses project  prioritization and comparison of assets against one another to create a  prioritized portfolio.
In 2015, IRS developed and implemented a process known as the  Portfolio Investment Planning process to prioritize its operations support  activities. This process addresses (1) prioritization and comparison of IT  assets against each other and (2) criteria for making selection and  prioritization decisions. Further, senior IRS executives stated that the final  funding decisions on IT proposals are based on IRS’s prioritization  process.
IRS uses priority groupings it has defined as criteria for making  prioritized selections. Specifically, a consideration in determining if an  activity (i.e., request for funding) will be selected is to determine the  extent to which it supports any of eight priority groupings. If the activity  is found to support one of the eight priorities, it is further assigned one  of four priority levels: must do, high, medium, or low. IRS has defined  the criteria that must be met in order to classify a funding activity at a  particular priority level. Table 5 provides an example of the criteria  used to make these decisions for the legislative provisions for the  FATCA and ACA priority group.
IRS prioritizes and compares IT assets against each other.  Specifically, IRS business units identify line item activities for which  they are requesting funding. For each activity, business units address,  among other things, placement within IRS’s established priorities;  proposed high-level capabilities and a cost estimate; a 1-year usable  segment; and the date funding is needed and subsequent mitigation  strategy if funding is not received by the specified date. Further,  several meetings are held to review requested funding activities.  According to IRS, the purpose of these meetings is to provide a cross- organization review and evaluation of IT-related demands.  Stakeholders include Associate Chief Information Officers, business  unit representatives, and staff from IRS’s IT Financial Management  Service.
Finally, IRS senior executives stated that its final funding decisions on  IT proposals are based on IRS’s prioritization process. According to  these officials, when the agency receives its appropriation, it  evaluates prioritized activities—starting with the highest priority  demands—until the total estimate of appropriated funding is allocated.  Officials have discussions relative to the items that will not be funded  and then engage the Office of the Chief Financial Officer to determine  the extent to which user fees and other sources of funding are  available to support priorities that exceed the appropriated amount.  Prioritized activities, which have been allocated funding for the  upcoming fiscal year, are presented to the Chief Technology Officer  for approval. IRS’s Senior Executive Team approves the Chief  Technology Officer’s funding recommendations and submits the  recommendations to the Commissioner and Deputy Commissioners  for final funding approval.
Despite these strengths, IRS has not fully documented its process for  prioritizing operations support activities. Specifically, while several  documents describe aspects of the operations support prioritization  process, including the criteria used and the meetings to review and  evaluate IT related demands, none fully describe the procedures  associated with the process. IRS officials stated this is because it is  relatively new and not yet stabilized. IRS officials who are stakeholders in  this process stated that documentation would have reduced the  uncertainty they faced during implementation and would have helped  them to better prepare the required data for the process. IRS senior  executives stated they plan to fully document this process; however, they  did not identify a time frame for when this would be done. Fully  documenting IRS’s portfolio investment process for operational activities  would help ensure consistent implementation of the process by all  stakeholders and provide transparency regarding how such prioritization  decisions are made.
In contrast with operations support, IRS does not have a structured  process for prioritizing funding among its modernization investments.  Specifically, IRS officials stated that discussions are held to determine the  modernization efforts that are the highest priority to meet IRS’s future  state vision and technology roadmap. Officials reported that staffing  resources and lifecycle stage are considered but there are no formal  criteria for making final determinations.
Senior IRS officials stated that they do not have a structured process for  selection and prioritization of business systems modernization activities  because the projects are set and there are fewer competing activities  than for operations support. While there may be fewer competing  activities, a structured, albeit simpler, process that is documented and  consistent with best practices would provide transparency into IRS’s  needs and priorities for appropriated funds. Such a process would better  assist Congress and other decision makers in carrying out their oversight  responsibilities.
section_title: Performance Varied for Selected Investments
paragraphs: Of the six selected investments in our review, two development  investments—FATCA and RRP—performed under cost, with varying  schedule performance, and delivered most of the scope that was  planned; however, performance information for these investments could  be improved by implementing best practices for determining actual work  performed. For portions of the two other development investments (CADE  2 and ACA) for which performance information was available, IRS  reported completing work under planned cost and on time. However,  neither investment reported information on planned versus actual delivery  of scope, in accordance with best practices. Further, ACA did not report  timely information on planned versus actual costs. Finally, one of the two  investments in operations and maintenance (MSSS) met all operational  performance goals, while the other investment in operations and  maintenance (TSS) met six out of eight goals.
subsections: section_title: Four Investments in Development Experienced Variances and Performance Reporting for These Investments Could Be Improved
paragraphs: Best practices highlight the importance of timely reporting on performance  relative to cost, schedule, and scope (both planned and actual).  According to these practices, one way to measure benefits of  development work is to approximate by measuring a project’s actual cost  and schedule progression (i.e., evaluating earned value), which is a  measure of the amount of planned work that is actually performed in  relation to the funds expended. IRS reported metrics for FATCA and  RRP, which allowed us to determine these investments’ performance.  The agency did not use such metrics or consistently develop planned and  actual cost, schedule, and scope information for all CADE 2 and ACA  projects and activities that were completed or ongoing during fiscal year  2015 and the first quarter of fiscal year 2016. As a result, we could only  determine the performance of portions of these investments.
FATCA and RRP: During fiscal year 2015 and the first quarter of fiscal  year 2016, IRS reported quarterly cost, schedule, and scope performance  information for each of the FATCA and RRP projects it was working on.  Specifically, it reported metrics for these investments via its Investment  Performance Tool. Table 6 summarizes the performance of the FATCA  and RRP investments (see appendix II for detailed analyses).
As shown in table 6, FATCA and RRP performed under cost, with varying  schedule performance, and delivered most of the scope that was  planned. Specifically,  IRS was developing 10 projects to support the FATCA investment  during fiscal year 2015 and the first quarter of 2016. IRS reported  completing work at $12.4 million less than budgeted and delivering  91.7 percent of planned scope with an 8 percent schedule overrun for  these projects. IRS stated that the reasons for these variances  include, among other things, issues with the requirements  management process; an overestimation of costs; and a reduction in  the amount of work completed versus what was planned.
IRS was developing three projects to support the RRP investment  during fiscal year 2015 and the first quarter of 2016. IRS reported  completing work at $24.5 million less than budgeted and delivering  99.9 percent of planned scope with a minor schedule overrun for  these projects. IRS stated that the reasons for these variances  include, among other things, overestimation of costs (including IRS  labor) and unplanned work that needed to be completed.
While the scope metric used for FATCA and RRP provides an indication  of performance, this metric would be more reliable if it incorporated best  practices for determining the amount of work completed for all activities.  Specifically, IRS uses a level of effort method beyond the amount  generally accepted by best practices to determine the amount of work  completed by its own staff. Our Cost Estimating and Assessment  Guide states that the level of effort method should be used sparingly (15  percent of the budget or less); however, the work performed by IRS staff  ranged from 22 to 100 percent of the work completed for the FATCA and  RRP projects that were ongoing during the time frame of our review. IRS  officials stated that measuring value for government work is a vague  concept to pursue. Nevertheless, revising the method for determining the  amount of work completed by IRS staff for these investments would  improve the reliability of the performance information.
CADE 2 and ACA: For the CADE 2 projects that were completed during  fiscal year 2015 and the first quarter of fiscal year 2016, IRS reported that  CADE 2 performed on time and $1.7 million under planned cost.  According to IRS, the positive cost variance for the CADE 2 investment is  the result of overestimation of costs and the ability to reuse existing code.  For the ACA activities that reported actual costs during fiscal year 2015  and the first quarter of 2016, IRS reported that ACA performed on time  and $10.3 million under planned costs. IRS stated that this variance was  primarily due to an overestimation of the labor needed to complete the  planned work. Table 7 shows the reported cost and schedule  performance for CADE 2 and ACA.
With respect to CADE 2, IRS does not report timely information on  planned versus actual delivery of scope. Specifically, a senior CADE 2  program official stated that, due to the nature of the methodology  being used to implement the projects, progress in delivering planned  scope cannot be determined until the end—after the testing phase.  For CADE 2, projects can be 16 to 60 months long. We requested  information from IRS regarding delivery of planned scope for those  projects that completed during the time frame of our review; however,  IRS was unable to provide this information.
Regarding ACA, IRS does not report timely cost or scope information.
A senior IRS official stated that the investment is being developed  using an iterative approach, the goal of which is to deliver  functionality in short increments. However, the agency does not report  actual costs for the activities comprising the projects until the activities  are completed; this delay in reporting could be as long as 9 months.  Instead, ACA calculates a cost projection, which provides an estimate  of cost to complete rather than cost of work completed, with which we  have previously identified weaknesses. In addition, IRS only  provided information on delivery of planned scope for one of the ACA  projects it was developing during the timeframe of our review.
Reporting of performance for the CADE 2 and ACA investments could  be improved by incorporating best practices for timely reporting of  cost, schedule, and scope performance information. As a result of the  lack of timely and complete performance information, Congress and  other external parties do not have pertinent information about CADE 2  and ACA with which to make oversight decisions.
section_title: One Investment in Operations and Maintenance Met All Operational Performance Goals, the Other Met the Majority of Goals
paragraphs: According to OMB’s Fiscal Year 2016 Capital Planning Guidance,  ongoing performance of operational investments is monitored to ensure  the investments are meeting the needs of the agency, delivering expected  value, and/or modernized and replaced consistent with the agency’s  enterprise architecture. To this end, OMB requires agencies to report on  at least five operational metrics for major IT investments and agencies  are specifically required to report on planned and actual operational  performance.
The two operations and maintenance investments in our review reported  on operational performance metrics, as required. MSSS met its five  operational performance goals during fiscal year 2015; however, TSS  consistently underperformed on two of its eight metrics.
Table 9 identifies the MSSS operational performance metrics, their  descriptions, and the performance against these metrics during fiscal year  2015.
IRS reported planned and actual performance for eight operational  performance metrics for TSS during fiscal year 2015. However, as  previously mentioned, TSS consistently missed operational performance  goals for two of the eight metrics. The two TSS metrics that were not met  illustrate pervasive challenges meeting its goals in deploying new  telecommunications capabilities. Specifically, IRS missed every monthly  target in fiscal year 2015 for deploying voice, video, and data  technologies. As a result, TSS did not deploy such technologies to  approximately 4,300 users that were originally included in the planned  deployment.
According to IRS officials, the operational performance goals for the two  metrics that were not met should have been updated to better reflect the  limited funding the agency intended to allocate to these activities.
Table 10 identifies the TSS operational performance metrics, their  descriptions, and the performance against these metrics during fiscal year  2015.
section_title: Conclusions
paragraphs: While IRS has developed a process for prioritizing funding for operations  support activities that adheres to best practices, it is not fully documented.  Further, IRS has not developed a priority setting process for  modernization activities for which the agency allocated nearly $300  million to for fiscal year 2016. Until IRS documents its process for  operations support activities and develops a process for modernization  activities, the agency will lack the transparency needed by Congress and  others to assist in carrying out their oversight responsibilities.
IRS has developed performance metrics for two investments—FATCA  and RRP—which include a measure of progress in delivering scope, a  measure we have been reporting on and recommending IRS address  since 2012. While these metrics represent an important step, their  reliability could be improved by incorporating best practices for measuring  the work performed by IRS staff by using the level of effort measure  sparingly. In addition, only partial performance information was available  for CADE 2 and ACA because IRS did not use the metrics it is positioned  to develop for these investments or consistently have cost, schedule, and  scope information for these investments. Continued efforts in this area  would substantially improve the performance reporting for the CADE 2  and ACA investments, and potentially for all major development efforts.
section_title: Recommendations for Executive Action
paragraphs: To help IRS improve its process for determining IT funding priorities and  to provide timely information on the progress of its investments, we  recommend that the Commissioner of IRS direct the Chief Technology  Officer to take the following four actions: document IRS’s process for selecting and prioritizing operations establish, document, and implement policies and procedures for  selecting new and reselecting ongoing business systems  modernization activities, consistent with IRS’s process for prioritizing  operations support priorities, which addresses (1) prioritization and  comparison of IT assets against each other, (2) criteria for making  selection and prioritization decisions, and (3) ensuring IRS executives’  final funding decisions on IT proposals are based on IRS’s  prioritization process; modify existing processes for FATCA and RRP for measuring work  performed by IRS staff to incorporate best practices, including  accounting for actual work performed and using the level of effort  measure sparingly; and  report on actual costs and scope delivery at least quarterly for CADE  2 and ACA. For these investments, IRS should develop metrics  similar to FATCA and RRP.
section_title: Agency Comments and Our Evaluation
paragraphs: We provided a draft of this product to IRS for comment. In its written  comments, reproduced in appendix III, IRS agreed with two  recommendations, did not agree nor disagree with one, and disagreed  with one. Specifically,  IRS agreed with our recommendations to better document its  prioritization process for operations support activities and extend that  process to its business systems modernization activities.
With respect to our recommendation to report on actual costs and  scope delivery at least quarterly for CADE 2 and ACA, IRS did not  agree nor disagree, but noted that IRS is continuing to try to improve  its processes in reporting investment performance.
Regarding our recommendation to modify existing processes for  FATCA and RRP for measuring work performed by IRS staff to  incorporate best practices, including accounting for actual work  performed and using the level of effort measure sparingly, IRS  disagreed and stated that modifying the use of the level of effort  measure would equate to a certified earned value management  system, which would add immense burden on IRS’s programs on  various fronts and would outweigh the value it provides. However, we  did not specify the use of an earned value management system in our  report and believe other methods could be used to more reliably  measure work performed. As noted in our report, 22 to 100 percent of  the work for selected projects was performed by IRS staff. As a result,  we believe that it is a reasonable expectation for IRS to reliably  determine the actual work completed, as opposed to assuming that  work is always completed as planned. Accordingly, we maintain our  recommendation is still warranted.
We are sending copies of this report to interested congressional  committees, the Commissioner of IRS, and other interested parties. This  report will also be available at no charge on our website at  http://www.gao.gov.
If you or your staffs have any questions on matters discussed in this  report, please contact me at (202) 512-9286 or pownerd@gao.gov.  Contact points for our Offices of Congressional Relations and Public  Affairs may be found on the last page of this report. GAO staff who made  major contributions to this report are listed in appendix IV.
section_title: Appendix I: Objectives, Scope, and Methodology
paragraphs: Our objectives were to (1) describe the Internal Revenue Service’s (IRS)  current information technology (IT) investment priorities and assess IRS’s  process for determining these priorities, and (2) determine IRS’s progress  in implementing key IT investments.
To address our first objective, we reviewed documentation, such as IRS’s  fiscal year 2016 Business Systems Modernization Operating Plan, as well  as financial reports to determine IRS’s IT funding priorities and funding  allocations.
In addition, we reviewed artifacts from IRS’s Portfolio Investment  Planning process, such as slide decks describing key stages of the  process, memorandums distributed to stakeholders, prioritized listings of  investment activities, and criteria for establishing priorities, to identify and  describe IRS’s process for determining its IT investment priorities.  Further, we interviewed officials in IRS’s Office of Strategy and Planning,  as well as stakeholders of the Portfolio Investment Planning process from  IRS business units. We then analyzed IRS’s processes against best  practices in our IT Investment Management Framework and the Office of  Management and Budget’s Capital Programming Guide to determine the  extent to which the processes met best practices and requirements.  Lastly, we met with officials at the Department of the Treasury who are  responsible for IT capital planning, including the Treasury Chief  Information Officer, to determine the department’s role in IRS’s process  for prioritizing IT funding.
For our second objective, we analyzed the performance of four key  development investments—Customer Account Data Engine 2 (CADE 2),  Return Review Program (RRP), Foreign Account Tax Compliance Act  (FATCA), and the Affordable Care Act Administration (ACA). Further, we  analyzed two key operational investments —Telecommunications  Systems and Support (TSS) and Mainframes and Servers Services and  Support (MSSS). We chose these investments because they represented  IRS’s most significant expenditures on development and operations for  fiscal year 2015 ($496.5 million and $777.8 million, respectively).
A tailored approach was necessary for analyzing the development  investments given the varying types and extent to which performance  information was available for these investments. To determine the  progress in implementing FATCA and RRP, we compiled and  analyzed quarterly output from IRS’s Investment Performance Tool for  the period of fiscal year 2015 through the first quarter of 2016. IRS  does not consider this tool to be a formal Earned Value Management  System. As a result, we did not evaluate the extent to which the tool  was compliant with the American National Standards Institute’s  guidelines for an Earned Value Management System. For CADE 2,  we analyzed IRS’s quarterly reporting of planned and actual costs, as  well as requirements reports and schedule reporting. For ACA, we  analyzed IRS’s financial reporting via the ACA business case  submissions, as well as performance reporting to management and  schedule reporting. In addition, we held multiple meetings with IRS  officials, including officials in the CADE 2, FATCA, ACA, and RRP  program offices.
To determine the progress in implementing TSS and MSSS, we  reviewed operational performance information reported for the  selected investments from October 2014 to September 2015; this  information included, where reported, the performance target and  actual results for each metric. In addition, we reviewed documentation  describing the performance metrics and interviewed IRS officials  regarding the process for reporting such metrics.
To determine the reliability of data used for our review, we obtained and  reviewed IRS’s guidance for its Investment Performance Tool, which  identifies, among other things, how data are to be entered within this tool,  sources of such data, and explanations of the methods used to calculate  performance metrics generated from the tool. Further, we held meetings  with officials responsible for overseeing the use of IRS’s Investment  Performance Tool. In addition, we relied on extensive work we previously  completed on IRS’s financial management system for relevant data used  for this review. In determining the reliability of the data supporting this  review, we determined that data regarding the delivery of planned scope  for the FATCA and RRP investments could be more reliable by  incorporating best practices. While these data were sufficiently reliable for  our purposes, we made recommendations to improve their reliability.
We conducted this performance audit from September 2015 to June 2016  in accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.
section_title: Appendix II: Performance Evaluation for Selected Investments in Development
paragraphs: This appendix illustrates the potential for reporting complete performance  information via IRS’s Investment Performance Tool. Specifically, the  following tables provide a detailed evaluation of cost, schedule, and  scope performance for Return Review Program and Foreign Account Tax  Compliance Act projects that were being developed by IRS during fiscal  year 2015 and the first quarter of fiscal year 2016.
subsections: section_title: Return Review Program
paragraphs: IRS reported working on three projects in support of Return Review  Program during fiscal year 2015 and the first quarter of fiscal year 2016.  The following tables identify the performance information reported via  IRS’s Investment Performance Tool; positive cost variances indicate that  the project was performing under planned cost and positive schedule  variances indicate that the project was performing ahead of schedule.
section_title: Foreign Account Tax Compliance Act
paragraphs: IRS reported working on 10 projects in support of Foreign Account Tax  Compliance Act during fiscal year 2015 and the first quarter of fiscal year  2016. The following tables identify the performance information reported  via IRS’s Investment Performance Tool; positive cost variances indicate  that the project was performing under planned cost, and positive schedule  variances indicate that the project was performing ahead of schedule.
section_title: Appendix III: Comments from the Internal Revenue Service
section_title: Appendix IV: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Staff Acknowledgments
paragraphs: In addition to the individual named above, the following staff made key  contributions to this report: Sabine Paul (Assistant Director), Bradley  Roach (Analyst in Charge), Rebecca Eyler, Charles Hubbard III, Paul  Middleton, Karl Seifert, and Marshall Williams, Jr.