Information Technology: Treasury Needs to Better Define and Implement Its Earned Value Management Policy
Published: 2008-09-22T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: In 2008, the Department of Treasury (Treasury) plans to spend approximately $3 billion on information technology (IT) investments--the third largest planned IT expenditure among civilian agencies. To more effectively manage such investments, in 2005 the Office of Management and Budget required agencies to use earned value management (EVM). EVM is a project management approach that, if implemented appropriately, provides objective reports of project status, produces early warning signs of impending schedule delays and cost overruns, and provides unbiased estimates of a program's total costs. GAO was asked to assess whether the department and its key component agencies (1) have the policies in place to effectively implement EVM and (2) are adequately using EVM techniques to manage critical system investments. GAO compared agency policies to best practices identified in the Cost Assessment Guide and reviewed the implementation of key EVM practices for several investments.
section_title: What GAO Found
paragraphs: The Department of Treasury's EVM policy is not fully consistent with best practices. Specifically, of seven best practices that leading organizations address in their policies, Treasury's policy fully addresses three, partially addresses three, and does not address the training component. According to the Director for Capital Planning and Investment Control, the department is currently working on revising its policy and according to Deputy Assistant Secretary for Information Systems and Chief Information Officer expects to finalize it by October 2008. Until Treasury develops a comprehensive policy to guide its efforts, it will be difficult for the department to optimize the effectiveness of EVM as a management tool. The department and its bureaus are not fully implementing key EVM practices needed to effectively manage their critical system investments. Specifically, the six programs at Treasury that GAO reviewed were not consistently implementing practices needed for establishing a comprehensive EVM system, ensuring that data from the system are reliable, and using the data to help manage the program. For example, when executing work plans and recording actual costs, a key practice for ensuring that the data resulting from the EVM system are reliable, only two of the six investments reviewed incorporated government costs with contractor costs. These weaknesses exist in part because Treasury's policy is not comprehensive and because the department does not have a process for ensuring effective EVM implementation. Unless the department consistently implements fundamental EVM practices, it may not be able to effectively manage its critical programs.

section_title: Letter
paragraphs: The Department of the Treasury (Treasury) and its nine bureaus rely  extensively on information technology (IT) to carry out the responsibility  of promoting the economic and financial prosperity and security of the  United States. In fiscal year 2008, the department plans to spend  approximately $3.0 billion for IT investments. This is the third largest  planned IT expenditure among civilian agencies.
To help federal departments more effectively manage their IT investments,  in 2005 the Office of Management and Budget (OMB) issued a memo that  requires departments to implement earned value management (EVM).  EVM is a project management approach that, if implemented  appropriately, provides objective reports of project status, produces early  warning signs of impending schedule delays and cost overruns, and  provides unbiased estimates of anticipated costs at completion. Given the  size and significance of Treasury’s IT investments, you asked us to  determine whether the department and its key component agencies   (1) have the policies in place to effectively implement EVM and (2) are  adequately using EVM techniques to manage critical system investments.
To address our objectives, we reviewed agency documentation including  Treasury-wide and bureau-level policies and plans governing the use of  EVM on IT acquisitions, documented EVM practices and performance  reports for six programs, and interviewed key agency officials. We  compared the department’s policies and practices to federal standards and  best practices identified in the Cost Assessment Guide to determine the  effectiveness of Treasury’s use of earned value data in managing its IT  investments.
We conducted this performance audit from August 2007 to July 2008 in  accordance with generally accepted government auditing standards. Those  standards require that we plan and perform the audit to obtain sufficient,  appropriate evidence to provide a reasonable basis for our findings and  conclusions based on our audit objectives. We believe that the evidence  obtained provides a reasonable basis for our findings and conclusions  based on our audit objectives. Further details on our objectives, scope,  and methodology are provided in appendix II.
subsections: section_title: Background
paragraphs: Treasury is the primary federal agency responsible for the economic and  financial prosperity and security of the United States, and as such is  responsible for a wide range of activities, including advising the President  on economic and financial issues, promoting the President’s growth  agenda, and enhancing corporate governance in financial institutions.
To accomplish its mission, Treasury is organized into departmental offices  and operating bureaus. The departmental offices are primarily responsible  for the formulation of policy and management of the department as a  whole, while the nine operating bureaus—including the Internal Revenue  Service and the Bureau of Public Debt—carry out specific functions  assigned to Treasury. Figure 1 shows the organizational structure of the  department.
subsections: section_title: Treasury Relies on Information Technology to Carry Out Its Mission
paragraphs: Information technology plays a critical role in helping Treasury meet its  mission. For example, the Internal Revenue Service relies on a number of  information systems to process tax returns, account for tax revenues  collected, send bills for taxes owed, issue refunds, assist in the selection of  tax returns for audit, and provide telecommunications services for  business activities, including the public’s toll-free access to tax  information. To assist with delinquent debt collections, Treasury is  engaged in the development of the FedDebt system. In fiscal year 2008,  Treasury plans to spend approximately $3 billion for 234 IT investments— including about $2 billion (about 71 percent) for 60 major investments.
section_title: Prior Reviews of IT Management Issues at Treasury Identified Weaknesses
paragraphs: In 2004, we identified weaknesses in Treasury’s IT investment  management processes. For example, Treasury did not describe or  document work and decision-making processes for agencywide board(s).  Additionally, it did not use the IT asset inventory as part of managerial  decision making. As a result of these and the other identified weaknesses,  we made recommendations to the Secretary of the Treasury to improve  the department’s IT investment management processes.
In 2007, we reported that Treasury had made progress in establishing  many of the practices needed to build an investment foundation and  manage its products as a portfolio. However, we identified additional  investment management weaknesses. Specifically, the department lacked  an executive investment review board that was actively engaged in the  investment management process. As a result of these weaknesses, we  made recommendations to Treasury for strengthening their investment  management capability. In response, Treasury stated that it would take  steps to strengthen its investment board operations and oversight of IT  resources and programs. For example, the department recently  established an executive-level investment review board.
In July 2008, we reported that Treasury’s rebaselining policy fully  addressed one of five practices leading organizations include in their  policies and partially addressed the remaining practices. Since the time of  our review, Treasury has improved its rebaselining policies and  procedures to be more consistent with those of leading organizations.
Several of Treasury’s projects have been deemed to be poorly planned and  managed by the OMB and have warranted inclusion on OMB’s  Management Watch and High Risk Lists. In recent testimony summarizing  our analysis of projects on these lists, we reported that Treasury had 4  projects on the Management Watch List as of July 2008, including one on  the list for the fourth consecutive year. We also reported that the  department had 21 high-risk projects determined to be poorly performing,  most of them because of cost and schedule variances exceeding 10  percent.
section_title: Earned Value Management Provides Insight on Program Cost and Schedule
paragraphs: Pulling together essential cost, schedule, and technical information in a  meaningful, coherent fashion is a challenge for most programs. In addition  to comparing budgeted to actual costs, EVM measures the value of work  accomplished in a given period. This technique compares the earned value  with the planned value of work scheduled and with the actual cost of work  accomplished for that period.
Differences in these values are measured in both cost and schedule  variances. Cost variances compare the earned value of the completed  work with the actual cost of the work performed. For example, if a  contractor completed $5 million worth of work and the work actually cost  $6.7 million, there would be a –$1.7 million cost variance. Schedule  variances are also measured in dollars, but they compare the earned value  of the work completed with the value of work that was expected to be  completed. For example, if a contractor completed $5 million worth of  work at the end of the month but was budgeted to complete $10 million  worth of work, there would be a –$5 million schedule variance. Positive  variances indicate that activities are costing less or are completed ahead  of schedule, whereas negative variances indicate activities are costing  more or are falling behind schedule. These cost and schedule variances  can be used to estimate the cost and time needed to complete a program.
Without knowing the planned cost of completed work (that is, the earned  value), it is difficult to determine a program’s true status. Earned value  provides information necessary for understanding the health of a program;  it provides an objective view of program status. As such, it can alert  program managers to potential problems sooner than expenditures alone  can, thereby reducing the chance and magnitude of cost overruns and  schedule delays. Moreover, EVM directly supports the institutionalization  of key processes for acquiring and developing systems and the ability to  effectively manage investments—areas which are often found to be  inadequate based on our assessments of major IT investments.
Because of the importance of ensuring quality earned value data, in May  1998 the American National Standards Institute (ANSI) and the Electronics  Industries Alliance (EIA) jointly established a national standard for EVM  systems. This standard delineates 32 guidelines on how to establish a  sound EVM system, ensure that the data coming from the system are  reliable, and use the earned value data to manage the program. See  appendix III for details on the 32 guidelines.
section_title: Federal Guidance Calls for Using Earned Value Management to Improve IT Management
paragraphs: In June 2002, OMB’s Circular A-11 included the requirement that agencies  use a performance-based acquisition management system based on the  May 1998 ANSI/EIA Standard to obtain timely information regarding the  progress of capital investments.
This requirement was restated in subsequent versions of the circular and,  in August 2005, OMB issued a memorandum that outlined steps that  agencies must take for all major and high-risk development projects to  better ensure improved execution and performance and to promote more  effective oversight through the implementation of EVM. Specifically, this  guidance directs agencies to:  1.  develop comprehensive policies to ensure that agencies are using EVM  to plan and manage development activities for major IT investments,  2. include a provision and clause in major acquisition contracts or agency  in-house project charters directing the use of an EVM system that is  compliant with the ANSI standard,  3.  provide documentation demonstrating the EVM system complies with  4.  conduct periodic surveillance reviews, and  5.  conduct integrated baseline reviews on individual programs to  finalize the cost, schedule, and performance goals.
Building on OMB’s guidance, in July 2007, we issued an exposure draft on  best practices for estimating and managing program costs. This draft  highlights policies and practices adopted by leading organizations to  implement an effective EVM program. Specifically, the guidance identifies  the need for organizational policies to require clear criteria for which  programs are required to use EVM, compliance with the ANSI standard, a  standard product-oriented structure for defining work products, integrated  baseline reviews, specialized training, criteria and conditions for  rebaselining programs, and an ongoing surveillance function. In addition,  the guidance identifies key practices that individual programs can use to  ensure that they establish a sound EVM system, that the earned value data  are reliable, and that they are used to support decision making. OMB  refers to this guide as a key reference manual for agencies in its 2006  Capital Programming Guide.
section_title: Treasury’s Approach to Earned Value Management
paragraphs: Treasury’s approach to EVM involves several entities, including the Office  of the Chief Information Officer (OCIO), the Office of the Procurement  Executive—both of which are under the Assistant Secretary for  Management and Chief Financial Officer, and Capital Planning and  Investment Control (CPIC) desk officers. Responsibility for the  administration and maintenance of Treasury’s EVM policy lies with the  OCIO. Specifically, the CPIC group within that office supports the  department’s investment management oversight process. CPIC desk  officers are responsible for oversight of one or more bureaus and serve as  the bureau CPIC coordinator’s primary point of contact, responsible for  scoring exhibit 300s and coordinating information sharing with the  departmental budget office and other critical partners. Further, they  develop bureau-level IT portfolio expertise and provide input and  recommendations to the bureaus, Treasury’s CIO, and Treasury’s  Investment Review Board.
Working with the OCIO to identify acquisitions which require earned value  management, the Office of the Procurement Executive is responsible for  ensuring that the identified acquisitions throughout Treasury and its  bureaus contain EVM requirements that are consistent with the Federal  Acquisition Regulation. According to agency officials, 40 investments are  currently using EVM.
Project managers and contractors are required to gather the monthly costs  and progress associated with each of their investments. The information  gathered includes the planned value, actual costs, and earned value. This  information is analyzed and used for corrective actions at the bureau level.  Quarterly, the bureaus forward investment performance reports to the  OCIO’s CPIC office, which reviews them and forwards summaries to  Treasury’s Technical Investment Review Board.
In January 2008, Treasury convened an EVM working group, which has  representation from every bureau. According to the CPIC Director, the  working group has several objectives including establishing (1) level of  reporting for contractors and government employees based on thresholds;  (2) rule sets, processes, and procedures for the development of work  breakdown structures, integrated baselines, standard roll-up into  milestones, and the use of EVM systems at the bureaus; (3) bureau  monthly recordkeeping requirements; (4) standard procedures for  quarterly uploading of data from the bureaus into Treasury’s automated  investment management tool; and (5) requirements for maintaining  documentation to support the project manager validations and bureau CIO  certifications of cost, schedule, and performance data for major and  nonmajor investments. According to the CPIC Director, the working group  is also working on revising the department’s EVM policy.
section_title: Treasury’s EVM Policy Is Not Fully Consistent with Best Practices
paragraphs: While Treasury has established policy to guide its implementation of EVM,  key components of this policy are not fully consistent with best practices.  Without a comprehensive policy, the department risks implementing  policies inconsistently and using inaccurate cost and schedule  performance data.
We recently reported that leading organizations establish EVM policies  that:    establish clear criteria defining which programs are to use EVM;  require programs to comply with a national ANSI standard;  require programs to use a standard structure for defining work  products;  require programs to conduct detailed reviews of expected costs,  schedules, and deliverables (called an integrated baseline review);   require and enforce EVM training;    define when programs may revise cost and schedule baselines (called  require system surveillance—routine validation checks to ensure that  major acquisitions are complying with agency policies and standards.
Table 1 further describes these seven key components of an effective EVM  policy.
In December 2005, Treasury developed the EVM Policy Guide, which  provides an approach for implementing EVM requirements for the  department’s major investments. The policy currently in place fully  addresses three of the seven components, partially addresses three, and  does not address one (see table 2).
Specifically, Treasury has policies and guidance that fully address criteria  for implementing EVM on all major investments, for the conduct of  integrated baseline reviews, and for rebaselining.
Criteria for implementing EVM on all major investments: The  department’s policy requires all of its major development, modernization,  and enhancement investments to use EVM. Investments in steady-state  (i.e., those with no development, modernization, or enhancement  milestones) and those ending prior to September 2007 were not required  by the department to implement the EVM requirement.
Integrated baseline review: In order to verify whether the performance  measurement baseline is realistic and to ensure that the government and  contractor mutually understand program scope, schedule, and risks,  Treasury’s policy calls for an integrated baseline review. According to the  policy, this review should be completed as soon as possible but no later  than 6 months after the contract is awarded. Furthermore, another review  may be required following any significant contract modifications.
Rebaselining criteria: Treasury developed a rebaselining policy which  specifies that a valid reason for requesting a new baseline must be clearly  understood and documented. The policy also specifies acceptable reasons  for an investment team to request a rebaseline. Further, to submit a  rebaseline request, investment teams are required to explain why the  current plan is no longer feasible and develop realistic cost and schedule  estimates for remaining work that has been validated and spread over time  to the new plan.
However, Treasury’s policy and guidance do not fully address the best  practices represented by the following three key components: addressing  compliance with the ANSI standard, defining a meaningful structure for  defining work products, and conducting system surveillance reviews.  Training is not addressed by the Treasury policy.
Compliance with the ANSI standard: Treasury policy states that major  investments are to comply with ANSI standards. Further, it outlines processes  and guidelines to assist its bureau in achieving ANSI-compliant processes.  However, the policy lacks sufficient detail for addressing some of the criteria  defined in the standard, including the use of standard methods for EVM data  collection across the department and cost performance reporting. For  example, the policy does not discuss the use of templates or tools to help  ensure that EVM data are collected consistently and reliably. Furthermore,  the policy does not discuss what cost performance report formats are to be  used. Until Treasury’s policy includes a methodology that standardizes data  collection and reporting, data integrity and reliability may be in jeopardy and  management may not be able to make informed decisions regarding the  investments and its next steps.
Standard structure for defining the work products: Treasury’s EVM  policy calls for a product-oriented work breakdown structure that  identifies and documents all activities associated with the investment.  However, it does not require the use of common elements in its  development. According to the CPIC Director, Treasury’s EVM working  group plans to establish rule sets, processes, and procedures for the  development of work breakdown structures. Until Treasury’s policy  provides more guidance on the systematic development and  documentation of work breakdown structures including the incorporation  of standardized common elements, it will be difficult to ensure that the  entire effort is consistently included in the work structure and that  investments will be planned and managed appropriately.
System surveillance: According to Treasury’s policy, the contractor’s  EVM system is to be validated using the industry surveillance approach  identified by the National Defense Industrial Association’s Surveillance  Guide. Additionally, Treasury is to require clear evidence that the system  continues to remain compliant or that the contractor has brought the  system back into compliance. However, the policy lacks guidance on  conducting surveillance reviews on the government’s (i.e., the  department’s) EVM system. Until Treasury’s policy specifies reviews of the  government’s systems, Treasury risks not being able to effectively manage  cost, schedule, and technical performance of its major investments.
Training requirements: Treasury’s policy does not specify EVM training  requirements for program management team members or senior  executives. Furthermore, the policy does not require the agency to  maintain training logs confirming that all relevant staff have been  appropriately trained. Until the department establishes policy for EVM  training requirements for relevant personnel, it cannot effectively ensure  that its program staff have the appropriate skills to validate and interpret  EVM data and that its executives fully understand the data they are given  in order to ask the right questions and make informed decisions.
According to the CPIC Director, Treasury’s EVM working group, which  was established in January 2008, is working on the development of a  revised EVM policy, which, according to Deputy Assistant Secretary for  Information Systems and Chief Information Officer, is expected to be  finalized by October 2008. Addressing these weaknesses could help  Treasury optimize the effective use of EVM.
section_title: Treasury Is Not Fully Implementing Key Earned Value Management Techniques to Manage Critical System Investments
paragraphs: While the six programs we reviewed were all using EVM, none had fully  implemented any of the practices for establishing a comprehensive EVM  system, ensuring that the data resulting from the system are reliable, or  using earned value data for decision-making purposes. These weaknesses  exist in part because, as previously noted, Treasury’s policy does not fully  address key elements and because the department does not have a  mechanism to enforce its implementation. Until Treasury adequately  implements EVM, it faces an increased risk that some programs will  experience cost and schedule overruns or deliver less capability than  planned.
In our work on best practices, we identified three key management areas  that leading organizations use to manage their acquisitions: establishing a  comprehensive EVM system, ensuring reliable data, and using earned  value data to manage the investment (see table 3).
Table 4 provides a summary of how each investment is using EVM in the  key practices areas and is followed by our analysis of these areas. The  investments we reviewed are the Financial Management Service’s FedDebt  and Financial Information and Reporting Standardization (FIRST); the  Departmental Office’s DC Pension System to Administer Retirement  (STAR); the Bureau of Public Debt’s Treasury Automated Auction  Processing System (TAAPS); and the Internal Revenue Service’s Integrated  Financial System/Core Financial System (IFS) and Enterprise Data Access  Strategy (EDAS). These investments were identified by the department as  major investments and all had milestones in development, modernization,  or enhancement at the time of our review. Appendix II includes  information regarding the selection of these investments and appendix IV  provides a description of each.
subsections: section_title: Investments All Partially Established a Comprehensive Earned Value Management System
paragraphs: Comprehensive EVM systems were not consistently established to manage  the six investments. Although aspects of a comprehensive system were  present, none of the investments fully met all the best practices  comprising this management area.
For example, of the six investments, only IFS and STAR adequately  defined the scope of effort using a work breakdown structure. Three  investments developed a work breakdown structure; however, the work  packages could not be traced back to EVM project management  documents, such as the project management baseline, the work  breakdown structure, and the statement of work or project charter. For  example, although EDAS had detailed work breakdown structures,  correlation could not be established among the work breakdown structure  elements, the contract deliverables, and the elements being reported in the  contract performance reports. Officials for the remaining investment— TAAPS—stated that there was a documented work structure; however,  they did not provide evidence of this.
As another example, performance measurement baselines were developed  for five of six investments. However, the baselines had noted weaknesses.  Specifically, four investments—FIRST, IFS, STAR, and TAAPS—had a  baseline, but some elements were not included, such as planned costs for  STAR. Further, for TAAPS, independent validation of the investment’s  baseline was not conducted. FedDebt had a performance measurement  baseline which underwent integrated baseline validation in March 2006.  However, the validation indicated that there was no time-phased planned  value at the individual contract level, nor was there a roll-up at the  program level. No explanation was provided of how the monthly  performance data on individual FedDebt projects were rolled up to the  investment level as required by OMB. Further, EDAS did not have a time- phased budget baseline or a performance measurement baseline.
section_title: Investments Did Not Fully Implement Steps to Ensure Data Reliability
paragraphs: None of the six investments fully implemented the steps to ensure data  reliability from their EVM systems. Five partially implemented the steps,  and one investment—EDAS—did not meet any of the steps.
When executing work plans and recording actual costs, two of the six  investments incorporated government costs with contractor costs. For  example, FedDebt included both government and contractor costs in their  quarterly reporting. However, while IFS had a mechanism for recording  monthly government costs, it did not have a method that combined both  contractor and government costs for review on a monthly basis. Also, few  if any checks are performed to measure the quality of EVM data and,  according to agency officials, Treasury currently focuses more on  reporting the data than on their reliability.
In addition, five of the six investments did not adequately analyze  performance data and record the variances from the baseline. The IFS  investment included monthly reviews of performance reports and included  cost and schedule variances. The remaining investments conducted  analyses of performance data, but did not all provide documentation to  show cost and schedule updates and variances. For example, according to  officials, TAAPS’ cost and schedule variances were calculated at the  project and program levels, but evidence of this could not be provided.  Further, as part of its performance reporting, STAR did not calculate the  cost variance and incorrectly calculated the schedule variance.
section_title: Most Investments Did Not Fully Implement Practices for Using Earned Value Data for Decision Making
paragraphs: None of the six investments fully implemented the two practices needed to  ensure the use of EVM data for decision-making purposes. Specifically,  EDAS did not take management action to mitigate risks identified through  their EVM performance data, or update the performance measurement  baseline as changes occurred; IFS addressed one of these practices, and  the remaining investments only partially addressed them.
In order to support management action to mitigate risks identified through  EVM performance data—variance analysis, corrective action planning, and  reviewing estimates at completion—the IFS project manager was provided  with monthly performance report that indicated when cost and/or  schedule variances exceeded acceptable tolerances. Further, investment- level status was provided to bureau-level and agency-level management to  allow them to make capital planning and investment control decisions.  However, the remaining five investments did not fully take action to  mitigate risks for a variety of reasons. For example, for FedDebt, although  some monthly EVM data were included in quarterly reports, no  documentation was provided on how such data were being used to  manage at the project or investment level. A similar situation exists for the  TAAPS investment where, although agency officials stated that meetings  were routinely held to discuss performance issues, no evidence was  provided that a systematic method existed to use EVM metrics for  decision-making purposes.
Regarding the update of performance measurement baselines as changes  occur, one investment team stated that it did not have any baseline  changes; however, documentation showed that the schedule for the  investment had been changed three times. In addition, although IFS  maintained a log for tracking changes, we could not determine that these  changes had been incorporated into the baseline. Further, according to  officials, EDAS had a scope change in fiscal year 2007; however, the  investment team was not able to provide documentation reflecting the  corresponding change in the performance measurement baseline.
section_title: Inconsistent Implementation Is Due in Part to Weaknesses in Policy and Level of Oversight
paragraphs: The inconsistent application of EVM across investments exists in part  because the department does not have a policy that fully addresses key  components including training and system surveillance and because the  department is leaving the implementation of the policy largely up to  bureaus. For example, project management staff had not consistently  received training, an item which is not addressed in the policy, and  surveillance reviews, which are partially addressed in the policy, had not  been performed for any of the investments.
Furthermore, the department does not have a process for ensuring  effective EVM implementation. However, in comments on a draft of this  report, the Deputy Assistant Secretary for Information Systems and Chief  Information Officer stated that the department is working with the  bureaus to establish mechanisms and tools to ensure full compliance with  the provisions of the updated EVM policy, which is to be finalized by  October 2008. These mechanisms and tools would help address the  implementation gaps we have identified.
section_title: Conclusions
paragraphs: Treasury has established a policy that addresses criteria for implementing  EVM, integrated baseline reviews, and project rebaselining consistent with  best practices. However, it does not fully address other elements including  compliance with the ANSI standard and system surveillance, which are  necessary for effective implementation. In regards to implementation, the  department is not fully addressing key practices needed to effectively  manage its critical investments. Specifically, none of the six programs we  reviewed were fully implementing any of the practices associated with  establishing a comprehensive EVM system, ensuring the reliability of the  data resulting from the system, or using earned value data to make  decisions. The gaps in implementation are due in part to the weaknesses  with the policy and to the low level of oversight provided by the  department. Until the department defines a comprehensive policy and  establishes a process for ensuring effective EVM implementation, it will be  difficult for Treasury to optimize the effectiveness of EVM as a  management tool and consistently implement the fundamental practices  needed to effectively manage its critical programs.
section_title: Recommendations for Executive Action
paragraphs: To improve Treasury’s ability to effectively implement EVM on its IT  acquisition programs, we recommend that the Secretary of Treasury direct  the Assistant Secretary for Management, in collaboration with the Chief  Information Officer, to take the following nine actions:  Define a comprehensive EVM policy that specifies    a methodology that standardizes EVM data collection and reporting  compliant with the ANSI standard;    a systematic approach to the development and documentation of work  breakdown structures including the incorporation of standardized  common elements;    guidance on conducting surveillance reviews on the government’s EVM   training requirements for relevant personnel.
Implement a process for ensuring effective implementation of EVM  throughout the department by    establishing a comprehensive EVM system by, among other things,    defining the scope of effort using a work breakdown structure that  allows for traceability across EVM project management documents;    ensuring the development of validated performance measurement  baselines that includes planned costs and schedules;    ensuring that the data resulting from the EVM system are reliable,  including    executing the work plan and recording both government and    ensuring that the program management team is using earned value data  for decision-making by    systematically using EVM performance metrics in making the  ongoing monthly decisions required to effectively manage the  investment; and    properly documenting updates to the performance measurement  baseline as changes to the cost and schedule occur.
section_title: Agency Comments and Our Evaluation
paragraphs: In written comments on a draft of this report, the Department of  Treasury’s Deputy Assistant Secretary for Information Systems and Chief  Information Officer generally agreed with our findings and stated that the  department will issue a revised version of the EVM policy that will address  our nine recommendations by October 2008. He also noted that the  department is working with the bureaus to establish mechanisms and tools  including processes for conducting system surveillance and monitoring of  EVM data to ensure compliance with the policy.  Treasury also provided  technical comments which we have addressed as appropriate. Treasury's  written comments are reprinted in appendix I.
We will be sending copies of this report to interested congressional  committees, the Secretary of Treasury, and other interested parties. In  addition, the report will be available at no charge on our Web site at  http://www.gao.gov. If you or your staffs have any questions on matters  discussed in this report, please contact me at (202) 512-9286 or by e-mail at  pownerd@gao.gov. Contact points for our Offices of Congressional  Relations and Public Affairs may be found on the last page of this report.  GAO staff who made major contributions to this report are listed in  appendix V.
section_title: Appendix I: Comments from the Department of Treasury
section_title: Appendix II: Objectives, Scope, and Methodology
paragraphs: Our objectives were to determine whether the Department of the Treasury  and its key component agencies (1) have the policies in place to effectively  implement earned value management (EVM) and (2) are adequately using  EVM techniques to manage critical system investments.
To assess whether Treasury has policies in place to effectively implement  EVM, we analyzed Treasury and its component bureaus’ policies and  guidance that support EVM implementation departmentwide as well as on  capital planning and investment control guidance. Specifically, we  compared these policies and guidance documents to both Office of  Management and Budget requirements and key best practices recognized  within the federal government and industry for the implementation of  EVM. These best practices are contained in an exposure draft version of  our cost guide. We also interviewed key agency officials, including the  Director for Capital Planning and Investment Control, to obtain  information on the agency’s ongoing and future EVM plans.
To determine whether Treasury is adequately using EVM techniques to  manage critical system investments, we reviewed 6 of the 40 systems the  department required to use EVM. Specifically, we selected investments  from each of the four component agencies identified as having eligible  investments. We selected one investment from the Bureau of Public Debt,  another from Departmental Offices, and two from the Financial  Management Service and the Internal Revenue Service since they had a  greater percentage of investments using EVM. With the exception of the  Bureau of Public Debt which had only one major investment, we selected  investments based on (1) size, (2) EVM history (i.e., use of EVM for a long  enough period of time to have some history of EVM data), and (3)  completion date (i.e., those that would not end during the course of our  review). The 6 projects selected were FedDebt and Financial Information  and Reporting Standardization from the Financial Management Service,  DC Pension System to Administer Retirement (STAR) from the  Departmental Offices, Treasury Automated Auction Processing System  (TAAPS) from the Bureau of Public Debt, and Integrated Financial  System/Core Financial System and Enterprise Data Access Strategy from  the Internal Revenue Service. Our review was not intended to be  generalizable, but instead to illustrate the status of a variety of programs.
To determine the extent of each program’s implementation of sound EVM,  we compared program documentation to the 11 fundamental EVM  practices implemented on acquisition programs of leading organizations,  as identified in the Cost Assessment Guide. We determined whether the  program fully implemented, partially implemented, or did not implement  each of the practices. Finally, we interviewed program officials to obtain  clarification on how EVM practices are implemented and how the data are  validated and used for decision-making purposes. Regarding the reliability  of cost data, we did not test the adequacy of agency or contractor cost- accounting systems. Our evaluation of these cost data was based on what  we were told by the agency and the information they could provide.
We conducted this performance audit from August 2007 to July 2008 in  Washington, D.C., in accordance with generally accepted government  auditing standards. Those standards require that we plan and perform the  audit to obtain sufficient, appropriate evidence to provide a reasonable  basis for our findings and conclusions based on our audit objectives. We  believe that the evidence obtained provides a reasonable basis for our  findings and conclusions based on our audit objectives.
section_title: Appendix III: Overview of Industry Guidelines that Support Sound Earned Value Management
paragraphs: Organizations must be able to evaluate the quality of an EVM system in  order to determine the extent to which the cost, schedule, and technical  performance data can be relied on for program management purposes. In  recognition of this, the American National Standards Institute (ANSI) and  the Electronics Industries Alliance (EIA) jointly established a national  standard for EVM systems—ANSI/EIA 748-B (commonly referred to as the  ANSI standard). This standard consists of 32 guidelines addressing  organizational structure; planning, scheduling, and budgeting; accounting  considerations; analysis and management reports; and revisions and data  maintenance. These standards comprise three fundamental management  functions for effectively using EVM: establishing a sound earned value  management system, ensuring that the EVM data are reliable, and using  earned value data for decision-making purposes. Table 5 lists the  management functions and the guidelines.
section_title: Appendix IV: Description of Selected Investments
paragraphs: Below is a description of the six investments we reviewed to assess  whether the department is adequately using EVM techniques to manage  critical system investments.
FedDebt supports the federal government’s delinquent debt collection programs, which were  centralized in the Financial Management Service (FMS) pursuant to the Debt Collection  Improvement Act of 1996. FedDebt also supports Treasury’s strategic goal to manage the  U.S. Government’s finances effectively and the FMS strategic goal to maximize collection of  government delinquent debt by providing efficient and effective centralized debt collection  services. FedDebt plans to integrate the collection services that FMS provides to Federal  Program Agencies through its other programs.
FIRST is intended to automate the maintenance and distribution of the U.S. Standard  General Ledger accounting rules and guidance. It also plans to integrate the general ledger  guidance with the collection of all accounting trail balance data, thus providing a  standardized method of collecting, storing, reporting, and analyzing such data. Furthermore,  the investment is expected to facilitate accounting validations of the agency trial balance data  to provide better feedback to agencies concerning the accuracy and consistency of these  data.
STAR is to assist Treasury and the District of Columbia Government by automating the  determination of eligibility, calculating pension benefits, and delivery of payments, therefore  allowing for (1) increased accuracy of pension benefit calculations and (2) improved  customer service. Key functionality for this investment includes serving annuitants and  survivors of the Judges Pension Plan; making benefit payments to 11,000 teachers, police,  and firefighters who retired before July 1997, as well as their survivors; and automatically  calculating the gross annuity and split benefit payment for teachers, police, and firefighters,  to service those annuitants who retired after June 1997.
TAAPS is intended to ensure that all auction-related operations are carried out flawlessly and  securely. Key among auction activities are the announcement of upcoming Treasury  auctions; bid submission and processing; calculation of awards; publication of results;  creation and dissemination of settlement wires; creation of accounting reports and reports  needed for auctions analysis; and the storage of all securities-, bidder-, and auction-related  information. TAAPS is expected to make numerous intersystem interfaces and manual  processes obsolete by consolidating auction processing requirements into one system and  providing appropriate backup and disaster recovery systems and services.
IFS is intended to operate as the Internal Revenue Services’ (IRS) new accounting system of  record, replacing IRS’s core financial systems, including expenditure controls, accounts  payable, accounts receivable, general ledger, budget formulation, and purchasing controls.  IRS intends to upgrade to software that provides federal accounting functionality. By  migrating to federal accounting practices, IFS is to provide benefits, such as eliminating  current work-around processes, improving project management capability, and enhancing  budget reports.
EDAS is intended to consolidate data from multiple Business Systems Modernization  applications and produce a consolidated data repository source to be used for issue  detection and case selection. The goal is to develop integrated data solutions that allow IRS  to retire duplicative and costly data extracts. The first major project is to develop an  Integrated Production Model as a central repository for corporate data and make those data  available to projects currently in development. Long-term benefits include the retirement of  multiple systems and efficiency gains from improved processes.
section_title: Appendix V: GAO Contact and Staff Acknowledgments
subsections: section_title: Staff Acknowledgments
paragraphs: In addition to the contact named above, Sabine Paul, Assistant Director;  Neil Doherty; Mary D. Fike; Nancy Glover; Sairah R. Ijaz; Rebecca LaPaze;  and Paul B. Middleton made key contributions to this report.